Navigate / search

Human computer interface: digitising your handwritten prose

Like pretty much every writer nowadays, I do most of my writing via a keyboard, whether that’s on my laptop, on the bluetooth keyboard tethered to my iPad, or (occasionally) using the software keyboard on my iPhone. However anything big enough to comfortably touch-type on is also too big to slip into my everyday shoulder bag, so I’ve been looking for alternative solutions. There’s also the issue that whilst I’ve long since become accustomed to writing the stories themselves on a keyboard, I still prefer to do background note-taking (world-building, plot brainstorming, etc) in longhand; it just feels more natural and organic. Unfortunately this means I end up with a lot of paper notebooks, which are difficult to search through!

There seem to be two major strategies for solving this problem: digitising your handwriting as you do it, and digitising the image of the handwritten text using OCR. Either way, the usefulness of the end result depends a great deal on both how neat your handwriting is, and how good the software is at recognition. If humans can barely read your handwriting, a computer isn’t going to have a snowball’s chance in Hell—and a garbled file full of nonsense words mixed with random characters is unlikely to be of much use to you.

With that caveat, here are my thoughts on the various hardware and software solutions I’ve tried.

On-screen handwriting recognition

This is the type of digitisation I was most familiar with for years. As a long-time user of the Palm series of PDAs, I became fluent in Graffiti, their stylised “handwriting” that allowed direct digital input using letters handwritten on the device’s screen with a stylus. As a result, I’ve spent a long time looking for an equivalent for iOS (I have several styli, as they come in handy in cold weather when I need to wear gloves), and I have to say that I’m deeply disappointed. All the apps I’ve tested assume that you’ll want to use your normal handwriting, and so they put a lot of processing power into full handwriting recognition, which makes the app painfully slow even on an iPhone 4S. I’ve yet to find one that, like Graffiti, expects you to learn a simplified alphabet which the computer can easily recognise, which is highly frustrating for me. If you know of such an app, please, please let me know!

Optical character recognition (OCR)

OCR has been around for quite a while, and is often used for digitising printed books that were never released in an electronic version. As software has become more powerful, however, it is now possible to digitise handwritten text as well.

The traditional method is to scan a document page-by-page using either a flat-bed scanner (necessary if your pages are bound into a book) or a more compact feed-through scanner. The latter takes up less desk space, but if you’re like me and mostly write in bound notebooks, a flat-bed scanner is your only option. Or rather it was, until very recently. The advent of smartphones with relatively high resolution cameras means you effectively have a portable scanner in your pocket—a fact that has now been exploited by popular note-taking app Evernote. Using the Page Camera option (available on the Add Note screen), you can take snaps of your notebook pages and slurp them into Evernote. They are even teaming up with Moleskine to create “smart notebooks” that make scanning more accurate. Naturally these don’t come cheap, but they might make a nice addition to your Christmas list!

The desktop version of Evernote also has basic handwriting OCR built in, so you can search the images of your notebook pages, but at the moment it doesn’t offer full digital conversion of text. Hence it’s no use for content that you need to put into a word processor, so you can’t handwrite your novel and then use Evernote to transfer it into, say, Scrivener. Also, I’ve tried using Evernote and Page Camera on my project notebook, but its handwriting recognition isn’t all that great unless you write very neatly, which I tend not to do in the heat of inspiration!

Finally, large high-resolution images use up a lot of bandwidth, which means that if you have more than a few notes, you’ll be obliged to pay for Evernote’s premium service (they cunningly include 3 months’ free subscription with the smart notebook) and presumably also consume more bandwidth on your mobile devices. If Evernote introduces the ability to do reasonably accurate OCR on files and then archive the actual images, I think it might be worth scanning in more of my notes, but right now it’s not a complete solution.


This is the latest, most high-tech solution: you write on real paper with an electronic pen, and the image produced by your writing movements can be uploaded to your computer and then run through OCR software to produce a digital version. There are two main kinds of smartpen: one requires special paper marked with a grid of dots, the other uses normal paper but relies on a receiver unit to detect the pen’s movements. I own one of the latter type, an IRISnotes Executive, and I have to say that it’s pretty nifty. I’ll review it in detail next week, but so far I’m impressed by its handwriting recognition capabilities, at least when it comes to non-fiction. Of course smartpens are pretty expensive (prices start around $100) and require more practice to get right than using a normal pen and paper and then scanning the page. On the other hand they’re lightweight and don’t require you to buy (or find room on your desk for) a scanner.


In summary, there are now a number of ways to get your handwritten text into your computer, but there’s no one-size-fits-all solution. Still, they offer a potential bridge between paper and screen for those of us who still enjoy writing the old-fashioned way!


Larry Kollar

I do my share of writing into Evernote on my iPhone, usually with a Bluetooth keyboard. In fact, I did 1000 words or more at lunch today that way. And sometimes, I prefer the notebook and pen. Switching between the two can help me from getting bogged down.

It’s really too bad we’ve been conditioned to expect everything to happen right away. The original Apple Newton got dinged pretty hard for its handwriting recognition doing funky things, for example. The thing is, if you worked with its “training” program for a couple weeks, it eventually figured out your handwriting and accuracy would go from 30% to 90% all at once. One time, I scrawled something on my Newton that I couldn’t even read, but it figured it out.

Then again, maybe it’s best that we don’t have easy handwriting OCR. During my vacation last month, I spent a lot of time with a pen and notebook, just because I didn’t feel like opening the laptop, and ended up writing about 10,000 words (mostly in the mornings). When I got home and started typing it all in, I wound up fixing a lot of stuff as I typed. If I’d just let the computer slurp it in, I might not have looked at it so carefully.

I’ll have to look into those smartpens, though.