Back in April 2011 I went to a Birds of the Feather event sponsored by BrightIdea, see here for my posting on that event. At the event one of the examples of an innovation was the use of digital pen technology to collect information on forms by workers in the field. I didn’t know what they meant by digital pens but soon learned that it was a combination of a pen and some special paper. The pen and paper let normal handwriting be captured and transferred to a digital format without any scanning.
I was intrigued because this is a particular pain point in many explicit knowledge management/document management projects I’ve worked on over the years. Trying to convince people to change from handwritten forms to online input forms is fraught with hardship and resistance. The arguments pro and con usually go something like this:
– The handwritten form is easy to use, can be signed for authorization and can then be retained as an authenticated final copy.
– The online form is difficult to use, requires a computer, a printer and a scanner and a whole lot of training.
– The handwritten form is difficult to share. We have to scan it and input the data manually and then verify the input. This slows down the workflow.
– We must have a printed and signed form for compliance purposes. We want a true, unchangeable authenticated copy so we will have to print the form, sign it and then scan it.
The core digital pen technology comes from the Swedish company Anoto. The technology has been licensed to other companies. For example, PaperIQ Digital Pen for Blackberry solution, and LiveScribe that produces digital pens combined with digital audio recorders.
In June 2011, I got in touch with Big Prairie Limited a local Hong Kong company that provide consulting services for Anoto based digital pens technology. Big Prairie has been working in the digital pen area in Asia for about ten years.
In exchange for helping out at the Big Prairie booth at the Greater China eHealth Forum 2011 on 7-8 October I was given a LiveScribe Echo 4GB digital pen. I didn’t agree to blog about the digital pen technology and no one from Big Prairie has reviewed this blog. Just to be clear, I got 1 pen, 3 notebooks and some software with a retail value of about HKD1,700 (about USD220) in exchange for 2 days of work at the forum. Clearly, I’m really a cheap date.
I prepared some demos and for two days I demonstrated the LiveScribe Echo digital pen. Subsequently, I’ve been using the pen almost every day and showing it around to my network. People are very impressed with the technology. The pen is a normal ball-point pen with a small camera and a digital recorder and player. The pen is lightweight and a bit big but still comfortable for writing. The pen can be used as just a normal ball-point pen or used with the Anoto dot paper to record what you are writing. The notebook is made from normal notebook paper which as been printed with thousands of very small blue-colored dots. These dots are locational markers; when combined with the camera your writing is recorded precisely and accurately. The digital recorder can be used stand-alone or it synchronizes with your handwritten notes. The notebook has printed commands at the bottom of the page that let you turn on, pause, turn off the audio recording, jump ahead in 10 second intervals, bookmark, jump to a % position, set the playback speed and adjust the volume of the playback. The small speaker on the pen is good enough for listening or you can buy some headphones that plug into the pen for both better listening quality and recording quality. You can play back an audio recording by simply touching any of the synchronized handwritten text.
I don’t want to go over all of the features of the LiveScribe pen because that information is easily available and better explained on the LiveScribe website.
The handwritten notes and recording can be easily exported to Adobe pdf format. This is called a pencast. I’ve made some pencast examples that illustrate what I think are the important points. Download these pencasts and open with Acrobat Reader version 9.3.2 or higher.
1. A pencast with handwritten notes and audio recording by listening to the BBC One-minute news broadcast. Pencast no 1 ….
– The black text has no synchronized recording.
– The green text has synchronized recording.
– Click on the green text to listen to the recording. The remaining green text will go light grey. You can jump around by clicking on the green or light grey text.
2. A pencast with handwritten notes, audio and a quick sketch. Added a few days’ later handwritten notes and audio recording from a BBC One-minute news broadcast. Pencast no 2 ….
– It captures the sketch easily and accurately.
– Notes and recordings can be added at different times.
3. A pencast of “Double Ninth: Missing my Shandong Brothers”, with handwritten Chinese characters, Putonghua pinyin and English translation. There is no audio recording. This is a poem that almost all Chinese children learn in school. Pencast no 3 ….
As a lonely stranger in a strange land,
At every holiday my homesickness increases.
Far away, I know my brothers have reached the peak;
They are planting flowers, but one is not present.
“Double Ninth, Missing My Shandong Brothers”
— Wang Wei (王維), Tang Dynasty
4. A pencast with handwritten Japanese kana and kanji. Pencast no 4 …
– Kana means katakana and hiragana writing. Kanji mean Chinese characters.
Handwritten notes can be converted to text using the MyScriptforLiveScribe tool from a company named VisionObjects . It costs about USD30.00. It is very easy to use from within the LiveScribe Desktop.
Here are the results.
1. A pencast with handwritten notes and audio recording by listening to the BBC One-minute news broadcast converted to text and then saved as a JPG. Pencast no 5 …
– As a JPG file it retains the layout and format of the handwritten notes.
2. A pencast with handwritten notes and audio recording by listening to the BBC One-minute news broadcast converted to text and then saved as a DOCX. Pencast no 6 ….
– It loses the layout and the format.
– The text can be edited.
– The text has 401 written letters and 10 errors marked in red so the error rate was 2.49%. This is remarkable for conversion of handwritten text to type text. I did a project last year where scanning handwriting and then converting to type text the error rates was between 25% to 50%. See here for information on ICR, intelligent character recognition.
3. A pencast of “Double Ninth: Missing my Shandong Brothers”, Chinese characters, Putonghua pinyin and English translation. I’ve converted the Chinese characters to text, the Putonghua pinyin to text and the English translation to text and saved as a DOCX. Pencast no 7 ….
– There are 37 characters in the original text plus 2 characters of the author’s name. There are 6 errors marked in red. The error rate is 15.38%. Once again, this is remarkable for handwritten Chinese character to type text conversion.
– There are 199 handwritten letters and diacritic marks in the Putonghua pinyin. There are 29 errors marked in red. The error rate is 19.33%. It is high because MyScriptforLiveScribe doesn’t have a dictionary for Putonghua pinyin.
– There are 188 handwritten letters and punctuation marks in the English translation. There are 15 errors marked in red. The error rate is 7.98%.
4. A pencast of Japanese kana and kanji. This text only has hiragana and kanji characters converted to characters and then saved as a DOCX. Pencast no 8 ….
– 65 handwritten hiragana and kanji.
– 3 errors marked in red. The error rate is 4.62%. Once again, this is remarkable. Projects in Japan I’ve worked on with handwritten Japanese to text conversion are lucky to get the error rate down to 25%.
The LiveScribe solution is for personal note-taking. It is most useful for students, business people, professionals such as lawyers, doctors, advisors and anyone who takes notes which is just about anybody. Being able to record the audio along with the notes can significantly increase the value of the notes and can improve your ability to add to the notes later. Being able to create a PDF pencast makes the notes easy to share with others. Being able to convert them into type text makes it much easier to produce formal reports from handwritten notes. I recommend it highly for an effective and inexpensive solution for taking notes and keeping them digitally.
In Hong Kong, information on ordering a LiveScribe digital can be found on the Smartpen Asia website. The LiveScribe website has an international store locator. LiveScribe pens and accessories can also be ordered thru Amazon.
There are enterprise solutions enabling forms to be created and specialized workflows integrated to enterprise backend solutions using the Anoto digital pen technology. You should contact Big Prairie Limited or Anoto for more information.