Evaluation of Feature Sets in the Post Processing of Handwritten Pitman's
Total Page:16
File Type:pdf, Size:1020Kb
Evaluation of Feature Sets in the Post Processing of Handwritten Pitman’s Shorthand Swe Myo Htwe, Colin Higgins Graham Leedham, Ma Yang The University of Nottingham, Nanyang Technological University, School of Computer Science and IT, School of Computer Engineering, Wallaton Road, Nottingham, NG8 1BB, UK N4-#2a-32 Nanyang Avenue, Singapore 639798 [email protected], [email protected] [email protected], [email protected] Abstract moving the stylus until a word or a sentence has been fully written) is not conducive to a natural feeling of Innovative ways to rapidly input text becomes essential writing. High-speed text entry is particularly essential for in today’s world of mobile computing. The paper mobile rapid note-takers. Today’s stenographers spend discusses the computer transcription of handwritten time transcribing paper-based shorthand notes because Pitman shorthand as a means of rapid text entry to pen- their handheld devices like Tablet PCs or Personal Digital based computers, particularly from the aspect of Assistants (PDAs) are not productive enough to record linguistic post processing. Feature-to-phoneme speech in real time. It is therefore appropriate to develop conversion is introduced as the first stage of a text- a technique in which text can be written on a digitizer in a interpreter and the application of various production very comfortable and natural way, preferably at the speed rules based on different pattern structures is discussed. It of speech (120 – 180 wpm). demonstrates that phoneme ordering is compulsory in This paper proposes the computer transcription of dictionary-based transcription and the use of an Pitman shorthand as a means to increase the compatibility approximate pattern-matching algorithm resolves the of handheld devices in the real-time reporting industry. problem of recognition confusion between similar With a Pitman shorthand recognizer, users could input patterns. Experimental results are promising and text at an average rate of over 100 words per minute by demonstrate an overall accuracy of 84%. using standard shorthand notations and semantic transcription can be achieved by the use of menus, 1. Introduction approximate phoneme matching, or an automatic collocation analyzer. In this paper, we overview the overall process of the recognition and interpretation of Handheld computing creates an environment where handwritten Pitman shorthand and then introduce an people have both mobility and the ability to send, receive efficient algorithm by which basic primitives of Pitman and process information. Whilst today’s handheld devices shorthand such as loops, circles, strokes or hooks are have transformed into powerful pocket-sized computers, interpreted into orthographic English. the transformation of a standard “QWERTY” keyboard into these handheld devices has not been so effective. 1.1. Background Miniature keyboards make text entry very slow (less than 10 words per minute (wpm) [1]. Handwriting recognition Pitman shorthand was invented by Sir Issac Pitman in systems like Unistroke and Graffiti are alternative means 1873. There are two main forms: - classic New Era and of text input to pen-based computers, but writing New Pitman 2000. Although both of them are based on individual characters on a smooth digitizer still results in the same principles, New Era notation is slightly faster to slow text input of less than 10wpm. Other rapid write but more difficult to learn than the Pitman 2000. It handwriting input methods like, T-Cube [2] and was widely used in offices in the UK and also taught in 74 Quickwriting [3], allow a user to compose entire words or other countries [4]. Pitman’s shorthand records speech even sentences as a single outline. However their phonetically and it comprises phonemes of 24 constants, restrictions towards gestures (e.g. a user must never stop 12 vowels, and 4 diphthongs. It also defines Proceedings of the 9th Int’l Workshop on Frontiers in Handwriting Recognition (IWFHR-9 2004) 0-7695-2187-8/04 $20.00 © 2004 IEEE approximately 90 of the most frequently used English primitives into related phonemes. We refer to this process words as short-forms i.e., special signs written without as “Feature to Phoneme Conversion (FtoP)”. In an early vowel components. Basic notations of Pitman shorthand part of “FtoP”, we use an approximate pattern-matching and sample outlines are illustrated in Figure 1. algorithm in which user-dependent stroke variations or wrong vowel allocations are adjusted. The output of “FtoP” is a ranked list of phonemes and they are later discriminated by a phonetic dictionary and matched to a list of homophones at the word level transcription. In sentence level transcription, the homophones are passed through a collocation analyzer, where each word is justified by its frequent use with another word or phrase and the most probable word is selected by the system i.e., “Internet” in the following example (Figure 2). Figure 1. Samples of Pitman shorthand outlines The potential of Pitman shorthand [5][6] as a means of rapid pen-driven text entry to a computer has been reported since the 1980’s. Subsequent works by Brooks et al [7][8] introduced the computer transcription of Pitman’s shorthand as an aid for the deaf by providing a real-time transcript of lectures and meetings. Research in the 1990’s[9][10] classified the whole Pitman shorthand into two types: vocalized outlines and shortforms and applied different AI methods in their recognition and transcription. Another recent approach by Nagabhushan et al [11][12] concentrated on the post processing of segmented basic features such as loops or hooks being interpreted to English text using heuristic methods. 2. Overview of Our System Shorthand outlines are fed into the recognition engine as shown in Figure 2 and differentiated between a vocalized outline and a short-form. Short-forms are Figure 2. Illustration of the computer recognized separately from vocalized outlines using a transcription of Pitman shorthand Template Matching Algorithm in which a ranked list of English words is produced. Therefore, the transcription 3. Transcription system no longer needs to be concerned with short-forms at the word level transcription. As for vocalized outlines, 3.1. Approximate pattern matching the recognizer segments them into basic primitives i.e., strokes, hooks, circles or loops using dominant point Handwritten outlines are bound to differ in angular information as a threshold value. Then, the segmented structures from writer to writer and even from the same data are matched up with the features of basic consonants writer from time to time. Another problem in the and categorized into a ranked list of phoneme primitives transcription of handwritten Pitman shorthand is that using a neural network classifier. Due to the difficulty of vowels are not written clearly between dot and dash and accurately detecting pen pressure on normal digitizers, neither are they put at an accurate position i.e., at the phonemes with different line thickness like “P” and beginning, middle or end of an outline. In the pre- “B” are clustered as the same type. In some cases, the processing, the recognition engine detects a ranked list of classifier is unable to classify a single primitive as a candidates for individual primitives, but it cannot cope whole consonant outline, for example, outlines of “W” with user-dependent or accidental pen strokes. For and “Y” are classified separately as a small hook example, if the “T” consonant is accidentally written as a ( or ) and a stroke written upwards . Therefore, an vertical curve instead of a vertical straight stroke , the additional step is needed to correctly interpret clustered recognizer does not estimate the curve as a vertical stroke. Proceedings of the 9th Int’l Workshop on Frontiers in Handwriting Recognition (IWFHR-9 2004) 0-7695-2187-8/04 $20.00 © 2004 IEEE Approximate pattern matching is, in fact, a heuristic vowel primitives. Samples of neighborhoods are shown approach in which uncertain pen strokes are coped with in Figure 3 and the use of approximate pattern matching and wrong vowel locations are estimated. It uses the algorithm is illustrated in Figure 4. nearest neighbor query and the heuristic is based on the similarity between the two patterns. 3.2 Features to Phonemes Conversion Function Name : Return value APPROXIMATE_PATTERN_MATCH(pattern) : Once a vocalized outline has been classified into A list of similar primitives pattern primitives of hooks, circles and strokes, it is Begin necessary to convert these feature primitives into a pattern: a sample input pattern phonetic representation. In fact, around 20% of the pattern Heu-Fn : a heuristic function primitives can be directly mapped to basic consonants, Return NEAREST-NEIGHBOR-QUERY(pattern, and the remaining 80% of the primitives need to be Heu-Fn) translated by the application of production rules of Pitman End shorthand. Similar to the work of Leedham and Downton The heuristic function (Heu-Fn) can be defined as [4], the production rules are applied with respect to the H (p) = nearest neighbor primitives which are neighbours stroke primitives of the consonant kernel, and their similar to pattern ‘p’ adjacent primitives. Primitives with similar structure are defined as the Basically, there are five production rules introduced in nearest neighbors and the whole Pitman primitives are our system: - “Direct Translation (DT)”, “Primitive located to seven neighborhoods where, four of them are Combination (PC)”,