EUROGRAPHICS Symposium on Sketch-Based Interfaces and Modeling (2012) L. B. Kara and K. Singh (Editors) I Don‘t Believe My Eyes! Geometric Sketch Recognition for a Computer Art Tutorial D. Cummmings & F. Vides & T. Hammond1 1Department of Computer Science & Engineering Texas A&M University, College Station, TX, USA Abstract Drawing is a common form of communication and a means of artistic expression. Many of us believe that the ability to draw accurate representations of objects is a skill that either comes naturally or is the result of hours of study or practice or both. As a result many people become intimidated when confronted with the task of drawing. Many books and websites have been developed to teach people step-by-step skills to draw various objects, but they lack the live feedback of a human examiner. We designed EyeSeeYou, a sketch recognition system that teaches users to draw eyes using a simple drawing technique. The system automatically evaluates the freehand drawn sketch of an eye at various stages during creation. We conducted frequent evaluations of the system in order to take an iterative development approach based on user feedback. Our system balances the flexibility of free-hand drawing with step-by-step instructions and realtime assessment. It also provides rigorous feedback to create a constructive learning environment to aid the user in improving her drawing. This paper describes the implementation details of the sketch recognition system. A similar implementation method could be used to provide sketching tutorials for a wide number of images. Categories and Subject Descriptors (according to ACM CCS): Computer Applications [J.5]: ARTS AND HUMANITIES—Fine Arts; Software [D.2.2]: SOFTWARE ENGINEERING—Design Tools and TechniquesUser Interfaces 1. Introduction determine the accuracy of what they are drawing, and then use this information to provide realtime interactive feedback. The presence of art and artistic expression can serve as an in- We have decided to explore this theory by focusing on how fluence to who we are as scientists and researchers [FK95]. to draw one of humans’ most expressive features: the eye. Art can be seen as a form of communication and a means Identifying basic structures and shapes within life and of expressing one’s interpretation of the world. We all aren’t recreating those shapes through various methods is the ba- born with the skill or have the time to learn how to accu- sis of most technical drawing instruction [Par79, WP88]. rately interpret and reproduce what we see around us through A proper representation of these structures falls under the drawing. While the task of learning how to draw perfect drawing approach taught by most art teachers and sum- reproductions of all what we see may be improbable on a marized by the phrase "draw what you see, not what you large scale and within a small time frame, it may be possi- know" [Edw99, Mar08]. The meaning of this direction is to ble on a much smaller scale and with the aid of technology. overcome the urge to draw the shape that the mind deter- Numerous books and websites have been written to explain mines is correct and to instead draw the shape that is actually in detail how to draw various objects through step-by-step seen. drawing tutorials [Mas04, Smi07]. As sketch recognition al- gorithms and technologies become more sophisticated, it is A novice artist can reference an extensive collection of now possible for a computer to watch a human draw, use videos and how-to literature focused on improving artistic sketch recognition techniques to identify what the human is skills. Many manuals have been written focusing on a similar trying to draw, apply difference classification algorithms to approach of breaking down drawing techniques into steps c The Eurographics Association 2012. DOI: 10.2312/SBM/SBM12/097-106 98 D. Cummmings & F. Vides & T. Hammond / EG LATEX Author Guidelines Figure 1: EyeSeeYou interface Figure 3: Eye portraits encased in small jewelry, such as pins and rings, were popular in the 18th century, often used to mourn loved ones. Cosway, The Eye of Mrs. Fitzherbert, ca. 1786. (Photographic Survey, Courtauld Institue of Art). The EyeSeeYou application uses a geometric-based ap- proach to recognition; it uses a low-level recognizer called PaleoSketch [PH08] to identify primitive shapes such as lines, arcs, ellipses, spirals, etc. The system uses a set of guidelines and constraints to verify that the user draws el- ements of the eye proportionally and in correct relation to Figure 2: Sample eye drawn with the EyeSeeYou system. each other. If the procedure is followed correctly, then the result should be a structurally correct eye that can then be enhanced similar to the one shown in Figure 2. that can be self-taught [DeR96, Par03]. However, as is often The main challenge in the creation of the system was to the case with how-to drawing methods, the final result is only find a balance of constraints that would allow the system as good as your ability to accurately recreate the progressive to deliver constructive feedback while at the same time pre- images. venting user frustration from over-correction. In order to ac- complish this task we attempted to create a systematic ap- Without direct observation and guidance from an instruc- proach to drawing the eye by breaking it down into simple tor, a novice artists may find it difficult to realize when steps that can be completed fairly quickly and with reinforc- they’re making a mistake; when they do realize, there is usu- ing feedback that both helps the user to identify where his ally no guide to show them how to get back on track. In or her drawing could use improvement. We also wanted to order to address this problem we took a simple technique for reinforce progress made toward the production of a realistic drawing an eye and developed a system that would provide eye with positive feedback. feedback on how well the user follows this technique; this system is called EyeSeeYou (Figure 1). An initial challenge was to determine how strict the con- straints for the drawing task had to be. If they were too loose EyeSeeYou is a pen-based system that uses sketch recog- the final result may not be sufficiently realistic, but if they nition to provide helpful feedback to users based on their were too strict they may have an adverse affect on the user drawing. As no two freehand drawn sketches are alike, accu- user experience as it may become too difficult to advance at rately recognizing hand-drawn sketches is difficult. By walk- each step. ing the user through a step-by-step drawing process similar to those found in paper-based drawing tutorials, the system An additional challenge involved trying to guide the artist is able to remove ambiguity in terms of the user’s intention, while still allowing a level of sketching freedom so as not to and provide informative and useful feedback. The system al- impede the creative process. Our intention was not to force lows users to sketch their drawing as they would with paper, user to create exact reproductions of the example image that enabling the user to learn a particular technique defined in is given, but instead, we want to provide them with the tech- a series of steps, while making sure the user follows the in- niques to create their own image of an eye that is proportion- structions at each step, so they can later reproduce the same ally accurate. The images that we obtained after evaluating technique in their drawings. our software show that although the eyes drawn by each of c The Eurographics Association 2012. D. Cummmings & F. Vides & T. Hammond / EG LATEX Author Guidelines 99 Figure 4: Two very different, but correctly shaded eyes in EyeSeeYou the participants may be similar, the level of detail and indi- vidual features are still unique. Figure 5: Wacom monitor with pen input 2. Previous Work The significance of the eye in communication can be seen through popular art forms of the 18th century [Fer75]. A els as a guide to get users started. As they begin to add method of portraiture emerged that consisted of small minia- strokes, the system prompts them with suggestions of 3D ture paintings of the eye [Gro06]. These unusual portraits parts that can be added to the model. These parts are aug- were often mounted on pins, brooches, rings and other keep- mented to the existing model thereby facilitating the drawing sake items and exchanged between friends, family and lovers process. This technique is very powerful and allows creative as a sign of affection. In some cases, eye portraits of the de- drawing. However the recognition of the sketch being drawn ceased (called mourning jewelry) were kept in remembrance is very limited or in existent which inhibits the ability of giv- of loved ones. An example is shown in Figure 3. Although ing meaningful feedback to the user. short-lived, the popularity of this type of portrait gives us While systems like these help the user by augmenting insight into the importance the eye has in conveying spe- their drawings, they do not provide feedback that would help cific meaning relating to both the subject (the owner of the the user create better drawings on their own. In order to pro- eye) and the viewer. It’s not the still image of the eye, but vide this extra level of interaction, sketch recognition was more importantly, the gaze that holds meaning and context used to create predictive and suggestive interfaces that guide for the viewer.