The Virtual Nose: a 3-Dimensional Virtual Reality Model of the Human

ORIGINAL ARTICLE The Virtual Nose A 3-Dimensional Virtual Reality Model of the Human Nose

A. John Vartanian, MD; Joi Holcomb, MA; Zhuming Ai, PhD; Mary Rasmussen, MA; M. Eugene Tardy, MD; J. Regan Thomas, MD

Background: The 3-dimensionally complex interplay Results: Anatomically correct 3-D model of the nose of soft tissue, cartilaginous, and bony elements makes the was produced. High-resolution images of the “virtual mastery of nasal anatomy difficult. Conventional meth- nose” demonstrate the nasal septum, lower lateral car- ods of learning nasal anatomy exist, but they often in- tilages, middle vault, bony dorsum, and other struc- volve a steep learning curve. Computerized models and tural details of the nose. Also, the model can be com- virtual reality applications have been used to facilitate bined with a separate virtual reality model of the face teaching in a number of other complex anatomical re- and its skin cover as well as the skull. The user can gions, such as the human temporal bone and pelvic floor. manipulate the model in space, examine 3-D anatomi- We present a 3-dimensional (3-D) virtual reality model cal relationships, and fade superficial structures to of the human nose. reveal deeper ones.

Methods: Human cadaveric axial cross-sectional Conclusions: The virtual nose is a 3-D virtual reality (0.33-mm cuts) photographic data of the head and model of the nose that is accurate and easy to use. It can neck were used. With 460 digitized images, individual be run on a personal computer or in a specialized vir- structures were traced and programmed to create a tual reality environment. It can serve as an effective teach- computerized polygonal model of the nose. Further ing tool. As the first virtual reality model of the nose, it refinements to this model were made using a number establishes a virtual reality platform from which future of specialized computer programs. This 3-D computer applications can be launched. model of the nose was then programmed to operate as a virtual reality model. Arch Facial Plast Surg. 2004;6:328-333

HE DECEPTIVELY SIMPLE EX- intraoperative observations. Many out- ternal shape of the human standing textbooks and nasal anatomy at- nose belies the complexity lases exist, and are usually the first edu- and individual variability of cational tools used in teaching anatomy.1-4 its underlying anatomy. Nevertheless, textbooks can be limited by TheT shape of human nose is formed by the the artist’s or photographer’s perspec- interplay of osseous, cartilaginous, and soft tive. Even texts with the highest level of tissue elements. The manipulation of this accuracy and image quality are limited by structural scaffolding is the primary mo- the 2-dimensional nature of printed ma- dality used to effect change in rhino- terial, which cannot impart 3-dimension- plasty. As such, an accurate and clear un- ality. Video material is similarly limited by derstanding of the anatomy of the human not being able to deliver a 3-dimensional From the Division of Facial nose is a prerequisite for the rhinoplasty (3-D) experience. Another limitation of Plastic and Reconstructive surgeon. A firm grasp of nasal anatomy most traditional methods of learning Surgery, Department of early in a rhinoplasty surgeon’s career can anatomy relates to their distinct lack of any Otolaryngology–Head and minimize negative consequences of mas- interactivity and feedback. Neck Surgery (Drs Vartanian, tering the “learning curve.” As a supplement to knowledge gained Tardy, and Thomas), and the Traditional methods of learning from books and observation, cadaver dis- School of Biomedical and anatomy have helped generations of sur- sections can play an important role in the Health Information Sciences (Mss Holcomb and Rasmussen geons familiarize themselves with nasal mastery of anatomy. Unfortunately, lim- and Dr Ai), University of anatomy. These learning modalities have ited availability and high cost limit the use Illinois at Chicago; and the included anatomy atlases, textbooks or lit- of cadavers in most teaching institutions. Lasky Clinic, Beverly Hills, erature descriptions, cadaver dissections, Even when cadavers are available, learn- Calif (Dr Vartanian). lectures by rhinoplasty experts, videos, and ing from cadaver dissection is limited by

(REPRINTED) ARCH FACIAL PLAST SURG/ VOL 6, SEP/OCT 2004 WWW.ARCHFACIAL.COM 328

©2004 American Medical Association. All rights reserved. Downloaded From: https://jamanetwork.com/ on 09/27/2021 the poorly delineated anatomical structures, by dissection planes that have been altered by chemical preserva- tion, and by the relatively brief time (typically) spent in the dissection laboratory. For most surgeons, the mastery of 3-D nasal anatomy is still largely dependent on extensive intraoperative experience. Observation during rhinoplasty is an effective way for many beginning rhinoplasty surgeons to learn nasal anatomy. Clearly, there is no substitute for the knowledge that is gained after hav- ing performed a large number of rhinoplasties through dissection, tactile appreciation, and manipulation of nasal structures. It is precisely this intimate understanding of nasal anatomy that the physician-in- training or the beginning rhinoplasty surgeon may be lacking. In such cases, virtual reality teaching tools aimed at teaching 3-D nasal anatomy with interactive capabilities could be particularly useful. Computerized anatomical models and virtual reality applications have been used to facilitate teaching in Figure 1. On axial cross sections, individual structures are traced and a number of other complex anatomical regions.5-7 Vir- labeled in a process called modeling and segmentation. tual reality models of the human temporal bone and the pelvic floor have already demonstrated the great poten- initial models were made using advanced 3-D modeling soft- tial of such teaching tools at our institution. In line with ware applications. our desire to create a 3-D virtual reality model of the hu- The surrounding bony craniofacial structures were pro- man nose and its surrounding structures, we set out to duced in a separate project and made compatible with the virtual nose for added functionality. Further refinements to the produce the “virtual nose.” models were made using computer modeling programs to im- prove surface detail. Small variations in scale were adjusted be- METHODS fore our 3-D nasal model was combined with our 3-D skull model. This final computer model of the nose was then pro- DATA ACQUISITION grammed to operate as a virtual reality application. The models were incorporated into the virtual reality en- Our goal was to create an accurate virtual reality model based vironment using an original set of modular software program- on anatomical data and observations. We based our model on ming tools.8 A number of computing tools were utilized to en- a freshly frozen female cadaver from the Visible Human Project able our model to function in a virtual classroom with (National Library of Medicine, Bethesda, Md). The head and audiovisual interaction and feedback. The application was pro- neck of the cadaver had been previously cut into fine (0.33- grammed on a commercially available graphics supercom- mm) axial cross sections and photographed. These high- puter (Silicon Graphics Onyx2; Silicon Graphics Inc, Moun- resolution color photographs had been digitized and were avail- tain View, Calif). A second set of final models was also simplified able as a set of consecutive images. Several other cadaver sources for viewing on personal computers as virtual reality modeling were closely evaluated. We chose to use this set of cadaver axial language–based models. cross sections because it provided (1) thin cross sections (0.33 mm), (2) frozen cadaver sections with minimal tissue distor- RESULTS tion, (3) high-resolution images of the cross sections, (4) cor- responding computed tomographic data from the cadaver’s skull A 3-D virtual reality model of the human nose and sur- and cervical spine, and (5) relative ease of visualization of dif- rounding structures was created. The virtual nose dem- ferent structures on cross-sectional analysis. Our data set con- onstrates the following anatomical structures: lower lat- sisted of 460 consecutive axial cross sections (0.33-mm cuts) eral cartilages, upper lateral cartilages, osseous nasal vault, of the midface and nose. nasal septum, and overlying skin cover (Figure 2 and PREPARATION OF DATA Figure 3). Also, the compatibility of the virtual nose with a separately produced craniofacial model allows the user The basic principle used involved combining hundreds of de- to inspect the skull and cervical spine.9 The virtual nose tailed and properly aligned 2-dimensional anatomical cross sec- is most functional when it is viewed in a virtual reality tions to generate a polygonal 3-D model. Each image was traced environment. The model was also programmed to run using computer software to outline the structures of interest. on a personal computer and displayed on a high- Individual structures were traced and given structure-specific resolution computer monitor. color codes at each axial level (n=460) (Figure 1). VIRTUAL REALITY VIEWING MODEL CREATION

The set of “tracings” were saved as a set of images that were Viewing the model in a 3-D virtual reality environment stacked, combined, and used to create 3-D models. Using a spe- greatly enhances the model’s interactivity and useful- cialized computer, several 3-D polygonal models of nasal and ness as a teaching tool. The virtual reality images can be midface structures were produced. Further refinements to these generated a number of ways. We used a virtual reality

(REPRINTED) ARCH FACIAL PLAST SURG/ VOL 6, SEP/OCT 2004 WWW.ARCHFACIAL.COM 329

C D

Figure 2. Different views of the cartilaginous nasal structures in the “virtual nose.”

display system called an ImmersaDesk (Board of Trust- movement around the 3-D model to generate different ees, University of Illinois at Chicago) to facilitate 3-D view- points of view of the model. ing of our model.10 The ImmersaDesk resembles an angled rear-projection big-screen television monitor that pro- EXTRA FEATURES vides the viewer with a large viewing space, stereovision, and a viewer-centered perspective (Figure 4). It The model affords the user additional features that are projects a high-resolution image that appears as a slightly not possible in the study of real physical models. The vir- offset double image to the naked eye. Lightweight inter- tual nose can be viewed with or without a variably trans- active glasses are used to support 3-D stereovision parent skin cover. A mouselike device called the wand (Figure 5). When the image is viewed with these eye- allows the viewer to manipulate and interact with the vir- glasses, the viewer perceives a free-floating 3-D image. tual reality models (Figure 6). The user can grab the Regardless of the viewing angle, the image retains its 3-D model with a virtual reality hand and move it in any di- hologramlike characteristics. The eyeglasses include a rection in space (Figure 7). The viewer can turn the built-in receiver that tracks head position, allowing the model using the wand or change head positions to view computer to continually compute the viewer’s unique per- the model from different perspectives and angles. The spective. The interactive experience is also enhanced by wand also can be used to activate on-screen menu op- the ImmersaDesk’s active tracking of the viewer’s head tions, much like the clicking action of a mouse on a per- position. The teaching system allows the viewer’s head sonal computer. With the wand, the user can zoom in

(REPRINTED) ARCH FACIAL PLAST SURG/ VOL 6, SEP/OCT 2004 WWW.ARCHFACIAL.COM 330

C D

Figure 3. Different views of the “virtual nose” shown in combination with a compatible virtual craniofacial model.

and out of particular areas of interest. The user can make ing perspective is constantly adjusted based on the pri- various structures disappear, such as the upper lateral car- mary viewer’s head position. tilages, lower lateral cartilages, cartilaginous septum, or The networking capabilities of our virtual reality skull background. Individual structures can be faded in model also expands its potential as a teaching tool. Sev- and out of view using intuitive and user-friendly inter- eral viewers can stand in front of the same ImmersaDesk faces, which have been built into the model. By convert- and share the same model, or they can share the same ing more superficial structures into phantom images, the models between multiple ImmersaDesks anywhere in the viewer can grasp the 3-D relationship of superficial and networked world. In this manner, virtual reality lec- deeper structures. For instance, by increasing the trans- tures can be carried out by a lecturer in one location with parency of the lower lateral cartilages, the relationship participants in several other locations. Networked par- of the medial crural footplates and the caudal septum can ticipants see the same 3-D models and are able to com- be clearly appreciated. During all maneuvers, the view- municate by voice (2-way audio communication) and to

(REPRINTED) ARCH FACIAL PLAST SURG/ VOL 6, SEP/OCT 2004 WWW.ARCHFACIAL.COM 331

©2004 American Medical Association. All rights reserved. Downloaded From: https://jamanetwork.com/ on 09/27/2021 Figure 4. A virtual reality display system called the ImmersaDesk displays a Figure 6. The “wand.” stereoscopic image that delivers an interactive 3-dimensional virtual reality experience when viewed with specialized eyeglasses. Multiple viewers can share the same virtual reality experience.

Figure 5. Specialized eyeglasses allow for stereoscopic viewing of the model.

indicate areas of interest by pointing with their own wands. Figure 7. Using the “wand,” the model can be “grabbed” and moved in This interactive system is intuitive, easy to use, and cur- space with an optional virtual hand. rently available in a number of centers. The virtual nose model is also programmed to run on a personal com- can provide an interactive virtual reality experience in puter using one of several software products capable of multiple networked remote locations via the Internet. This displaying virtual reality modeling language models. can enable interactive virtual reality lectures on nasal anatomy from multiple remote networked locations over COMMENT the networked world. Mindful of the specialized equipment needed to run The virtual nose represents a useful application of cut- such an interactive virtual reality model, we also devel- ting edge virtual reality technology to the field of facial oped a simpler version of the model that runs on a cur- plastic surgery. The model provides the user with an ide- rent personal computer mated with a high-resolution alized yet accurate model of the human nose and skull. monitor. This simpler model is still quite impressive and A clear understanding of intricate nasal anatomy is es- functional but does not deliver a true virtual reality ex- sential for all surgeons who operate on the nose. Knowl- perience. The ability to operate a slightly downscaled ver- edge of nasal anatomy can also be useful to medical stu- sion of the virtual nose on a personal computer further dents, medical artists, prosthetic fabricators, and other leverages the teaching potential of this model. It also pro- “students” of human anatomy. Clearly, there is no sub- vides the user with a more accessible and convenient op- stitute for intraoperative observation and experience. Un- tion for exploring the model. fortunately, intraoperative experiences may be limited or The current model represents a prototype, which, unavailable to some. Thus, early exposure to anatomi- like all prototypes, can benefit from improvements. We cal models (such as the virtual nose) may help to are working on a second-generation virtual reality model accelerate the learning curve in the mastery of nasal of the nose that will demonstrate greater anatomical de- anatomy. Better understanding of nasal anatomy may tail and accuracy and will contain more interactive fea- also contribute to the reduction of undesirable surgical tures. Specific improvements will include the addition outcomes. of realistic skin color textures, endonasal structures, vir- The current model is most impressive in a setting tual (lateral and medial) osteotomy paths, better- that allows the viewer to have a complete virtual reality defined subcutaneous soft tissue layers, and the outline experience. The networking potential built into our model of basic rhinoplasty incisions.

(REPRINTED) ARCH FACIAL PLAST SURG/ VOL 6, SEP/OCT 2004 WWW.ARCHFACIAL.COM 332

©2004 American Medical Association. All rights reserved. Downloaded From: https://jamanetwork.com/ on 09/27/2021 Besides its immediate use as an experimental model Accepted for publication June 17, 2004. and a teaching tool, the virtual nose establishes a reli- The modular software programming tools used in this able virtual reality model of the nose, which can facili- study were developed at VRMedLab (Virtual Reality in Medi- tate exciting near future applications based on currently cine Laboratory) at the University of Illinois at Chicago. available technology. Deformable computer models al- Correspondence: A. John Vartanian, MD, Lasky Clinic, low the modification of polygonal shapes or subunits 201 S Lasky Dr, Beverly Hills, CA 90212 (drvartanian based on user preference.11 The introduction of deform- @yahoo.com). able virtual reality nasal models will allow the creation of an almost limitless list of variant nasal anatomical struc- REFERENCES tures. For instance, by using such a deformable model, the participant will be able to select from a menu of different alar cartilage variants and observe the net contribution of 1. Tardy ME Jr. Surgical Anatomy of the Nose. New York, NY: Raven Press; 1990. each variant to the shape of the nose. 2. Larrabee WF Jr, Makielski KH. Surgical Anatomy of the Face. New York, NY: Raven The virtual nose, with its deformable features, can Press; 1993. 3. Tardy ME, Thomas JR, Brown RJ. Facial Aesthetic Surgery. St Louis, Mo: Mosby– also serve as the platform for the development of a sim- Year Book Inc; 1995. plified virtual reality rhinoplasty simulator. In such a 4. Toriumi DM, Becker MD. Rhinoplasty Dissection Manual. Philadelphia, Pa: Lip- model, simple surgical maneuvers and their effects can pincott Williams & Williams; 1999. be demonstrated in 3-D virtual reality. The combina- 5. Ai Z, Dech F, Rasmussen M, Silverstein J. Radiological tele-immersion for next generation networks. In: Westwood JD, Hoffman HM, Mogel GT, Robb RA, tion of 3-D visualization and interactive immediacy can Stredney D, eds. Medicine Meets Virtual Reality 2000. Amsterdam, the Nether- make a rhinoplasty simulator a useful educational tool. lands: IOS Press; 2000:4-9. In summary, many exciting virtual reality applica- 6. Pearl RK, Evenhouse R, Rasmussen M, et al. The virtual pelvic floor, a tele- tions may find future use in facial plastic surgery. We hope immersive educational environment. Proc AMIA Symp. 1999;345-348. that our project is the first of many steps that will be taken 7. Mason TP, Applebaum EL, Rasmussen M, Millman A, Evenhouse R, Planko W. Virtual temporal bone: creation and application of new computer-based teach- worldwide toward this end. ing tool. Otolaryngol Head Neck Surg. 2000;122:168-173. 8. Schroeder W, Martin K, Lorenson B. The Visualization Toolkit: An Object Ori- ented Approach to 3-D Graphics. Upper Saddle River, NJ: Prentice Hall Com- CONCLUSIONS puter Books; 1997. 9. Vartanian AJ, Holcomb J, Ai ZM, Rasmussen M, Tardy ME, Thomas JR. A virtual reality human craniofacial model. Paper presented at: American Academy of Oto- The virtual nose is a 3-D virtual reality model of the nose laryngology–Head and Neck Surgery Annual Meeting; September 23, 2002; San that is accurate and easy to use. It can be run on a per- Diego, Calif. sonal computer or in a specialized virtual reality envi- 10. Czernuszenko M, Pape D, Sandin D, DeFanti T, Dawe GL, Brown MD. The Im- ronment. It can also serve as an effective teaching tool. mersaDesk and Infinity Wall projection-based virtual reality displays. Comput Graph (ACM). 1997;31:46-49. As a working virtual reality model of the nose, it estab- 11. Singh A, Goldgof D, Terzopolous D. Deformable Models in Medical Image Analy- lishes a virtual reality platform from which future appli- sis. Los Alamitos, Calif: Institute of Electrical and Electronic Engineers– cations can be launched. Computer Society; 1998.

(REPRINTED) ARCH FACIAL PLAST SURG/ VOL 6, SEP/OCT 2004 WWW.ARCHFACIAL.COM 333