A Review of Augmented Reality and Virtual Reality in Plastic Surgery
Total Page:16
File Type:pdf, Size:1020Kb
applyparastyle "fig//caption/p[1]" parastyle "FigCapt" applyparastyle "fig" parastyle "Figure" Research Aesthetic Surgery Journal Special Topic 2019, Vol 39(9) 1007–1016 © 2019 The American Society for Aesthetic Plastic Surgery, Inc. Reprints and permission: journals. Downloaded from https://academic.oup.com/asj/article-abstract/39/9/1007/5316200 by ASAPS Member Access user on 08 May 2020 The New Frontier: A Review of Augmented [email protected] DOI: 10.1093/asj/sjz043 Reality and Virtual Reality in Plastic Surgery www.aestheticsurgeryjournal.com Lohrasb R. Sayadi, MD; Alexandra Naides, BA; Maddie Eng; Arman Fijany, BS; Mustafa Chopan, MD; Jamasb J. Sayadi, BA; Ashkaun Shaterian, MD; Derek A. Banyard, MD, MBA, MS ; Gregory R.D. Evans, MD, FACS; Raj Vyas, MD; and Alan D. Widgerow, MBBCh, MMed, FACS Abstract Mixed reality, a blending of the physical and digital worlds, can enhance the surgical experience, leading to greater precision, efficiency, and improved outcomes. Various studies across different disciplines have reported encouraging results using mixed reality technologies, such as augmented and virtual reality. To provide a better understanding of the applications and limitations of this technology in plastic surgery, we performed a systematic review of the literature in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. The initial query of the National Center for Biotechnology Information database yielded 2544 results, and only 46 articles met our inclusion criteria. The majority of studies were in the field of craniofacial surgery, and uses of mixed reality included preoperative planning, intraoperative guides, and education of surgical trainees. A deeper understanding of mixed reality technologies may promote its integration and also help inspire new and creative applications in healthcare. Editorial Decision date: February 5, 2019; online publish-ahead-of-print February 12, 2019. Mixed reality, a blending of the physical and digital worlds, Advances in computer processing, motion-tracking offers unique opportunities in the field of plastic and recon- sensors, and human-computer interaction have helped structive surgery. Surgical simulations that incorporate achieve the sophisticated interplay between spatial visual and tactile feedback to promote technical proficiency have been adopted across many surgical fields.1-4 New Dr L.R. Sayadi is a Research Fellow, Mrs Naides is a Medical Student innovations have made it easier to harness mixed reality to Researcher, Ms Eng is an Undergraduate Researcher, Mr Fijany is improve surgical outcomes through more robust preopera- a Research Assistant, Drs Shaterian and Banyard are Residents, Dr Vyas is an Associate Professor, Dr Evans is the Chief, and tive planning and projection of 3D models intraoperatively. Dr Widgerow is the Director of the UC Irvine Center for Tissue Virtual and augmented reality (VR and AR) are classes of Engineering, UC Irvine Department of Plastic Surgery, Center for mixed reality. In its strictest sense, VR completely replaces Tissue Engineering, Orange, CA. Dr Chopan is a Resident, Division of Plastic and Reconstructive Surgery, University of Florida, the real world with a digital environment, whereas AR Gainesville, FL. Mr J.J. Sayadi is a Medical Student, Stanford overlays computer-generated content (ie, graphics or vid- Medical School, Stanford, CA. eos) onto the physical landscape. Moreover, the degree of Corresponding Author: immersion and correspondence with the physical environ- Dr Alan D. Widgerow, University of California, Irvine, Suite 108A, ment can also “augment” virtual realities. For clarity, the Building 55, 101 S. City Drive, Orange CA, 92868, USA. continuum of mixed reality is depicted in Figure 1. E-mail: [email protected]; Twitter: @awidgerow 1008 Aesthetic Surgery Journal 39(9) Downloaded from https://academic.oup.com/asj/article-abstract/39/9/1007/5316200 by ASAPS Member Access user on 08 May 2020 Figure 1. Continuum of mixed reality. data-capturing techniques, rendering of 3-dimensional utilized: “plastic surgery,” “cosmetic surgery,” “aesthetic (3D) models or graphics, and visual projection that is surgery,” “reconstructive surgery,” “virtual reality,” “VR,” required for this technology to be implemented in a sur- “augmented reality,” and “AR.” Referenced articles from gical setting. Many VR and AR systems make it possible the selected studies were further evaluated for potential for digital content to be projected onto body surfaces inclusion in our analysis. In total, 2544 unique studies or visualized through head mounted devices (HMD). were identified from our search. When utilized in a mixed reality application, these vir- tual images and models may need to be registered, or Selection Criteria aligned, with the physical world utilizing anatomical markers. All abstracts were assessed and selected if they employed Various studies on mixed reality across different dis- AR/VR technologies for the purposes of plastics, cosmetic, ciplines have reported enhancements of the surgical aesthetics, or reconstructive surgery. Articles were experience with encouraging data to suggest greater generally excluded if they employed AR/VR for purposes precision, accuracy, and efficiency and improved out- other than the surgical disciplines listed in the inclusion comes.5-8 Delving deeper into AR’s and VR’s current criteria. The full text of each article was read critically by uses in plastic surgery may help provide practitioners research assistants (M.E., A.F., and A.N.) to ensure each with a better understanding of the applications and lim- article meet selection criteria. Each summary and article itations of this technology as well as promote integration were then read and cross-referenced for completeness of these technologies into surgical practice. A deeper (R.S.). In total, 46 articles were included for review understanding of the technology behind mixed reality (Tables 1-3). For the complete search strategy, please see can help inspire new and creative applications in health- Figure 2 and Supplemental Figure 1. care. To this end, we performed a systematic review of the literature regarding VR and AR. Many of the mixed reality systems discussed in this paper have applications RESULTS in multiple stages of the surgical process, but for ease of organization we have divided the systems into 3 broad Augmented Reality/Virtual Reality in categories: preoperative planning, intraoperative tools, Preoperative Planning and surgical education. AR/VR systems were first applied to the medical field in the 1990s, aiding in the preoperative planning of surgical procedures and displaying medical information for METHODS surgeons in the operating theater. Since then, the range Study Design and complexity of the medical applications for AR/VR technology have expanded significantly, especially in the A literature search was conducted on June 12, 2018 field of plastic surgery, where it is employed extensively utilizing the National Center for Biotechnology Information in craniofacial operations. In this section, we review database to search for articles employing AR/VR in plastic studies pertaining to utilization of AR/VR technology for and reconstructive surgery. The following keywords were preoperative planning of plastic surgery procedures. Sayadi et al 1009 Table 1. Studies Evaluating the Utilization of AR/VR in Preoperative Plan- Table 2. Studies Evaluating the Utilization of AR/VR in the Operating ning Room Study AR/VR application Study AR/VR application Muhlbauer et al9 VR models to predict rhinoplasty outcomes Jiang et al24 AR system capable of displaying a patient’s vasculature, muscle, and bone Fernandez- VR models for preoperative planning of craniofacial procedures Alvarez et al10 Pratt et al25 AR system capable of displaying a patient’s vasculature onto the surgical site being operated on Downloaded from https://academic.oup.com/asj/article-abstract/39/9/1007/5316200 by ASAPS Member Access user on 08 May 2020 Eckardt et al11 VR simulation of left mandible resection Cutting et al26 VR system to track bone segment positioning in multi-segment Ikawa et al12 VR system to design titanium mesh tray for mandibular reconstruction midface osteotomies Hsieh et al13 VR system for preoperative planning and simulation of difficult oste- Zhu et al27 AR system to create a virtual plan and guide intraoperative osteoto- otomy and musculoskeletal fusion procedures mies in patients with orbital hypertelorism Seruya et al14 VR system to guide craniosynostosis procedures Zhu et al28 AR system for mandibular angle osteotomy Landes et al15 VR system to create 3D reconstructions of labial, palatal, and velo- Mischkowski AR system for guiding and tracking the translocation of maxilla pharyngeal anatomy et al29 during Le Fort I osteotomy Gacto-Sanchez CTA guided 3D virtual reconstruction (VR) for localization of DIEP flap Doscher et al30 VR system to guide mandibular distraction osteogenesis et al16 perforators Qu et al31 AR system for distractor placement in distraction osteogenesis Gomez-Cia et al17 CTA guided 3D virtual reconstruction (VR) for DIEP microsurgical breast reconstruction Mitsuno et al32 AR system for intraoperative comparisons of facial symmetry Suffee et al18 Comparison of CTA images interpreted by a radiologist vs CTA guided Lin et al33 Combination of both AR and VR with robot-assisted surgery 3D virtual reconstruction (VR) interpreted by a surgeon for preop- erative planning of DIEP flaps Shi et al34 Combination