Stereoscopic Virtual Realistic Surgical Simulation in Intracranial Aneurysms

Original Article Stereoscopic virtual realistic surgical simulation in intracranial aneurysms

Hiromichi Nakabayashi, Keiji Shimizu1

Department of Neurosurgery, International University of Health and Welfare, Atami Hospital, Atami, 1Department of Neurosurgery, Kochi Medical School Kochi University, Nankoku, Japan

Abstract

Background: Three-dimensional (3D)-computed tomographic angiography (CTA) has been widely used for surgical simulation of intracranial aneurysms. Stereo imaging technology is progressing rapidly in recent years and stereo imaging may make more realistic surgical simulation possible. Therefore, we aimed at the establishment of a technique for stereoscopic viewing of minute volume rendering images while pursuing a low cost. Materials and Methods: Between January 2009 and June 2011, 54 patients with ruptured intracranial aneurysms were enrolled in this study. CTA data was transferred to the workstation equipped with image-processing software, and multilayer fusion images were processed by neurosurgeons. Image data for stereoscopic viewing of multilayer fusion image from arbitrary directions were collected form rotational trajectories around an aneurysm and were output to MPEG file. Stereoscopic viewing using MPEG data was achieved by the freeware named Stereo Movie Maker. Stereo viewing method using QuickTime VR format was also tried. Results: Multilayer fusion image created from CTA Address for correspondence: Mr. Hiromichi Nakabayashi, data displayed clearly the anatomical information about not only the aneurysm but also Department of Neurosurgery, the surrounding structures, such as parent artery, venous system, brain tissue, skull bone, International University of Health and scalp. The quality of the resulting multilayer fusion image was suitable for surgical and Welfare, Atami Hospital, 13-1 simulation with virtual reality. Virtual realistic surgical simulation became possible by the Higashikaigan-cho, Atami 413-0012, combination of minute multilayer fusion image and stereoscopic viewing by our method. Japan. Conclusions: Our method for stereo viewing of multilayer fusion images resulted in an E-mail: [email protected] improvement in the capability of diagnostic imaging and the image-guided support for Received : 17-12-2011 neurosurgical procedures in intracranial aneurysm. Review completed : 05-01-2012 Accepted : 04-03-2012 Key words: 3D-CTA, intracranial aneurysm, stereo imaging, surgical simulation

Introduction disabled.[1] The factor that most correlates with prognosis is the severity of illness at the time of ﬁ rst rupture of Prognosis of aneurysmal subarachnoid hemorrhage aneurysm, and aneurysm rerupture can greatly aggravate (aSAH) remains poor with a mortality rate of 25–50%, the condition of the patient. In Japan, surgical treatment and 10–20% of the affected patients remain severely for aSAH, craniotomy, or endovascular treatment is usually performed “early,” wherein a surgical procedure for a ruptured aneurysm is performed within 72 h Access this article online of the onset of SAH in order to prevent rerupture.[2,3] Quick Response Code: Website: Moreover, for patients with Hunt and Kosnik grades I– www.neurologyindia.com IV, urgent surgical procedure is usually performed within a few hours of the onset of aneurysmal rupture. PMID: *** Preoperative accurate diagnostic imaging and optimal surgical planning are keys to the success of the surgical DOI: procedure. Although there is sufficient preparation 10.4103/0028-3886.96399 time in a scheduled surgery, this may not be the case

Neurology India | Mar-Apr 2012 | Vol 60 | Issue 2 191 Nakabayashi and Shimizu: Stereoscopic surgical simulation in aneurysms in emergency surgeries. Therefore, it is necessary to set Bayer, Leverkusen, Germany) was injected into the up the optimal surgical plan promptly in an emergency cubital vein with an automated injector at a ﬂ ow rate of surgery. In the recent years, three-dimensional (3D) 5 mL/s. Then, enhanced images (140 kV and Auto mA; images can be created easily by advanced imaging gantry rotation speed, 0.6 s/rotation) were obtained by modalities and computer technologies, and 3D medical helical scanning (slice thickness, 0.625 × 64 collimation). imaging has become indispensable to surgical planning. The scan delay was set using the Real Prep mode. The Although imaging modalities other than computed scan range included the craniovertebral junction up to tomography (CT) and a surgical navigation system can the vertex. be used in a scheduled surgery, in an emergency surgery, they cannot be used owing to shortage of preparation Multilayer image fusion time. The imaging modalities that can be used in an CTA data were transferred to the GE Advantage emergency surgery are 3D-CT angiography (CTA) and/ workstation (GE Medical Systems, Waukesha, WI, USA) or digital subtraction angiography (DSA). Thus, it is equipped with an AW VolumeShare™ (GE Medical important to perform accurate surgical simulations using Systems). To shorten image processing time, an intra- the data of 3D-CTA and/or DSA. Although DSA remains hospital local area network (LAN) system was built so the gold standard for the diagnosis of intracranial that our department could acquire Digital Imaging and aneurysms, it is invasive and time consuming, and Communications in Medicine (DICOM) data directly has the risk of aneurysmal rerupture and associated from every imaging modality. This system facilitated complications, with approximately 0.7% occurrence us to respond to emergency cases. Image processing of permanent neurological injury and less than 0.1% began with the segmentation and optimization of mortality rate.[4] Considering these situations, we aimed each structure, such as the arterial system, venous to create realistic surgical simulation images only using system, brain tissue, skull, and soft tissue, using AW 3D-CTA data. Meanwhile, stereoscopic imaging using VolumeShare [Figure 1d–h]. First, the image of the advanced technology to generate more realistic surgical simulation should be pursued. Then, we aimed to develop a low-cost technique for stereoscopic viewing of precise surgical simulation images, with a low set-up time to enable its use in emergency surgeries. a b c

Materials and Methods

Patients Between January 2009 and June 2011, 54 patients (25 men and 29 women) with ruptured intracranial aneurysm treated by craniotomy (neck clipping) were enrolled d e g in this study. Patients treated with endovascular f procedures were excluded. The mean age of the patients included was 58.4 (range, 19–79) years. The locations of the aneurysms were as follows: internal carotid artery–posterior communicating artery (IC–PC) (n = h 24), anterior communicating artery (Acom) (n = 21), and middle cerebral artery (n = 9). All patients in this series underwent CTA. It is our policy to treat ruptured basilar artery aneurysm and IC-ophthalmic aneurysms with endovascular procedures and to treat other aneurysms with craniotomy. In patients with suspected carotid stenosis due to neck bruit, we considered it necessary to evaluate the real circulatory status. Therefore, we undertake angiography only for these patients. i j k CTA Figure 1: The processes of multilayer image fusion are shown in a case Multislice helical CT scanner of 64 detector rows of ruptured right middle cerebral artery aneurysm in a 72-year-old man. (a–c) Maximum intensity projection (MIP) images of the head. (d–h) (Aquilion 64; Toshiba Medical Systems, Tochigi, Japan) Segmented structures (d, artery; e, venous system; f, brain and soft was used for CTA. All patients were positioned supine tissue; g, scalp; and h, skull bone). (i–k) Multilayer fusion images (i and j, fusion image that merged images of the artery, venous system, and skull with the head maintained in a neutral position; 80 mL bone; k, fusion image that merged images of the brain tissue, artery, of nonionic iodinated contrast medium (Iopamiron 300; venous system, and scalp)

192 Neurology India | Mar-Apr 2012 | Vol 60 | Issue 2 Nakabayashi and Shimizu: Stereoscopic surgical simulation in aneurysms

Figure 3: Screenshot of a set of stereo images for the cross-eye method (upper) and a stereo color anaglyph (lower) of the right MCA aneurysm created by the stereo movie maker b were segmented and optimized. Necessary image components were merged and again optimized to obtain the ﬁ nal multilayer fusion image [Figure 1i–k]. Processing of segmentation and fusion of volume data from CTA were performed only by neurosurgeons.

Data acquisition for arbitrary directional stereoscopic images First, using the newly obtained multilayer fusion image, the most suitable rotational trajectory to observe the aneurysm was set and the image data from 50 directions around the aneurysm were collected while rotating by c 7.2° [Figure 2a and b]. Next, the rotational trajectory Figure 2: The process of image data acquisition for stereoscopic was tilted 10° up and down, and data were collected imaging. (a) The image data from 50 directions around an aneurysm similarly. In all cases, image data from 150 or more were collected while rotating the viewing angle by 7.2°. (b) A part of the directions were collected from rotational trajectories image data obtained from the rotation trajectory is shown. (c) First, the rotation trajectory that was most suitable for observing an aneurysm around the aneurysm. When image data from further was set (1). Then, the rotation trajectory was tilted 10° up (2) and down directions were necessary, rotational trajectories for the (3), and data were collected similarly. When image data from other directions were needed, rotation trajectories for further observation were observation were added [Figure 2c]. Theoretically, image added (5–8) data for stereoscopic viewing from arbitrary directions can be obtained by increasing the rotational trajectories arterial system was segmented [Figure 1d]. To secure to collect image data. Finally, the collected image data the accuracy of volume rendering, the images were were stored as an MPEG ﬁ le. segmented while referring to a maximum intensity projection (MIP) image [Figure 1a–c]. After the Stereo surgical simulation using MPEG data opacity and color of the image of the arterial system We adopted two approaches for stereoscopic imaging. were optimized for detailed observation, the image In the first method, MPEG data from the freeware data were preserved. Visual observation was used to named Stereo Movie Maker (http://stereo.jpn.org/ ensure that the aneurysm and surrounding vessels eng/stvmkr/) were used. This software enables various were clearly described in the images. Then, by using a modes of stereo imaging, such as cross-eyed viewing semiautomatic threshold-based segmentation technique, mode, parallel viewing mode, anaglyph mode, and volume images of the brain tissue, skull, and soft tissue liquid crystal (LC) shutter glass mode [Figure 3].

Neurology India | Mar-Apr 2012 | Vol 60 | Issue 2 193 Nakabayashi and Shimizu: Stereoscopic surgical simulation in aneurysms

a b Figure 5: (a) Screen shot of a stereoscopic surgical simulation image displayed on a tablet PC wrapped in a transparent sterile sheet. (b) Screen shot shows a digital eraser pen wrapped in a transparent sterile ﬁ lm

such as a 5-layer fusion image simulating craniotomy, were completed in less than 40 min. Volume rendering Figure 4: Screen shot of a set of stereo images for the cross-eye method images created only from the artery information (upper) of the right MCA aneurysm displayed on QuickTime player provided a satisfactory description of aneurysm in all 34 cases. In the cases in which DSA was performed, images Stereo surgical simulation using QuickTime Virtual of the arterial layer were equivalent to those obtained by Reality (QTVR) format data DSA for evaluating the characteristics of each aneurysm In the second method, we used the QTVR format. (location, size, and direction). Arterial layer images First, screen shots of each frame were collected from also provided an adequate description of the main the MPEG fi le. These screen shots were converted to perforating arteries such as the anterior choroidal artery the QTVR format using Object 2VR software (Garden and Heubner’s artery. The venous system consisting of Gnome Software e.U., Vienna, Austria), which created the sylvian vein, superior sagittal sinus, and cortical vein a stereoscopic image viewable with QuickTime player that are encountered during surgery was also described (Apple Computer Inc., Cupertino, CA, USA) [Figure 4]. clearly. Even in aneurysms located at the skull base, clear separation and depictions of the aneurysms and bone Viewing of stereoscopic surgical simulation image in were possible in three-layer fusion images of the artery, the operating room venous system, and skull. Viewing of stereoscopic surgical simulation images aseptically in the operation room was achieved by Stereoscopic visualization and virtual surgical wrapping a touch-screen tablet PC (HP Elite Book 2730 simulation p; Hewlett-Packard Development Company, Palo Alto, Stereoscopic surgical simulations were performed in all CA, USA) within a sterile transparent sheet [Figure 5a]. patients scheduled for craniotomy. In a usual emergency The displayed image could be rotated freely with a surgery, we first performed a stereoscopic surgical digital eraser pen also wrapped in a transparent sterile simulation using MPEG data. In most cases, image fi lm [Figure 5b]. data from 150 directions collected from three rotational trajectories were sufficient for stereoscopic surgical DSA simulation of a ruptured aneurysm. The time needed DSA was performed using a biplane DSA unit (Infi nix for the export of an MPEG fi le from collected image Celeve; Toshiba Medical Systems, Otawara city, Japan) data was about 2 min. In this method, virtual realistic with a matrix resolution of 1024 × 1024 pixels. surgical simulation was attained by combining minute multilayer fusion images and stereoscopic viewing. Results Then, the stereo images were converted to the QTVR format. Subsequently, stereoscopic surgical simulation Multilayer fusion images using QuickTime player was also achieved. The QTVR Multilayer fusion images were successfully created in all format enabled us to rotate or enlarge images freely and cases. The mean time needed for image processing after to observe them from particular angles. DICOM data acquisition was as follows: artery, 5 min; artery and venous system, 12 min; artery, venous system, In aneurysms of complex shapes, or in Acom aneurysm and skull, 15 min; and artery, venous system, skull, and surrounded by blood vessels of complicated structures, brain tissue, 23 min. More complicated fusion images, stereoscopic viewing was especially useful. For

194 Neurology India | Mar-Apr 2012 | Vol 60 | Issue 2 Nakabayashi and Shimizu: Stereoscopic surgical simulation in aneurysms

a b

e Figure 6: (Case 1) Lt IC–PC aneurysm in a 63-year-old woman with subarachnoid hemorrhage. (a) Cranial CT shows diffuse subarachnoid hemorrhage. 3D-CTA (b) shows the aneurysm (arrow). (c) Enlarged stereo surgical view is shown (upper: a set of stereo images for the cross-eye method, lower: a stereo color anaglyph). The stereo image shows vital structures (anterior choroidal artery, arrow head; posterior communicating artery, white arrow head) around the aneurysm (arrow). (d, e) Stereoscopic simulation images for every step of the left pterional approach

Acom aneurysms, there are usually three kinds of complicated form. Stereoscopic simulation was used surgical approaches: right pterional, left pterional, to select the surgical approach and understand the 3D and interhemispheric. The selection of an appropriate structure around the aneurysm. Stereoscopic simulation surgical approach using 2D-DSA data is usually of the aforementioned three surgical approaches difﬁ cult; however, this selection was greatly aided by indicated that the left pterional approach was reasonable stereoscopic reality simulation based on multilayer [Figure 7]. Neck clipping of the aneurysm was performed fusion images. successfully. The postoperative hospital course was uneventful. We used a wide screen to obtain more accurate surgical simulation. Anaglyph or viewing by LC shutter glasses Discussion was useful for the doctor who cannot perform cross- eyed viewing. The goal of treatment for ruptured aneurysms is to ensure complete, quick, and safe exclusion of the Illustrative cases aneurysm. To achieve this, it is essential to understand Although the photographs of the images shown are the anatomy of the aneurysm and its surrounding monoscopic, the actual surgical planning was carried structures before surgery. Only information about out in stereo. the aneurysm and the surrounding arteries is needed for endovascular surgery; however, more detailed Case 1: A 63-year-old woman presented with anatomical information such as that of the venous subarachnoid hemorrhage. 3D-CTA revealed Lt IC–PC system, skull, and soft tissues is required to reduce aneurysm. Stereoscopic simulation was used to better the surgical risk in craniotomy. In recent years, understand the 3D structure around the aneurysm advancements have been made in 3D image processing [Figure 6]. Neck clipping of the aneurysm was performed technology and 3D images can be easily created from successfully, and her hospital course was uneventful. data obtained from various imaging modalities. As a result, 3D images are used not only for diagnosis Case 2: A 57-year-old man presented with subarachnoid but also for surgical simulation. Several simulation hemorrhage. 3D-CTA revealed Acom aneurysm of a techniques can be used for cerebral aneurysm surgery,

Neurology India | Mar-Apr 2012 | Vol 60 | Issue 2 195 Nakabayashi and Shimizu: Stereoscopic surgical simulation in aneurysms

the Dextroscope is expensive, and therefore difﬁ cult to be introduced in many hospitals.

It is not feasible to use MR imaging immediately after the onset of aneurysmal subarachnoid hemorrhage because a b of body movement and difﬁ culty in dealing with the rerupture of the aneurysm while obtaining the images. The simulation of surgery for intracranial aneurysm using 3D-DSA has also been reported.[6] The 3D-angio system can be a powerful tool in neuroradiology, c especially for surgical intervention of intracranial aneurysms. 3D-DSA provides vivid 3D descriptions of the structure of blood vessels and is extremely effective for the evaluation of intracranial aneurysms. However, 3D-DSA has the disadvantage that data for bony structures cannot be obtained. Thus, 3D-DSA is d the only method by which simulation of craniotomy is difﬁ cult. Moreover, 3D-DSA is invasive, and, at present, is not used by many hospitals. Currently, 3D-CTA is thought to be used for intracranial aneurysm surgery simulation in most hospitals. To date, there are many reports on the use of 3D-CTA for surgical simulation of e intracranial aneurysms.[7-11] In most reports, however, surgical simulation was performed using only data of the arterial system, and it is far from simulation with virtual reality. Meanwhile, studies that reported surgical simulation using data of the bony structures and venous system, apart from the artery system, are f rare.[12] 3D-CTA image data contain information that is essential for craniotomy, such as information about the venous system, skull, and soft tissues, besides the arterial system. To obtain detailed anatomical information, it is important to set optimal viewing conditions for each structure. Therefore, we adopted an image-processing g technique of multilayer fusing of two or more images after optimizing the image segmented from the DICOM data of CTA for each structure.

Meanwhile, the volume rendering image processed from CTA data is only a 2D image when it is still, and it h becomes a pseudo-3D image on rotation. Stereoscopic Figure 7: (Case 2) Anterior communicating artery aneurysm in a 57-year- viewing from arbitrary directions should be more old man with subarachnoid hemorrhage. (a) Cranial CT shows diffuse subarachnoid hemorrhage. 3D-CTA (b) shows the aneurysm (arrow). effective in realistic surgical simulation. To achieve more Stereoscopic simulation images of the three surgical approaches: (c) realistic surgical simulation, we devised a new method interhemispheric approach, (d) right pterional approach, and (e) left for stereoscopic viewing from arbitrary directions using pterional approach. (f, g) Stereoscopic simulation images for every step of the left pterional approach. (h) Enlarged stereo surgical view is shown the Stereo Movie Maker or QTVR technology. By our method, realistic stereoscopic viewing was realized promptly at a low cost. such as methods using 3D-CTA, 3D-DSA, and MR, and methods that involve the use of an expensive surgical Up to now, there have been a few reports that have used simulator. The Dextroscope™ (Volume Interactions QTVR technology in medicine. Friedl et al. used QTVR Pvt. Ltd., Singapore) is a surgical simulator with as an educational tool for aortocoronary bypass grafting, stereoscopic virtual reality using a hologram and is and their prototypical implementation into a database- known to have high accuracy. There are a few reports driven and internet-based educational system in heart on the use of the Dextroscope in cases of neurosurgical surgery.[13] Balogh et al. acquired intraoperative images diseases, including intracranial aneurysm.[5] However, during neurosurgical procedures for later reconstruction

196 Neurology India | Mar-Apr 2012 | Vol 60 | Issue 2 Nakabayashi and Shimizu: Stereoscopic surgical simulation in aneurysms with a QTVR stereoscopic image system. They are using following neuroangiography: A prospective study. Stroke 1987;18:997- this system to learn microsurgical techniques.[14] Another 1004. 5. Wong GK, Zhu CX, Ahuja AT, Poon WS. Craniotomy and clipping of application of QTVR in medicine is as an educational tool intracranial aneurysm in a stereoscopic virtual reality environment. to study anatomy. The present study is the fi rst report Neurosurgery 2007;61:564-8. describing the application of QTVR movie maker in 6. Thines L, Taschner C, Lejeune JP, Le Thuc V, Pruvo JP, Bourgeois P, clinical medicine. et al. Surgical views from three-dimensional digital subtraction angiography for the planning of aneurysm surgery. J Neuroradiol 2007;34:205-11. For emergency operations with a high degree of 7. Matsumoto M, Sato M, Nakano M, Endo Y, Watanabe Y, Sasaki T, et al. diffi culty, such as aneurysmal clipping, preoperative Three-dimensional computerized tomography angiography-guided surgery realistic imaging and accurate surgical simulation is of acutely ruptured cerebral aneurysms. J Neurosurg 2001;94:718-27. 8. Kato Y, Sano H, Katada K, Ogura Y, Hayakawa M, Kanaoka N, et al. critical for a successful surgical outcome. Newly devised Application of three-dimensional CT angiography (3D-CTA) to cerebral arbitrary directional stereoscopic viewing of multilayer aneurysms. Surg Neurol 1999;52:113-21. fusion images enabled detailed surgical planning with 9. González-Darder JM. ACoA angle measured by computed tomographic 3D virtual reality. The surgical strategy selected on the angiography and its relevance in the pterional approach for ACoA aneurysms. Neurol Res 2002;24:291-5. basis of preoperative surgical simulation proved to 10. Futami K, Nakada M, Iwato M, Kita D, Miyamori T, Yamashita J. be the correct choice during the actual operation, and Simulation of clipping position for cerebral aneurysms using three- the operation was carried out without complications dimensional computed tomography angiography. Neurol Med Chir in all cases. Stereoscopic virtual realistic surgical (Tokyo) 2004;44:6-12. 11. Chen W, Yang Y, Qiu J, Peng Y, Xing W. Clinical application of 16-row simulation is helpful in designing minimally invasive multislice computed tomographic angiography in the preoperative and intracranial procedures. Our imaging technique has postoperative evaluation of intracranial aneurysms for surgical clipping. useful applications in surgery for not only cerebral Surg Neurol 2009;71:559-65. aneurysms but also other neurosurgical diseases. 12. Suzuki Y, Nakajima M, Ikeda H, Ikeda Y, Abe T. Preoperative evaluation of the venous system for potential interference in the clipping of cerebral aneurysm. Surg Neurol 2004;61:357-64. References 13. Friedl R, Preisack MB, Klas W, Rose T, Stracke S, Quast KJ, et al. Virtual reality and 3D visualizations in heart surgery education. Heart 1. Hop JW, Rinkel GJ, Algra A, van Gijn J. Case-fatality rates and Surg Forum 2002;5:E17-21. functional outcome after subarachnoid hemorrhage: A systematic review. 14. Balogh A, Preul MC, Schornak M, Hickman M, Spetzler RF. Stroke 1997;28:660-4. Intraoperative stereoscopic QuickTime Virtual Reality. J Neurosurg 2. Yamamoto K, Ezuka I, Takai N, Kakinuma K. Comparison of late and 2004;100:591-6. early stage surgery for ruptured intracranial aneurysms. Neurol Med Chir (Tokyo) 1992;32:1-4. How to cite this article: Nakabayashi H, Shimizu K. Stereoscopic 3. Miyaoka M, Sato K, Ishii S. A clinical study of the relationship of timing virtual realistic surgical simulation in intracranial aneurysms. Neurol to outcome of surgery for ruptured cerebral aneurysms. A retrospective India 2012;60:191-7. analysis of 1622 cases. J Neurosurg 1993;79:373-8. 4. Dion JE, Gates PC, Fox AJ, Barnett HJ, Blom RJ. Clinical events Source of Support: Nil, Confl ict of Interest: None declared.

Announcement iPhone App

A free application to browse and search the journal’s content is now available for iPhone/iPad. The application provides “Table of Contents” of the latest issues, which are stored on the device for future offline browsing. Internet connection is required to access the back issues and search facility. The application is Compatible with iPhone, iPod touch, and iPad and Requires iOS 3.1 or later. The application can be downloaded from http:// itunes.apple.com/us/app/medknow-journals/id458064375?ls=1&mt=8. For suggestions and comments do write back to us.

Neurology India | Mar-Apr 2012 | Vol 60 | Issue 2 197 Copyright of Neurology India is the property of Medknow Publications & Media Pvt. Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.