Virtual Reality Training for Micro-Robotic Cell Injection
Total Page:16
File Type:pdf, Size:1020Kb
Virtual Reality Training for Micro-robotic Cell Injection by Syafizwan Nizam Mohd Faroque BSc (Hons), MEd Submitted in fulfilment of the requirements for the degree of Doctor of Philosophy Deakin University August, 2016 Abstract Micro-robotic cell injection is a procedure requiring a high degree of precision to manoeuvre a motorised micropipette and deposit a small quantity of material at a particular location inside a cell. Performing the procedure requires three-dimensional movement of the micro-robot with individual or simultaneous control of the three normal Cartesian axes. Currently the micro-robotic cell injection procedure is performed manually by an expert bio-operator where lengthy training and a great deal of experience are required to become proficient. In order to reduce the training cost, duration and ethical issues surrounding the use of real cells for training, this thesis introduces two virtual reality micro-robotic cell injection training platforms. The first is a desktop-sized haptically-enabled virtual reality training system which supports portability and flexibility and employs common personal computer or laptop peripherals. A reconfigurable multipurpose haptic interface was also developed as part of the platform in order to provide convenience in setting up and mobility. The second platform provides a large-scale virtual reality platform for micro-robotic cell injection training. It utilises three large screens which are arranged in different display configurations to provide immersive virtual environment. The three display configurations project a large virtual replication of a micro-robotic cell injection setup in two-dimensional, three-dimensional and three-dimensional i immersive displays respectively. The large display area of the two- dimensional large-scale display can provide a very detailed representation of the environment. The three-dimensional large-scale display, in addition, can deliver depth perception which contributes to an increased level of immersion and presence in the virtual environment while the three-dimensional immersive display provides views from three different angles in the virtual environment in order to enhance user’s spatial and orientation understanding. In order to achieve the required high precision movement of the micropipette an appropriate input device and control method are required. This thesis investigates the use of three distinct input devices for control of the micro-robot. The first two input control methods are the computer keyboard and the Phantom Omni haptic device utilised in the desktop-sized virtual reality training system introduced in this thesis. The keyboard is a commonly used interface due to its low-cost and simplicity. From an ergonomics perspective the keyboard control method presented in this thesis is advantageous in that the bio-operator’s hand/arm can be easily supported by the table or desk surface. The Phantom Omni haptic device as an input control method is then considered. A positional mapping between the haptic device stylus and the tip of the micropipette was considered to provide an intuitive control as if the bio-operator is holding a handheld needle insertion device. In addition, haptic feedback is provided to the bio-operator as guidance during injection. The haptic guidance provided in two forms, virtual fixtures and ii force feedback. Three virtual fixtures, conical, axial and planar, were evaluated in their ability to guide the bio-operator in achieving the suitable penetration and deposition points. The force feedback is provided to assist the bio-operator in estimating appropriate force during penetration of the cell membrane in order to increase the survivability of the cell. In order to achieve this, a gradual repulsive force is provided to the bio-operator’s hand during penetration attempt to simulate the contact between the micropipette and the cell membrane. Additionally a sudden drop of force will occur immediately when the cell membrane is penetrated to indicate the rupture as indication to the bio-operator to arrest the force exerted. The third input control method is the large workspace haptic device, INCA 6D, utilised in the large-scale virtual reality training system introduced in this thesis. Utilising the large workspace haptic device coupled with the large display provides several benefits such as delivering an immersive virtual environment and enhancing user’s spatial awareness through intuitive handling. Additionally the utilisation of user’s gross motor skill when manipulating the large workspace haptic device can provide several advantages such as less responsive to insignificant or unintentional movements like vibrations, hand tremors and minor deviations, so that consideration can be focussed on more important criteria such as user’s spatial awareness and understanding of the three-dimensional environment. The approaches provide an alternative method by which to train bio- operators comprised of two different scales of virtual reality platforms. The iii interface for the systems is achieved using computer keyboard and haptic devices which provide the bio-operator with effective and intuitive control. It is also suggested that the acquired skills, knowledge and understanding from the virtual training such as the spatial awareness, depth estimation and hand- eye coordination can be transferred into physical micro-robotic cell injection or similar real tasks. The key contributions of this thesis are: 1) presenting two different scales of haptic VR system for micro-robotic cell injection training, 2) introducing and evaluating a new input control method using a computer keyboard for the desktop-sized VR system, 3) considering the usability and effectiveness of using the Phantom Omni haptic device as an input control method for the desktop-sized VR system, 4) developing and considering the training effects of a large-scale haptic VR system providing three distinct display configurations with a large workspace haptic device as an input control method. iv Acknowledgement I would like to express my heartfelt gratitude to my principal supervisor, Dr Ben Horan, including my associate supervisors, Associate Professor Matthew Joordens and Dr Siva Chandrasekaran, thank you for supporting me all the way through. Dr Horan, especially, had undoubtedly dedicated countless hours guiding and inspiring me to delve into the field of virtual reality and haptics. His ongoing discussion on the detail of my work in ensuring that my research work meets the best quality, standard and criteria as possible should be lauded. Such accomplishments would not be possible too without such undivided assistance and attention. You have been very supportive and I find absolute pleasure to work with you especially in terms of you contributing brilliant ideas and also being a great person to bounce ideas off. Not forgetting fellow researchers, Dr Mulyoto Pangestu and Mr Michael Mortimer, who helped to further my research and inputting excellent ideas. Also my special mention goes to all members of the CADET VR Lab, Deakin University who rendered much assistance and encouragement. I find that it was truly a great pleasure and honour to work with you guys. To my dear wife, Farah Hida Mohiddin, thank you very much for having absolute faith in me and for your deep understanding, patience, support and sacrifice in making it possible. Without you, I would not have achieved such lofty ambition. To my precious children, Hawa, Hana and Hasya, my v special dedication for providing the joy, beauty and truth towards my life. I love you even more than I can ever say and I am extremely honoured and proud to be your father. Lastly, I would like to also express my deep appreciation to my family for their unending support. My special mention goes to Abah, my loving father, for always having full faith in me in pursuing my dream. To my siblings, I am forever grateful for looking up to me as it gives me a real sense of pride. Sad to say, my late mother, could not share my joy and achievement but her love and inspiration shall always linger in my mind. May Allah bless her soul. vi Table of Contents Abstract ............................................................................................................. i Acknowledgement ............................................................................................ v List of Figures ................................................................................................. xi List of Tables ................................................................................................ xiv Chapter 1 Introduction ................................................................................ 1 1.1 Research Significance .................................................................. 3 1.2 Research Problems ....................................................................... 6 1.3 Research Objective ....................................................................... 7 1.4 Thesis Layout and Contributions ................................................. 8 1.5 List of Publications ..................................................................... 10 Chapter 2 Literature Review ..................................................................... 13 2.1 Introduction ................................................................................ 13 2.2 Cell Injection Skills .................................................................... 16 2.2.1 Accuracy and Trajectory ...............................................