A Scalable, High-Performance, Real-Time Control Architecture With
Total Page:16
File Type:pdf, Size:1020Kb
A Scalable, High-Performance, Real-Time Control Architecture with Application to Semi-Autonomous Teleoperation by Zihan Chen A dissertation submitted to The Johns Hopkins University in conformity with the requirements for the degree of Doctor of Philosophy. Baltimore, Maryland October, 2017 ⃝c Zihan Chen 2017 All rights reserved Abstract A scalable and real-time capable infrastructure is required to enable high-performance control and haptic rendering of systems with many degrees-of-freedom. The specific platform that motivates this thesis work is the open research platform da Vinci Re- search Kit (dVRK). For the system architecture, we propose a specialized IEEE-1394 (FireWire) broad- cast protocol that takes advantage of broadcast and peer-to-peer transfers to mini- mize the number of transactions, and thus the software overhead, on the control PC, thereby enabling fast real-time control. It has also been extended to Ethernet via a novel Ethernet-to-FireWire bridge protocol. The software architecture consists of a distributed hardware interface layer, a real-time component-based software frame- work, and integration with the Robot Operating System (ROS). The architecture is scalable to support multiple active manipulators, reconfigurable to enable researchers to partition a full system into multiple independent subsystems, and extensible at all levels of control. This architecture has been applied to two semi-autonomous teleoperation applica- ii ABSTRACT tions. The first application is a suturing task in Robotic Minimally Invasive Surgery (RMIS), that includes the development of virtual fixtures for the needle passing and knot tying sub-tasks, with a multi-user study to verify their effectiveness. The second application concerns time-delayed teleoperation of a robotic arm for satellite servic- ing. The research contribution includes the development of a line virtual fixture with augmented reality, a test for different time delay configurations and a multi-user study that evaluates the effectiveness of the system. Thesis Advisor Peter Kazanzides Ph.D., Johns Hopkins University, Baltimore Thesis Committee Russell H. Taylor Ph.D., Johns Hopkins University, Baltimore Louis L. Whitcomb Ph.D., Johns Hopkins University, Baltimore iii Acknowledgments First I would like to thank my advisor Professor Peter Kazanzides. His guidance, motivation, firm and continuous support and encouragement was a constant presence during my Ph.D. and Master work over the past years. He was always willing to share his deep knowledge in electronics, software and robotics, always available whenever I wanted to discuss ideas, issues and progress. I could not have had a better advisor and mentor for my Ph.D. study. I would like to thank Professor Louis L. Whitcomb for his guidance and for par- ticipating on my thesis and GBO committee. I would also like to thank Professor Russell H. Taylor for the wonderful Computer Integrated Surgery course, which intro- duced me to the medical robotics field and gave me the opportunity to work with Dr. Kazanzides, for advising the virtual fixture for suturing project, and for participating on my thesis committee. I would additionally like to thank Professor Iulian Iordachita for his help in me- chanical design and manufacture, as well as for participating on my GBO committee. I would like to thank Professor Scott Smith for participating on my GBO committee, iv ACKNOWLEDGMENTS for sharing his insight into software engineering and for writing my annual review let- ter. Special thanks to Professor Allison Okamura for advising me during my master program. Thanks to Professor Simon Leonard for sharing his knowledge in software, robotics and computer vision. Many thanks to Anton Deguet for helping me write and debug numerous lines of code, teaching me almost every aspect of programming, sharing his broad wisdom in software design as well as his appreciation of fine coffee. I am also thankful for the many fun conversations we had. I would also like to thank Balazs P. Vagvolgyi for sharing his expertise in computer vision and software. I would like to thank members of the Sensing, Manipulation, and Real-Time System (SMARTS) group for all the guidance, help, support and discussion over the years. Special thanks to Dr. Paul Thienphrapa for his patient guidance for software design, FPGA programming and FireWire. Thanks to Dr. Tian Xia for help in compiling and debugging code and advice on Ph.D. life. Thanks to Dr. Min Yang Jung for his help on various software/hardware topics. Thanks to Dr. H. Tutkun Sen and Ehsan Azimi for the endless wonderful conversations. Thanks to Long Qian for his help in developing FPGA firmware. Thanks to Jie Ying Wu for organizing allthe lab lunches. Thanks also to the other members of SMARTS: Nathan Cho, Sungmin Kim, Andrew Hundt, Paul Wilkening and Isha Kandaswamy. I was also lucky to be part of the larger Laboratory for Computational Sensing and Robotics. I owe special thanks to Dr. Jonathan Bohren for sharing his insights in software and robotics and v ACKNOWLEDGMENTS introducing the Robot Operating System. Thanks to Dr. S. Swaroop Vedula for his help in designing the user study, as well as analyzing data. Thanks to Dr. Anand Malpani for his help in designing virtual fixtures, and for paper writing. Thanks to Dr. Kelleher R. Guerin for sharing his expertise in mechanical design and robotics. Thanks also to Preetham Chalasani for all the great conversations and the good food. Finally, I would like to thank my parents, Yuewei Chen and Ping Chen, for their constant love, guidance, encouragement, and support. vi Dedication I dedicate my dissertation work to my parents for all the help, support, encour- agement and love during the long journey. This Thesis would not have been possible without you! vii Contents Abstract ii Acknowledgments iv List of Tables xv List of Figures xvi 1 Introduction 1 1.1 The da Vinci Research Kit (dVRK) . .2 1.2 Challenges of a scalable and high performance architecture for dVRK5 1.3 Overview of Architectures . .7 1.4 Proposed Approach . 10 1.5 Applications . 12 1.6 Broader Impact . 13 1.7 Overview of Contributions . 13 1.7.1 A broadcast-based fieldbus communication protocol . 13 viii CONTENTS 1.7.2 A bridge design to enable real-time control over a conventional network . 14 1.7.3 An analytical model of the timing performance for the proposed protocols and EtherCAT . 14 1.7.4 A scalable and extensible software architecture . 15 1.7.5 Virtual fixtures for suturing and knot tying tasks . 15 1.7.6 Virtual fixtures for time-delayed teleoperation for satellite ser- vicing . 16 2 System Architecture 17 2.1 Introduction and Overview of Thesis Contributions . 17 2.2 Historical Context . 19 2.3 Fieldbus Survey . 20 2.4 Distributed I/O System . 26 2.4.1 Introduction to IEEE 1394 . 29 2.4.2 FireWire Transactions Timing Performance . 33 2.4.3 System Performance . 33 2.4.4 Analysis . 34 2.5 Broadcast Communication Protocol . 36 2.5.1 Transmission model . 37 2.5.2 Bus optimizations . 39 2.5.3 System Characteristics . 41 ix CONTENTS 2.5.3.1 Determinism . 41 2.5.3.2 Error tolerance . 42 2.5.3.3 Backward compatibility . 43 2.5.4 Experiments . 44 2.5.4.1 FPGA hardware-based measurement . 44 2.5.4.2 Model parameter estimation with PC software-based measurement . 45 2.5.5 Discussion . 53 2.6 Ethernet-to-FireWire Bridge for Real-time Control . 54 2.6.1 Introduction . 55 2.6.2 Ethernet-to-FireWire Bridge Design . 57 2.6.2.1 Prototype Bridge Board Design . 57 2.6.2.2 Frame Transmission Protocol . 58 2.6.2.3 Status Control . 61 2.6.2.4 Ethernet Software . 61 2.6.3 Experiments . 63 2.6.3.1 FireWire Transaction over Ethernet . 64 2.6.3.2 System Performance . 66 2.6.3.3 Cross-platform Capability . 68 2.6.3.4 Ethernet Bridge Timing Model . 69 2.6.4 Discussion . 70 x CONTENTS 2.7 Performance Comparison with EtherCAT . 73 2.7.1 Introduction to EtherCAT . 73 2.7.2 EtherCAT Timing Performance . 75 2.7.2.1 EtherCAT Timing Model . 76 2.7.2.2 Model Parameter Measurement . 78 2.7.2.3 EtherCAT Model Verification . 80 2.7.3 Time Performance Comparison on dVRK . 82 2.7.4 Discussion . 82 2.8 Conclusions . 85 3 Software Architecture 89 3.1 Introduction . 89 3.2 Thesis Contributions . 92 3.3 Related Work . 93 3.4 Low-Level Hardware Interface Layer (Fieldbus) . 96 3.5 Real-time framework for robot control . 98 3.5.1 Design Goals . 98 3.5.2 Design Analysis . 99 3.5.3 Implementation . 104 3.6 System integration via ROS interfaces . 106 3.6.1 CISST to ROS Bridge . 107 3.6.2 ROS Ecosystem . 109 xi CONTENTS 3.7 Discussion and Conclusion . 109 4 Application to Virtual Fixture Assisted Suturing 112 4.1 Introduction . 112 4.2 Thesis Contributions . 115 4.3 Virtual Fixtures . 116 4.3.1 Task Description and Analysis . 116 4.3.2 Impedance Virtual Fixture . 117 4.3.3 Needle Passing Virtual Fixture . 120 4.3.4 Knot Tying Virtual Fixture . 123 4.4 Experiment . 124 4.4.1 System Implementation on da Vinci Research Kit . 125 4.4.2 Needle Passing Sub-task . 126 4.4.3 Knot Tying Sub-task . 127 4.4.4 Test Procedure . 128 4.4.5 Data Collection and Analysis . 130 4.5 Results and Discussion . 130 4.5.1 Needle Passing Sub-task . 131 4.5.1.1 Statistical Analysis . 131 4.5.1.2 Trajectory Analysis . 131 4.5.1.3 Operator Workload . 133 4.5.1.4 Subjective Evaluation . 135 xii CONTENTS 4.5.2 Knot Tying Sub-task .