Cubesat Cloud: a Framework for Distributed Storage, Processing and Communication of Remote Sensing Data on Cubesat Clusters
Total Page：16
File Type：pdf, Size：1020Kb
CUBESAT CLOUD: A FRAMEWORK FOR DISTRIBUTED STORAGE, PROCESSING AND COMMUNICATION OF REMOTE SENSING DATA ON CUBESAT CLUSTERS By OBULAPATHI NAYUDU CHALLA A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2013 © 2013 Obulapathi Nayudu Challa 2 I dedicate this to my family, my wife Sreevidya Inturi, my mother Rangamma Challa, my father Ananthaiah Challa, my sister Sreelatha Chowdary Lingutla, my brother-in-laws Ramesh Naidu Lingutla, Sreekanth Chowdary Inturi, my father-in-law Sreenivasulu Chowdary Inturi, my mother-in-law Venkatalakshmi Inturi, my brothers Akshay Kumar Anugu and Dheeraj Kota and my uncle Venkatanarayana Pattipaati, for all their love and support. 3 ACKNOWLEDGMENTS It has been a great experience being a student of Dr. Janise Y. McNair for the last five and half years. There was never a time that I did not feel cared for, thanks to her constant support and guidance. I would like to thank my committee Dr. Xiaolin (Andy) Li, Dr. Norman G. Fitz-Coy and Dr. Haniph A. Latchman for agreeing to serve on my committee. I would like to thank them for providing valuable feedback in completing my dissertation. Thanks to the professors at University of Florida Dr. Patrick Oscar Boykin, Ms. Wenhsing Wu, Dr. Ramakant Srivastava, Dr. Erik Sander, Dr. A. Antonio Arroyo, Dr. Jose A. B. Fortes, Dr. John M. Shea, Dr. Greg Stitt, Dr. Sartaj Sahni and Dr. Shigang Chen, for teaching me what all I know today. Thanks to staff at University of Florida Ray E. McClure II, Jason Kawaja, Shannon M Chillingworth, Cheryl Rhoden and Stephenie A. Sparkman, for their patience with my countless requests and administrative questions. I would like to take this opportunity to thank all my Wireless and Mobile Group colleagues, past and present, for being there with me and helping me all along in one way or other. I would like to thank Alexander Verbitski for his mentorship during my internship. I would like to thank my teachers Sreedevi, Uma Kantha, Nalini Sreenivasan, K. Ramakrishna, K. Bhaskar Naidu, Sambasiva Reddy, A Koteswar Rao, A. K. Rama Rao, Dr. Vijay Kumar Chakka, Dr. Gautam Dutta and Dr. Prabhat Ranjan who greatly influenced my life. Internet and Open Source have made this world a true Vasudhaika Kutumbam for me. I would like to thank Linus Torvalds, creator of Linux; Richard Matthew Stallman, founder of GNU; Vint Cerf, father of Internet; Tim Berners-Lee, inventor of the World Wide Web; Guido Rossum, creator of Python programming language; Satoshi Nakamoto, inventor of Bitcoin; Masashi Kishimoto, creator of Naruto; Mark Shuttleworth, founder of Ubuntu and Tim O’Reilly, the founder of O’Reilly Media. Life at University of Florida has been always fun and exciting, thanks to the wonderful friends around here: Dan Trevino, Dante Buckley, Gokul Bhat, Hrishikesh 4 Pendurkar, Jimmy (Tzu Yu) Lin, Karthik Talloju, Krishna Chaitanya, Kishore Yalamanchili, Madhulika Dandina, Manu Rastogi, Paul Muri, Rakesh Chalasani, Ravi Shekhar, Seshu Pria, Shruthi Venkatesh, Subhash Guttikonda, Udayan Kumar, Vaibhav Garg, Vijay Bhaskar Reddy and Vivek Anand. I would like to thank Mr. Iqbal Qaiyumi, Dr. Shaheda Qaiyumi, Mr. Jagat Desai and Mrs. Vatsala Desai for taking care of me like their son. Thanks to my long-distance friends Bhargavi Vanga, Praveen Kumar, Radha Vummadi, Uday Kumar, Uzumaki Naruto and Vijay Kumar, who have been close even when they were far. Lastly, I would like to thank my family - my wife Sreevidya Inturi, my mother Rangamma Challa, my father Ananthaiah Challa, my sister Sreelatha Chowdary Lingutla, my brother-in-laws Ramesh Naidu Lingutla, Sreekanth Chowdary Inturi, my father-in-law Sreenivasulu Chowdary Inturi, my mother-in-law Venkatalakshmi Inturi, my brothers Akshay Kumar Anugu and Dheeraj Kota and my uncle Venkatanarayana Pattipaati. Their endless love and support throughout the years has meant more to me than words can express. I would like to dedicate my dissertation to them. 5 TABLE OF CONTENTS page ACKNOWLEDGMENTS .................................. 4 LIST OF TABLES ...................................... 10 LIST OF FIGURES ..................................... 11 ABSTRACT ......................................... 13 CHAPTER 1 INTRODUCTION ................................... 15 1.1 CubeSat Cloud ................................. 17 2 BACKGROUND ................................... 20 2.1 Remote Sensing ................................ 20 2.2 Evolution of CubeSat Networks ........................ 21 2.2.1 Summary and Limitations of CubeSat Communications ...... 24 2.3 Distributed Satellite Systems ......................... 25 2.4 Classification of Distributed Satellite Systems ................ 27 2.5 Related Work .................................. 27 2.5.1 Distributed Storage Systems ..................... 27 2.5.2 Distributed Computing Techniques .................. 30 3 NETWORK ARCHITECTURE OF CUBESAT CLOUD .............. 34 3.1 Components of the CubeSat Network .................... 35 3.1.1 Space Segment ............................. 35 3.1.2 Ground Segment ............................ 37 3.2 System Communication ............................ 38 3.2.1 Cluster Communication ........................ 38 3.2.2 Space Segment to Ground Segment Communication ....... 39 3.2.3 Ground Segment Network Communication .............. 40 3.3 CubeSat Cloud ................................. 40 3.3.1 Storage, Processing and Communication of Remote Sensing Data on CubeSat Clusters .......................... 40 3.3.2 Source Coding, Storing and Downlinking of Remote Sensing Data on CubeSat Clusters .......................... 41 4 DISTRIBUTED STORAGE OF REMOTE SENSING IMAGES ON CUBESAT CLUSTERS ...................................... 45 4.1 Key Design Points ............................... 45 4.1.1 Need for Simple Design ........................ 45 4.1.2 Low Bandwidth Operation ....................... 45 6 4.1.3 Network Partition Tolerant ....................... 45 4.1.4 Autonomous ............................... 46 4.1.5 Data Integrity .............................. 46 4.2 Shared Goals Between CDFS, GFS and HDFS ............... 46 4.2.1 Component Failures are Norm .................... 46 4.2.2 Small Number of Large Files ..................... 46 4.2.3 Immutable Files and Non-existent Random Read Writes ...... 47 4.3 Architecture of CubeSat Distributed File System .............. 47 4.3.1 File System Namespace ........................ 49 4.3.2 Heartbeats ............................... 49 4.4 File Operations ................................. 50 4.4.1 Create a File .............................. 50 4.4.2 Writing to a File ............................. 51 4.4.3 Deleting a File .............................. 51 4.5 Enhancements and Optimizations ...................... 52 4.5.1 Bandwidth and Energy Efficient Replication ............. 52 4.5.1.1 Number of nodes on communication path = replication factor ............................. 54 4.5.1.2 Number of nodes on communication path >replication factor ............................. 54 4.5.1.3 Number of nodes on communication path <replication factor ............................. 54 4.5.2 Load Balancing ............................. 55 4.5.3 Chunk Size and Granularity ...................... 56 4.5.4 Fault Tolerance ............................. 57 4.5.5 Master Failure .............................. 57 4.5.6 Worker Failure ............................. 57 4.5.7 Chunk Corruption ............................ 58 4.5.8 Inter CubeSat Link Failure ....................... 58 4.5.9 Network Partitioning .......................... 58 4.6 Simulation Results ............................... 59 4.7 Summary of CubeSat Distributed File System ................ 59 5 DISTRIBUTED PROCESSING OF REMOTE SENSING IMAGES ON CUBESAT CLUSTERS ...................................... 60 5.1 CubeSat MapMerge .............................. 60 5.2 Command and Data Flow during a CubeSat MapMerge Job ........ 61 5.3 Fault Tolerance, Failures, Granularity and Load Balancing ......... 63 5.3.1 Fault Tolerance ............................. 63 5.3.2 Master Failure .............................. 64 5.3.3 Worker Failure ............................. 64 5.3.4 Task Granularity and Load Balancing ................. 64 5.4 Simulation Results ............................... 64 5.5 Summary of CubeSat MapMerge ....................... 65 7 6 DISTRIBUTED COMMUNICATION OF REMOTE SENSING IMAGES FROM CUBESAT CLUSTERS ............................... 66 6.1 CubeSat Torrent ................................ 66 6.2 Command and Data Flow During a Torrent Session ............. 67 6.3 Enhancements and Optimizations ...................... 67 6.3.1 Improve Storage Reliability and Decrease Storage Overhead . 67 6.3.2 Using Source Coding to Improve Downlink Time .......... 69 6.3.3 Improving the Quality of Service for Real-time Traffic Applications Like VoIP ................................ 70 6.4 Fault Tolerance, Failures, Granularity and Load Balancing ......... 71 6.4.1 Fault Tolerance ............................. 71 6.4.2 Master Failure .............................. 72 6.4.3 Worker Failure ............................. 72 6.4.4 Task Granularity ............................ 72 6.4.5 Tail Effect and Backup Downloads .................. 73 6.5 Simulation Results and Summary of CubeSat Torrent ........... 73 7 SIMULATOR, EMULATOR AND PERFORMANCE ANALYSIS .......... 75 7.1 Hardware and Software of Master and Worker CubeSats for Emulator . 75 7.2 Hardware and