Methods and Design Issues for Next Generation Network-Aware Applications Andrei Hutanu Louisiana State University and Agricultural and Mechanical College
Total Page:16
File Type:pdf, Size:1020Kb
Louisiana State University LSU Digital Commons LSU Doctoral Dissertations Graduate School 2009 Methods and design issues for next generation network-aware applications Andrei Hutanu Louisiana State University and Agricultural and Mechanical College Follow this and additional works at: https://digitalcommons.lsu.edu/gradschool_dissertations Part of the Computer Sciences Commons Recommended Citation Hutanu, Andrei, "Methods and design issues for next generation network-aware applications" (2009). LSU Doctoral Dissertations. 2803. https://digitalcommons.lsu.edu/gradschool_dissertations/2803 This Dissertation is brought to you for free and open access by the Graduate School at LSU Digital Commons. It has been accepted for inclusion in LSU Doctoral Dissertations by an authorized graduate school editor of LSU Digital Commons. For more information, please [email protected]. METHODS AND DESIGN ISSUES FOR NEXT GENERATION NETWORK-AWARE APPLICATIONS A Dissertation Submitted to the Graduate Faculty of the Louisiana State University and Agricultural and Mechanical College in partial fulfillment of the requirements for the degree of Doctor of Philosophy in The Department of Computer Science by Andrei Hut¸anu Computer Engineering Diploma, Politehnica University Bucharest, 2002 December 2009 Acknowledgments This work would not be possible without the help and support of my current and former colleagues at the Center for Computation & Technology (Louisiana State University) and Zuse Institute Berlin, Germany. I would like to thank all my collaborators from LONI, Masaryk University, MCNC, National Lambda Rail, Internet2, CESNET, the G-Lambda project team. It is a pleasure to thank my advisory committee (Gabrielle Allen, Edward Seidel, Tevfik Kosar, Daniel Katz) and the eaviv team: Jinghua Ge for developing the parallel renderer and helping to integrate it in eaviv and Cornelius Toole for integrating tangible interaction in eaviv and for designing the optimization algorithm for remote data access. I shall thank all my collaborators, in particular the following (not in any particular order): Ravi Paruchuri, Adam Yates, Steffen Prohaska, Andre Merzky, Jon MacLaren, Brygg Ullmer, Petr Holub, MiloˇsLiˇska, Erik Schnetter, Ludˇek Matyska, Peter Diener, Robert Kooima, Mehmet Balman, Ken- neth Welshons, Lonnie Leger, Gigi Karmous-Edwards, Jon Vollbrecht, Rajesh Sankaran, Thomas Sterling, Tomohiro Kudoh, Brian Cashman, Shalini Venkataraman, Jarek Nabrzyski, Steven R. Thorpe, Yufeng Xin, John Moore, Stephan Hirmer, Hartmut Kaiser, Stephen David Beck, Archit Kulshrestha, Sam White, Marc Steinbach, Hans-Christian Hege, Ralf K¨ahler. I thank Radu Chi¸sleag(see [Chi03]) and the ARSIP program1 for providing me the early oppor- tunity to start my international career. This work was supported by the EU GridLab project, the NSF Enlightened and CyberTools (NSF award #EPS-0701491) projects, the eaviv project (NSF award #OCI 0947825) and by the Center for Computation & Technology at LSU. This project has been supported by a research intent \Optical Network of National Research and Its New Applications" (MSMˇ 6383917201) and \Parallel and Distributed Systems" (MSMˇ 0021622419). 1http://www.arsip.com/ ii Portions of this research were conducted with high performance computational resources provided by the Louisiana Optical Network Initiative (http://www.loni.org/). Finally, I would like to thank my family and friends for their support throughout this journey. iii Table of Contents Acknowledgments . ii List of Tables . vi List of Figures . vii Abstract . ix Chapter 1: Introduction . 1 1.1 Commercial Internet . 3 1.2 Research Networks; Scheduling . 5 1.3 Applications . 7 1.4 Research Contribution . 8 1.5 Related Work . 9 Chapter 2: Distributed Interactive Visualization . 12 2.1 Related Work . 14 2.1.1 Visualization of Remote Data . 18 2.1.2 Video Streaming . 19 2.2 System Architecture and Design . 20 2.2.1 Remote I/O . 22 2.2.2 Rendering . 24 2.2.3 Video Streaming . 24 2.2.4 Interaction . 25 2.2.5 Deployment (Grid Computing and Co-Allocation) . 26 2.3 Results and Discussion . 26 2.3.1 iGrid 2005 System . 27 2.3.2 iGrid 2005 Results . 29 2.3.3 eaviv System . 29 2.3.4 eaviv Results . 31 2.4 Conclusions and Future Work . 36 Chapter 3: Remote Data Access . 38 3.1 Implementation Architectures for Remote Data Access . 41 3.1.1 Related Work . 42 3.1.2 Architectures Description . 44 3.1.3 Benchmarks and Performance Analysis . 48 3.1.4 Conclusions . 59 3.2 eavivdata System Architecture . 60 3.2.1 Related Work . 61 3.2.2 System Design . 63 3.2.3 Integrated System Architecture . 66 3.2.4 Benchmarks and Results . 71 iv 3.2.5 Application Integration . 74 3.3 Conclusions and Future Work . 76 Chapter 4: Transport Protocols . 78 4.1 TCP and Alternatives . 79 4.2 Application Options . 82 4.3 Benchmarks . 83 4.4 Conclusions . 86 Chapter 5: HD Classroom . 88 5.1 Overview . 89 5.2 Network . 89 5.3 Conclusions . 90 Chapter 6: Conclusions and Future Work . 92 Bibliography . 97 Vita . 113 v List of Tables 2.1 Data throughput and rendering scalability results. 32 2.2 Resolution effect on frame rate and video streaming bandwidth requirements . 33 2.3 Comparison of visualization systems features and performance . 34 3.1 Transfer time and computed throughput depending on message size using UDT and TCP............................................. 50 3.2 Operations throughput when multiple operations are executed using the five imple- mentation architectures . 51 3.3 Overhead per operation when a single operation is executed using the five implemen- tation architectures . 53 3.4 Execution time/operation of the asynchronous architecture with varying number of operations active at the same time, using TCP over WAN . 55 3.5 Connection set-up time for pool system . 56 3.6 Average data throughput for eavivdata and GridFTP with variable number of operations 73 4.1 Transfer rate (in Gbps) achieved over a shared 10 Gbps link (7.6 ms round-trip-time) using various transport protocol algorithms . 84 4.2 Transfer rate (in Mbps) achieved over a dedicated 10 Gbps link (149 ms round-trip- time) using various congestion control algorithms in the UDT library . 85 vi List of Figures 2.1 Illustration of a motivating visualization scenario . 13 2.2 Visualization pipeline showing the five different stages . 15 2.3 Visualization pipeline for remote data access: data server and filtering are on the server; rendering and display on the client . 19 2.4 Visualization pipeline for video streaming, all the stages except for the display are carried out on the server; the display is carried out on the client . 20 2.5 Architecture of the eaviv system providing three-way distributed visualization using video streaming and remote data access. 21 2.6 Remote data access in the eaviv visualization system . 23 2.7 Four stages in the progressive visualization process of a single dataset . 25 2.8 Illustration of the eaviv architecture used for the visualization server-based collabora- tive environment at iGrid 2005 . 28 3.1 The five implementation architectures positioned in a phase space with axes . 45 3.2 Synchronous implementation architecture . 46 3.3 Bulk implementation architecture . 46 3.4 Threaded/pool implementation architecture . 47 3.5 Pipeline implementation architecture . 48 3.6 Computed throughput of TCP and UDT depending on message size . 51 3.7 Maximum throughput (operations per second) when multiple operations are executed using the three implementation architectures with the highest throughput (Bulk, Pool and Pipeline) . 52 3.8 Overhead per operation when a single operation is executed using the three implemen- tation architectures with the lowest overhead (Synchronous, Bulk and Pipeline) . 54 3.9 Execution time/operation on the WAN when executing 300000 operations using the pool system . 57 vii 3.10 Execution time/operation when using the pipeline system over WAN . 58 3.11 Proposed system architecture combining the best features of the bulk and pipeline architectures . 60 3.12 Control channel design including encoding and decoding of the RPC request as well as encoding and decoding of the RPC response . 64 3.13 Complete eavivdata system diagram . 67 3.14 Average throughput for remote data access using single data streams and variable number of operations . ..