Current Problems and Tools Support for Testing Distributed Systems Anh Thu Le A dissertation submitted for the degree of Bachelor of Applied Science with Honours in the Department of Information Science at the University of Otago, Dunedin, New Zealand October 2006 1 Acknowledgments First of all, I would like to thank my supervisor, Dr. Maryam Purvis for her valuable supervision, suggestions, and encouragements during the course of my thesis. A special thank goes to Tony Bastin Roy Savarimuthu for being a thoughtful and accessible supporter. I thank the University of Otago Language Centre in Dunedin for making a number of systems available for testing. I also thank Professor Paul Strooper at the University of Queensland in Australia for his willingness in helping me to implement the ConAn testing tool. Finally, I would like to thank my parents for their precious support during my four years undergraduate education in Dunedin, New Zealand. 2 Abstract Testing is a critical activity in designing and implementing software and computer systems, especially distributed systems (Eickelmann & Richardson, 1996). In addition, validating distributed systems is a complex process, and it requires the knowledge of “the communication taking place and the order of communication messages exchanged between the individual components to avoid deadlocks or livelocks during runtime” (Ulrich, Zimmerer, Chrobok-Diening, 1999, p.2). This dissertation attempts to briefly emphasise the role of testing in distributed systems. In addition, it addresses few common issues in building distributed systems together with a number of solutions to these issues. The main focus of this dissertation is on three main features of distributed systems that often challenge the programmers and testers, which are concurrency, scalability, and fault tolerance. For the first two features, a number of open source testing tools are chosen to compare and contrast the strengths and weaknesses of each tool. A number of testing techniques are applied to show the importance of the third feature, namely, the fault tolerance. Besides, the importance of each feature is highlighted by testing a number of sample distributed systems. 3 Table of Contents Acknowledgments ............................................................................................................. 2 Abstract.............................................................................................................................. 3 Table of Contents .............................................................................................................. 4 Lists of Figures .................................................................................................................. 6 List of Tables ..................................................................................................................... 7 1. Introduction............................................................................................................... 8 2. Background ............................................................................................................. 10 2.1 Definition of a Distributed System ................................................................... 10 2.2 Characteristics of a Distributed System............................................................ 11 3. Research Objectives................................................................................................ 15 4. Testing Distributed Systems................................................................................... 16 4.1 Unit testing........................................................................................................ 20 4.2 Integration testing ............................................................................................. 20 4.3 Regression testing ............................................................................................. 21 4.4 System testing ................................................................................................... 21 4.5 Acceptance testing ............................................................................................ 22 5. Issues and Solutions................................................................................................ 23 5.1 Openness........................................................................................................... 23 5.2 Latency.............................................................................................................. 24 5.3 Global clock...................................................................................................... 24 5.4 Security ............................................................................................................. 25 5.5 Heterogeneity.................................................................................................... 26 6. Concurrency Testing .............................................................................................. 27 6.1 Concurrency in Distributed System.................................................................. 27 6.2 Systems: Cathy’s Online Car Store, CalME, and E-shopping.......................... 29 6.2.1 Cathy’s Online Car Store.......................................................................... 29 6.2.2 CalME....................................................................................................... 34 6.2.3 E-Shopping ............................................................................................... 39 6.3 Tools: GroboTestingJUnit and ConAn ............................................................. 42 4 6.3.1 GroboTestingJUnit (version 1.1.0) ........................................................... 42 6.3.2 ConAn (version 5.0).................................................................................. 43 6.3.3 Summary................................................................................................... 44 7. Scalability Testing................................................................................................... 45 7.1 Scalability in Distributed System...................................................................... 45 7.2 Systems: OLC Assets, DCRR, and Employee Directory ................................. 47 7.2.1 OLC Assets ............................................................................................... 48 7.2.2 DCRR........................................................................................................ 51 7.2.3 Employee Directory.................................................................................. 53 7.3 Tools: TestMaker, OpenSTA, and httperf ........................................................ 56 7.3.1 TestMaker (version 4.4)............................................................................ 56 7.3.2 OpenSTA (version 1.4.3).......................................................................... 57 7.3.3 httperf........................................................................................................ 58 7.3.4 Summary................................................................................................... 58 8. Fault Tolerance Testing.......................................................................................... 60 8.1 Fault Tolerance in Distributed System ............................................................. 60 8.2 Systems: CalME and OLC Assets .................................................................... 61 8.2.1 CalME....................................................................................................... 62 8.2.2 OLC Assets ............................................................................................... 62 9. Conclusion and Future Work ................................................................................ 63 10. References................................................................................................................ 64 11. Appendix.................................................................................................................. 70 11.1 Appendix 1: DeleteAndOrderCarTest.java....................................................... 70 11.2 Appendix 2: CreateGroupTest1.java and CreateGroupTest2.java ................... 71 11.3 Appendix 3: CreateAppTest1.java and CreateAppTest2.java .......................... 73 11.4 Appendix 4: EditExistingCategoriesTestA.java and ........................................ 75 EditExistingCategoriesTestB.java ................................................................................ 75 11.5 Appendix 5: OLC Asset XSTest Scalability Index........................................... 77 11.6 Appendix 6: DCRR XSTest Scalability Index.................................................. 78 11.7 Appendix 7: Employee Directory XSTest Scalability Index............................ 79 5 Lists of Figures Figure 1: A Distributed System Organized as Middleware Figure 2: Traditional Testing Process Figure 3: Test and Code Cycle in XP Figure 4: Waterfall Model Figure 5: V Software Life-cycle Model Figure 6: Spiral Model Figure 7: Message Transmission Time Figure 8: User adds a Car - Cathy’s Online Car Store Figure 9: User Views/Deletes/Orders a Car - Cathy’s Online Car Store Figure 10: Scenario 1 – Cathy’s Online Car Store Figure 11: Scenario 2 – Cathy’s Online Car Store Figure 12: Scenario 3 – CalME Figure 13: Scenario 4 – CalME Figure 14: User Enters a New Category - E-Shopping