PC2 V10 PERFORMANCE TEST FRAMEWORK A Project Presented to the faculty of the Department of Computer Science California State University, Sacramento Submitted in partial satisfaction of the requirements for the degree of MASTER OF SCIENCE in Computer Science by Bhushan Arun Ladde FALL 2018 © 2018 Bhushan Arun Ladde ALL RIGHTS RESERVED ii PC2 V10 PERFORMANCE TEST FRAMEWORK A Project by Bhushan Arun Ladde Approved by: __________________________________, Committee Chair John Clevenger, Ph.D. __________________________________, Second Reader Doan Nguyen. Ph.D. ____________________________ Date iii Student: Bhushan Arun Ladde I certify that this student has met the requirements for format contained in the University format manual, and that this project is suitable for shelving in the Library and credit is to be awarded for the project. __________________________, Graduate Coordinator ___________________ Jinsong Ouyang, Ph.D. Date Department of Computer Science iv Abstract of PC2 V10 PERFORMANCE TEST FRAMEWORK by Bhushan Arun Ladde The PC2 Contest Control System provides for the ability of teams in a programming contest to submit runs – that is, programs which are attempts at solving a contest problem specified by the contest judges. In Version 10 (V10) of PC2, runs are submitted via web pages served by a webserver embedded in a module called the AppServer. On the server- side, AppServer automatically inserts a submitted run into a database. The Dispatcher queues run notifications and makes them available to another module called the Checker. Checker returns the result to the Dispatcher that whether run correctly solved the problem or not. The Dispatcher in turn forwards the result to the AppServer for subsequent display to the team which submitted the run. There is a Resource Manager which supports the ability of the Contest Administrator to define multiple different contests and to enter contest configuration information for each defined contest. The architecture allows only a single Dispatcher and a single Resource Manager to be running at any given time. Each V10 module runs on its own separate machine. v There are some limitations in the current system. A fundamental premise of the PC2 V10 system is that it is intended to handle large numbers of teams simultaneously participating in multiple different contests. The problem is that there is currently no mechanism for testing the performance of the system under such a large load and verify the performance capabilities of the V10 system. The proposed solution includes several important points. A. Build a large collection of programming contest problems and corresponding run submissions. B. Investigate a variety of available performance and load-testing tools. C. Implement a framework for submitting the collection of contest problems and corresponding submissions to PC2 V10 and monitoring the resulting performance using the selected tool(s). D. In the event that Jetty is found to be unable to support the desired load, investigate and propose mechanisms for substituting alternative webservers into V10 (e.g, using the nGinX webserver). _______________________, Committee Chair John Clevenger, Ph.D. _______________________ Date vi ACKNOWLEDGEMENT I would like to express my gratitude to my project supervisor Dr. John Clevenger for the useful comments, remarks, guidance and engagement through the learning process of this master’s project. I would also like to thank Dr. Doan Nguyen for taking time to review my report and giving me important feedback. Added to that, I would like to thank the Department of Computer Science at California State University, Sacramento for providing an opportunity to pursue my Masters Degree and guiding me all the way to become a successful student. vii TABLE OF CONTENTS Page Acknowledgement.………..............................................................................................vii List of Figures………………………………………………………………………….. x Chapter 1. INTRODUCTION……………………………………………………………………. 1 1.1 What is a Programming Contest?…………………………………………...........1 1.2 PC2…………………………………………………………………….................1 1.3 PC2 V10 Introduction……………………………………………………………3 1.3.1 Main Components of PC2 V10…………………………………………….4 1.3.2 Technology Stack for PC2 V10……………………………………………5 2. EXISTING PROBLEM AND PROPOSED SOLUTION…………………………....6 2.1 Performance Measurement Tool ………………………………………………...6 2.2 Architecture of JMeter…………………..……………………….........................8 3. TESTING TOOLS……………………….….............................................................10 3.1 Initial Excel Utils Project Set Up…...…………………………………………..10 3.2 JMeter Installation…………………………......................................................12 3.3 JMeter Test Plan Creation ……………………..……………...........................14 viii 3.4 Create Test User in JMeter ………………………………................................19 3.5 JVM(Java Virtual Machine) Monitor ………....................................................20 3.6 Postman ...………………...................................................................................22 4. IMPLEMENTATION STRATEGY ..………………………....................................23 4.1 Contest Data Creation Using PC2 V10 Tool.......................................................24 4.2 Steps to Implement PC2 V10 Test Plan..............................................................26 4.3 Measured Points..................................................................................................27 5. SIMULATION RESULTS…………………………..................................................31 5.1 Load Capacity Result............................................................................................31 5.2 Memory Dump Issue.............................................................................................32 5.3 Graphical Analysis of Performance……………………………….…………….32 5.4 Do Not Use XAMPP as a Database......................................................................33 6. FUTURE SCOPE…………………………................................................................36 6.1 Distributed Environment......................................................................................36 6.2 Testing on Different Platforms.............................................................................36 6.3 Security Test.........................................................................................................36 6.4 Alternative Webservers into V10…………........................................................37 References…..….……………………….........................................................................38 ix LIST OF FIGURES Figures Page 1. PC2 V10 Architecture………….….……………….....................................................4 2. JMeter Architecture Block Diagram….…....................................................................8 3. JMeter Installation Step 1………………...................................................................12 4. JMeter Installation Step 2………………...................................................................13 5. JMeter Initial Screen………………...........................................................................13 6. JMeter Test Plan Creation Step 1………………........................................................14 7. JMeter Test Plan Creation Step 2………………........................................................15 8. JMeter Test Plan Creation Step 3………………........................................................15 9. JMeter Test Plan Creation Step 4…………................................................................16 10. JMeter Test Plan Creation Step 5…………...............................................................17 11. JMeter Test Plan Creation Step 6…………...............................................................18 12. JMeter Test Plan Creation Step 7…………...............................................................18 13. JMeter Recorder Creation…..………….....................................................................19 14. JVM Test Monitor - 1….…..…………......................................................................21 15. JVM Test Monitor - 2….…..…………......................................................................22 16. JMeter PC2 V10 Login Test Plan Implementation.....................................................23 17. JMeter PC2 V10 Test Plan Implementation ……………..........................................24 18. Contest.yaml Configuration...………….....................................................................25 x 19. PC2 V10 Tool…………….....…………..................................................................26 20. Performance Graph of PC2 V10……………………………………………...……33 21. XAMPP…………...………......................................................................................34 xi 1 1. INTRODUCTION 1.1 What is a Programming Contest? A programming contest is an internet-based contest. There are two type of contests, first one is an individual programming contest where the individual programmer will get a problem statement in which he/she has to provide its individual solution, second one is group programming contest in which there are group of people who will get the problem statement. Users have to provide solution to that problem. A vast majority of problems appear in programming contest are based on mathematics and logical concepts. Once any user has submitted its solution then the automation process verifies the answer. The verification process is performed by the host machine, it is also called as judging the answer. There are a certain number of test cases to those problem