UKRAINIAN CATHOLIC UNIVERSITY BACHELOR THESIS Performance and scalability analysis of Java IO and NIO based server models, their implementation and comparison Author: Supervisor: Petro KARABYN Oleg FARENYUK A thesis submitted in fulfillment of the requirements for the degree of Bachelor of Science in the Department of Computer Sciences Faculty of Applied Sciences Lviv 2019 ii Declaration of Authorship I, Petro KARABYN, declare that this thesis titled, “Performance and scalability analy- sis of Java IO and NIO based server models, their implementation and comparison” and the work presented in it are my own. I confirm that: • This work was done wholly or mainly while in candidature for a research de- gree at this University. • Where any part of this thesis has previously been submitted for a degree or any other qualification at this University or any other institution, this has been clearly stated. • Where I have consulted the published work of others, this is always clearly attributed. • Where I have quoted from the work of others, the source is always given. With the exception of such quotations, this thesis is entirely my own work. • I have acknowledged all main sources of help. • Where the thesis is based on work done by myself jointly with others, I have made clear exactly what was done by others and what I have contributed my- self. Signed: Date: iii UKRAINIAN CATHOLIC UNIVERSITY Faculty of Applied Sciences Bachelor of Science Performance and scalability analysis of Java IO and NIO based server models, their implementation and comparison by Petro KARABYN Abstract This thesis focuses on the scalability and performance analysis of the web applica- tions built with the IO packages present in the Java programming language. One of the objectives is to compare the thread-oriented and event-driven models and exam- ine the implications of using them in Java IO and NIO-based systems. Three types of concurrency strategies are proposed for dealing with blocking and non-blocking IO and then applied in practice to build a chat engine. Results of the experiments show that in the case of active long-lived connections the difference in throughput between Java IO and NIO is marginal. However, NIO has an edge when it comes to large amounts of simultaneous connections and offers minimal memory consump- tion. iv Acknowledgements I would like to express words of gratitude to my project advisor, Oleg Farenyuk, for guidance throughout my student years. Your professionalism and deep competence in the wide area of subjects have always inspired me to put a lot of effort into learn- ing. Your valuable advice has helped a great deal and directed me in my professional career path. Also, I would like to acknowledge current and past members of the Applied Sciences Faculty and everyone who at some point contributed to the growth and success of the Computer Science program. The work you did will continue to give back to society for the years to come. v Contents Declaration of Authorship ii Abstract iii Acknowledgements iv 1 Introduction1 2 Background4 2.1 Thread-based model.............................5 2.2 Event-driven model..............................7 2.3 Reactor pattern................................9 2.4 The C10K Problem.............................. 11 3 Related works 12 3.1 Duality argument............................... 13 3.1.1 Message-oriented system...................... 13 3.1.2 Procedure-oriented system..................... 14 3.1.3 Differences and a modern look................... 15 3.1.4 Conclusions.............................. 15 3.2 SEDA...................................... 16 3.2.1 High-level overview......................... 16 3.2.2 Types of stages............................ 17 3.2.3 Anatomy of a stage.......................... 18 3.2.4 Conclusions.............................. 19 3.3 Thousands of Threads and Blocking I/O.................. 20 4 Scalable I/O in Java 21 4.1 Java IO and NIO................................ 22 4.1.1 IO streams vs. NIO channels and buffers............. 22 4.1.2 Blocking vs. non-blocking IO.................... 23 4.1.3 Selectors and design implications................. 24 4.1.4 Summary............................... 25 4.2 I/O concurrency strategies.......................... 26 4.2.1 Multithreaded Blocking I/O.................... 26 4.2.2 Single Threaded Non-Blocking I/O................ 27 4.2.3 Multithreaded Non-Blocking I/O................. 29 4.3 I/O trends in enterprise software...................... 30 5 Experiments 32 5.1 Implementation................................ 33 5.2 Test setup.................................... 34 5.3 Real world simulation test.......................... 35 5.3.1 Metrics................................. 36 vi 5.4 High load test................................. 38 5.4.1 Metrics................................. 39 5.5 Analysis.................................... 41 6 Conclusion 44 Bibliography 46 vii List of Figures 2.1 Thread-based server design. This figure illustrates a thread-based model for a server where each incoming client request is processed in a separate thread. The request related computations and internal I/O operations are done by the allocated thread which sends the re- sponse to the client upon completion. This model is also referred to as thread-per-request..............................6 2.2 Event-driven server design. The main thread processes the events from the event queue passing them to the event handler that would execute the associated code. The events can be requests from clients or any other notifications from external systems. Note, that this is a conceptual diagram with a single event handler displayed, in prac- tice, there can be multiple event handlers or one per each type of the expected events.................................8 2.3 Reactor pattern UML............................. 10 3.1 Conceptual illustration of a message-oriented system. Processes have message ports to receive messages in queue and their own data. A process operates data contained by another process by means of ex- plicit message passing mechanism..................... 13 3.2 Conceptual illustration of a procedure-oriented system. Global data is shared across the processes. Monitor encapsulates the data and pro- vides methods for synchronized access. In order to manipulate the global data, a process has to first acquire the mutex. Any other pro- cess can’t change the state of the global data until the mutex is re- leased. This way the shared data is manipulated one process at a time and sychronization is achieved........................ 14 3.3 A high-level overview of the SEDA processing scheme.......... 16 3.4 SEDA types of stages............................. 17 3.5 Anatomy of a SEDA stage. Consists of an incoming queue for the events, an event handler, a dynamically sized thread pool, and a con- troller responsible for scheduling, monitoring the load, and adjusting the parameters at runtime........................... 18 4.1 Stream oriented IO.............................. 23 4.2 Buffer oriented IO............................... 23 4.3 Thread-per connection design schema of standard IO-based application 24 4.4 Design schema of a NIO application with a Selector........... 25 4.5 Multithreaded Blocking I/O......................... 26 4.6 Single Threaded Non-Blocking I/O..................... 28 4.7 Multithreaded Non-Blocking I/O...................... 29 5.1 Test setup.................................... 34 5.2 Chat server latency.............................. 36 viii 5.3 Comparison of chat server latency at 10 000 concurrent clients..... 36 5.4 Chat server CPU utilization......................... 37 5.5 Chat server heap usage............................ 37 5.6 Chat server threads.............................. 38 5.7 Chat server throughput........................... 39 5.8 Chat server outgoing messages....................... 39 5.9 Chat server CPU utilization at high load.................. 40 5.10 Chat server heap usage at high load.................... 40 5.11 Chat server threads at high load....................... 41 ix List of Tables 3.1 Duality mapping................................ 15 3.2 Duality mapping: modern look. Adapted to thread-based and event- driven concepts (Li and Zdancewic, 2007)................. 15 4.1 Differences between Java IO and NIO................... 25 x List of Abbreviations CPU Central processing unit FD File descriptor GUI Graphical user interface IOCP I/O completion ports IO Input/Output JDK Java Development Kit JSR Java Specification Request OS Operating System MOS Message-oriented system POS Procedure-oriented system Req/s Requests per second RAM Random access memory SEDA Staged event-driven architecture TCP Transmission Control Protocol xi Definitions Process an independent active program managed by the operating system that is not sharing the same address space with other programs. Thread a smallest unit of execution that can be managed by a scheduler and may share memory and resources with its peer threads within a process, often re- ferred to as a lightweight process. Message an item of data that is sent to a specific destination. Event a signal emitted by a component upon reaching a given state or any other action detected by a program. Event-driven programming a programming paradigm in which the execution of a program is determined by the events it listens and reacts to. Synchronous execution a mode in which a program is executed line-by-line in a predetermined sequence of tasks, where each subsequent task has to wait be- fore the previous