00 Multi-Threaded Servers: Design with reusable components, Performance Measurements and Analysis Gunidas Somadder B.E. A thesis submitted to the Faculty of Graduate Studies and Research in partial fMïlhent of the reqairements for the degree of Master of Engineering Ottawa-Carleton Institute for Electrical Engineering Faculty of Engineering Department of Systems and Cornputer Engineering Carleton University, Ottawa, Ontario, Canada February 12,1997 O 1997, Gurudas Somadder National Library Bibliothèque nationale of Canada du Canada Acquisitions and Acquisitions et Bbliographic Services services bibliographiques 395 WefIington Street 395, rue Wellington Ottawa ON KIA ON4 Ottawa ON KIA ON4 Canada Canada Ywr61B vmréffnnicd Our Isle Nom nirthence The author has granted a non- L'auteur a accordé une licence non exclusive licence allowing the exclusive permettant à la National Library of Canada to Bibliothèque nationale du Canada de reproduce, loan, distri'bute or seIl reproduire, prêter, distribuer ou copies of this thesis in microform, vendre des copies de cette thèse sous paper or electronic formats. la forme de microfiche/nlm, de reproduction sur papier ou sur format électronique. The author retains ownership of the L'auteur conserve la propriété du copyright in this thesis. Neither the droit d'auteur qui protège cette thèse. thesis nor substantid extracts fkom it Ni la thèse ni des extraits substantiels may be p~tedor othemise de celle-ci ne doivent être imprimés reproduced without the author's ou autrement reproduits sans son permission. autorisation. canada Abstract The first object of this research was to develop an efficient approach to build multi- threaded servers by using reusable components provided in the object-oriented fhmework Adaptive Communication Environment (ACE) descrii in [Sch94Aprl]. The thesis presents the pattern-based design, implementation and perfofmance measurements of different dtematives of multi-threaded servers for distributed applications. The multi- threaded approaches are compared based on their performance under different loads and system architectures, their execution overheads and cotz~umptionof shared system resources (such as memory and I/O ports). It is shom that for cenain system architectures, server multi-threading can improve the ovexall performance of the system by allowing for a more efficient use of the hardware resources. The impact of assigning priorities to the service requests (as opposed to the clients) and a rationale for establishg such priorities depending on the workload characteristics are also discussed. Another objective of the research was to fkd a way to exploit the design pattern used m object-oriented systems for pedormance analysis. Design pattems desmi recurriflg solutions to common problems fiom various application domains, and provide a more abstract view of the system behavior as they concentrate on the main interactions between the system components. It is shown that patterns can guide developers in choosing what to measure, in instrumenting the code for measurements and in interpreting the results. It is emphasized that an abstract behavioral model, such as the one provided by design patterns, is especially usenil in systems built with reusable components, both for understanding what the reused software does and its contriion to the overall performance. Acknowledgrnents 1 would like to express my deep gratitude to my supervisor, Dr. D. C. Petriu, for her vaiuable guidance, advice and encouragement throughout this research. 1 would aIso like to thank Dr. CM. Woodside and Dr. J. Rolia of the Real-Time and Distnbuted Systems (.R4DS) Group of the Department of Systems and Compter Engineering for their support and advice diiring the research. Many thanks to my colleagues, Istabrak, Fahim, Hesham, Marc, Alex, Cheryl and Leslie for their support, advice, help and encouragement during the course of this thesis. The f?nancial assistance of the Telecommunications Research Institute of Ontario (TRIO), through its opemting gant program, is gratefully acknowledged. Finally 1 would like to thank my parents, sisters and Iast but not the least, my brother-in- laws, for their invaluable support, patience, understanding and encouragement which made this thesis possible. Contents 1 Introduction 2. Threads and Threading Models Introduction .............................................................................................10 SoIaris 2.5 Multi-threading Architecture ................................................. 12 2.2.1 Processes and Threads ............................................................... 13 2.2.2 Solaris Multi-threaded Architecture ...........................O............. 14 Threading Models ............,,...........................œ...........œœœœ...œœ......o.œœœ.........1 7 Thread Priorities and Tbread-based Measmements ................................ 18 2.4.1 Thread Prionties ........................................................................ 18 2.4.2 ThTead-based Measurements ..................................................... 19 Benefits/Drawbacks of Thread-based Concurrent Rogramming ............ 20 2.5.1 Benefits of using Thread-based Concment Programming ....... 20 2.5-2 Drawbacks of using Thread-based Concurrent Programming ... 2 1 3 . The Adaptive Communication Environment 24 Introduction .......................................................................................... The ACE Tookit ................................................................................... 3.2.1 C* Wrappers ........................................................................ 3 .2.2 Class Categories and Frameworks ..................................O...... Design Patterns: Concept and Use .., ...... .. ......... .. 3.3. 1 The Reactor Pattern ................................................................ 3.3.2 The Acceptor Pattern .............................................................. 3.3.3 The Connector Pattern ............................................................ 3.3.4 The Active Object Pattern ...................................................... 4. Client/Server Design and Implementation 4.1 Introduction ........................................................................................... 4.2 Client,Server Architecture fiom an ACE Viewpoint ., ......................... 4.2.1 The Client ................................................................................ 4.2.2 The Server e............................................................................... 4.2.2.1 Single Threaded Server .. ..............................W......e. 4.2.2.2 The Thread-per-Reguest Server .............o......eO....... 4.2.2.3 The Thread-per-Client Server ................................ 4.2.2.4 Tbe Thread-Pool Server .................W.....................e. 4.3 System Architecture ....O. ...... .. .......~...mœ~.e.e.~œo~~.~...qo...m.~*..~~~.o.~o....~...~..... 4.3.1 Single-Threaded C/SArchitecture ..............................O............ 4.3.2 Thread-per-Request C/S Architecture .................... .................. 4.3.3 Thread-per-Client CIS Architecture ................. ........................ 4.3.4 Thread-Po01 C/S Architecture ................................................... 64 4.4 A Use Case Map Representation of the System Execution Cycle .........* 64 4.5 Systems with Layered Servers ... .....,...l...........w.......m.*m*....*..*............... 66 Experimental Setup and Measurements 68 Introduction ............................................................................................. 68 Experimental Setup .....,w..........,..oa........***.e*.*......*...*....*.......*...............69 5.2.1 Experiment ControUer and Data Logger ................. ..... ..*....*..... 70 5.2.2 DECALS Architecture .............................................*.................71 Measuement Results and Performance Analysis ...,........................ ... 72 Pattern-guided Mea~u~ementsand Analysis ... ....w..,.........m.œ..*....*...e**.... 79 5.4.1 Frequently executed patterns ..................... ............................... 80 5 -4-1 .1 Reactor Pattern ......*.......................*............*.*..........80 5.4.1.2 Active Object Pattern .........,......... .......................... 5.4.2 Detailed Measurement Results and Performance Anaiysis ....... Cornparison between the Thread-per-Client and the Thread-Pool Semer Models ........................................................................................ 6.3 Service Rior@ ............, ...,....*...............*...................................... 99 6.4 Experimental Setup ,........,...,....,*..........~.*................................... 1O0 6.5 Measuement Rdtsand Andysis for Service Priority ..........................* 101 vii 6.5.1 Measurement Red&for the Pure Server Arc.tecture ..................... 102 6.5.1.1 Effect of Changing Service Time Ratios ............O.....106 6-5-12 Impact of Changing the Number of Tbreads in the Thread-Pool ........................................................ 6.5.2 Measurement Results for the Layered Server Architecture ........ 109 6.6 Measurement Rdtsand Analysis for Class Priorify ............... .............. 1 12 7. Conchions 115 7.1 Summaxy ....................... ................. .......................... ........................... 115 7.2 FutureWork ....................... ........... ........................................................ 117 Appendix A Notations for Use Case Maps and UML 118 k 1 Use Case Map