A Performance Study of Protocols Used in a Print on Demand Server
Total Page：16
File Type：pdf, Size：1020Kb
A performance study of protocols used in a print on demand server SAM SAM Master’s Thesis at ICT Examiner: Fredrik Kilander Supervisor: Fredrik Kilander Industrial supervisor: Joe Armstrong TRITA-ICT-EX-2015:16 iii Abstract A performance evaluation of different protocols, both textual and binary based, is done by implementing a simple print-on-demand service as the test bench. The measurements are done with respect to the serialization method, a method defining the procedure to encode structured data to a stream of bytes. The result of this evaluation may play a key in forming a decision regarding which protocol to utilize for mobile networks or networks with high demand. The evaluated protocols are Protocol Buffers, BERT, ASN.1, JSON, and XML. The result shows that Protocol Buffers had best performance for most of the tests and that XML had worst performance for all tests. iv Sammanfattning En prestandautvärdering av olika protokoll, både text och binärbaserade, har gjorts genom att implementera en enkel print-on-demand service som testbänk. Mätningarna är gjorda med avseende på serialiseringsmetoden, en metod som definerar tillvägagångsättet att koda strukturerad data till en ström av bytes. Resultatet av denna evaluering kan spela en roll i bestämmandet av vilken protokoll att använda för mobila nätverk eller nätverk med hög efterfrågan. De evaluerade protokollen är Protocol Buffers, BERT, ASN.1, JSON och XML. Resultatet visar att Protocol Buffers presterade bäst för de flesta testen och XML presterade sämst för alla test. Contents Contents v 1 Introduction 1 1.1 Motivation . 1 1.2 Problem Statement . 2 1.3 Scope . 2 1.4 Purpose . 3 1.5 Sustainable development . 3 1.6 Methodology . 3 1.7 Thesis outline . 3 2 Background 5 2.1 Related work . 5 2.2 Environment . 7 2.2.1 Erlang . 7 2.2.2 0MQ . 9 2.2.3 LibHaru . 10 2.2.4 ZLib . 10 2.3 Protocol Description Languages . 11 2.3.1 Protocol Buffers . 12 2.3.2 BERT . 14 2.3.3 ASN.1 . 15 2.3.4 JSON . 17 2.3.5 XML . 18 3 Methodology 21 3.1 Literature study . 21 3.2 Implementation . 23 3.3 Measurements and analysis . 24 4 Implementation overview 27 4.1 The 0MQ connection . 28 4.2 The client . 28 4.3 The server . 28 v vi CONTENTS 5 Code generation 31 5.1 Overview . 31 5.2 LibHaru API documentation . 31 5.3 Structured API specification . 32 5.4 Generated code . 33 5.4.1 Port code generation . 34 5.4.2 Erlang module template . 36 5.4.3 Protocol buffers code generation . 36 5.4.4 Bert code generation . 37 5.4.5 ASN.1 code generation . 38 5.4.6 JSON code generation . 39 5.4.7 XML code generation . 40 6 Test and measurement programs 41 6.1 Overview . 41 6.2 Measurements on the client side . 42 6.3 Measurements on the server side . 43 6.4 Test data sets . 43 7 Analysis tools 45 7.1 Overview . 45 7.2 From data to charts . 46 8 Empirical evaluation 49 8.1 Test design . 49 8.2 Test implementation and setup . 49 8.3 Test results and analysis . 51 8.3.1 Encoding performance . 51 8.3.2 Decoding performance . 54 8.3.3 Size . 56 8.3.4 Round trip time . 58 8.3.5 Overall Performance . 60 8.3.6 Speed and size performance . 62 9 Conclusions 65 9.1 Summary . 65 9.2 Conclusion . 65 9.3 Discussion . 66 9.4 Future Work . 67 References 69 A All test results 73 A.1 Encoding performance . 73 A.2 Decoding performance . 75 CONTENTS vii A.3 Size . 77 A.4 Round trip time . 79 A.5 Overall Performance . 81 A.6 Speed and size performance . 83 Chapter 1 Introduction This chapter introduces the reader to the thesis and presents the motivation. This work investigates the performance of a network service by sending data (or messages) between a client and a server and measure how the performance varies in respect to changes of the serialization mechanism (used to pack the messages) and the size of the messages. Messages, have specific data, structured according to a specification with predefined syntactic rules. The result of this evaluation may play a key in forming a decision regarding which protocol to utilize for which specific purpose. For instance, to determine which of the evaluated protocols in this study are most fit for mobile networks or networks under load. The protocols that have been evaluated are Protocol Buffers, BERT, ASN.1, JSON and XML. For evalua- tion of the different protocols, a simple print-on-demand service is implemented for the various protocols. A measurement of performance is made and the results are evaluated. The network service in this case creates PDF-files from messages sent from a client over a network. The service has two parts, a server and a client, where the former receives messages from network sockets and creates PDF-files, and the latter is the application that sends messages of what to include in the PDF-files. We measure the time to encode and decode messages as well as message density, and also the throughput of the system measured in PDF pages per second. 1.1 Motivation In wireless communication, the bandwidth is limited. An example of this is LTE 4G (Long-term evolution fourth generation), a standard for wireless communication of high-speed data for mobile phones and data terminals. An LTE 4G cell have a limited capacity of active users with a peak of 300MB/s [20]. This means that a user connected to such a network have a theoretical download speed of 300MB/s, however if several users are sharing this bandwidth the speed per user would de- 1 2 CHAPTER 1. INTRODUCTION crease. Another important aspect to consider is that most mobile network operators sell limited data plans. It is therefore important, when using a wireless network, to be frugal with what you send. In many wired services it is instead more important to package messages quickly than to send small packets. An example of this is a bank where the load is high with many transactions being made in a short time. Given the motivations above, which protocol to use is important. Thus the project providers at Ericsson proposed to evaluate protocol description languages for a PDF-generator as a simple practical implementation of this problem. 1.2 Problem Statement This thesis evaluates packet size and packaging speed of various combinations of protocols and data set types sent over a network. This is to determine the different optimal combinations and answer the following questions for the protocols Protocol Buffer, BERT, ASN.1 JSON and XML: • Which of the packaging mechanisms are the fastest regardless of data set? • Which of the packaging mechanisms produce the smallest packet size regard- less of data set? The protocols are used to send messages locally in one computer with and without compression for further evaluation of the protocols compactness. A print on demand service is the test bench and the following performance metrics are studied: • The number of PDF pages created per second • The round-trip time of sending a message and receiving an acknowledgment • The time to serialize a message • The time to deserialize a message • The size of a message 1.3 Scope Given there are many protocols, this thesis focuses on 5 protocols as representatives of most protocols. The measurements that are made focuses on the method of when a message is encoded/decoded and the relation between number of bytes and the actual content of the message. As this thesis is interested in the different serialization mechanisms, it is not relevant to send the messages in a parallel system. Meaning, every next message will only be sent when an acknowledgment is received from the server. 1.4. PURPOSE 3 1.4 Purpose The purpose of this performance study is to answer the questions presented in sec- tion 1.2 by evaluating a variety of protocols. The author of this thesis hopes of giving a good understanding to the reader of the difference between packaging mechanisms in a practical sense. The author would also like to convey the importance of choos- ing the right protocol when developing a service with a limited network or when using a network with great load. 1.5 Sustainable development The effect of a performance study on protocols could be in many ways beneficial for sustainable development. From an economical aspect, it will most likely be to an advantage for mobile network users to use services with minimal data transfer. It is also ecologically beneficial to apply protocols that use less CPU-power and smaller packets as these two factors have great impact on battery usage. Of course, this does not only apply to mobile phones. The internet of things(IoT) has extended the usage of mobile networks tremendously over the years. Using a more compact and faster protocol for internet devices can have a great influence on both economical and ecological aspects. And naturally, the cheaper the usage of a device becomes, the more people can use it. 1.6 Methodology A short investigation of different tools and libraries was made. After defining what libraries to use, a print on demand service was implemented together with a spe- cific protocol to create the test bench. The purpose of the experiment is to un- derstand how the protocols perform in a practical sense and the purpose of the PDF-generating service is to have a test bench that can handle a wide variety of data types.