Performance Testing Tools Jan Bartoň 30/10/2003
Total Page:16
File Type:pdf, Size:1020Kb
CESNET technical report number 18/2003 Performance Testing Tools Jan Bartoň 30/10/2003 1 Abstract The report describes properties and abilities of software tools for performance testing. It also shows the tools comparison according to requirements for testing tools described in RFC 2544 (Benchmarking Terminology for Network Intercon- nection Devices). 2 Introduction This report is intended as a basic documentation for auxiliary utilities or pro- grams that have been made for the purposes of evaluating transfer performance between one or more PCs in the role of traffic generators and another one on the receiving side. Section 3 3 describes requirements for software testing tools according to RFC 2544. Available tools for performance testing are catalogued by this document in section 4 4. This section also shows the testing tools com- patibility with RFC 2544 and the evaluation of IPv6 support. The summary of supported requirements for testing tools and IPv6 support can be seen in section 5 5. 3 Requirements for software performance testing tools according to RFC-2544 3.1 User defined frame format Testing tool should be able to use test frame formats defined in RFC 2544 - Appendix C: Test Frame Formats. These exact frame formats should be used for specific protocol/media combination (ex. IP/Ethernet) and for testing other media combinations as a template. The frame size should be variable, so that we can determine a full characterization of the DUT (Device Under Test) performance. It might be useful to know the DUT performance under a number of conditions, so we need to place some special frames into a normal test frames stream (ex. broadcast, management, routing update frames . ). These modifiers could have a significant impact on an ability of a router to forward data frames. The testing tool should be able to use a random destination address to simulate multiple streams of data. For more details about recommended frame formats see Appendix C included with RFC 2544. 3.2 Verifying received frames The receiver should discard any frames received during a test run that are not actual forwarded test frames (ex. management frames, routing update frames . ). It should verify the length of received frames and report the number of dropped, duplicated frames, frames that were received out of order and the number of gaps in the received frame numbering sequence. The testing tool should verify that the all of the routing updates (see above in section User defined frame format) were processed by the DUT. 3.3 Bidirectional traffic Real network traffic is not in a single direction. To test the bidirectional perfor- mance of a DUT, we need a testing tool, which can be run with the same data rate in each direction. The sum of the data rates should not exceed the theoretical limit for the media. 3.4 Setting inter-frame time gap All the tests should be performed with both steady state traffic and with traffic consisting of repeated bursts of frames. Because of needs to determine the minimum interval between bursts, which the DUT can process with no frame loss, we need to set the inter-frame time gap between defined frames (bursts). 4 Tools Each tool is defined as follows: Description A description of the tools construction, and the implementation methodology of the tests. Automation What steps are required to complete the test? What human intervention is required? CESNET technical report number 18/2003 2 Settings possibilities Summary of program settings possibilities. Availability How do you retrieve this tool and get more information about it? Required Environment Compilers, OS version, etc. required to build and/or run the associated tool. RFC 2544 compatibility Summary of requirements to testing tools according to RFC 2544 as defined in section 3 3. References A list of publications relating to the tool, if any. 4.1 DBS 1.1.5 4.1.1 Description DBS (Distributed Benchmark System) is aiming to give performance index with multi-point configuration and also in order to measure changes of throughput. It measures the performance of entire TCP functions in various operational environments. DBS has the capability of both measuring and analyzing TCP performance more in details. DBS is able to evaluate the three TCP control mechanisms - flow control, retransmission control and congestion avoidance control. The DBS can generate various situations where the three controls are working together. The DBS can also generate UDP traffic for more realistic benchmarking. Figure 1: DBS architecture CESNET technical report number 18/2003 3 The DBS is composed of three programs. Figure shows an overview of the DBS System Structure. The dbsc is a control program to handle monitoring program launched on observed hosts. dbsd is a program for sending and receiving data among observed hosts. These two programs are used for actual benchmarking. The dbs view is used for data analysis. The details of these programs are described below: <h4>dbsc: DBS controller (Controlling Host)</h4> The DBS controller is a program controlling the experiment of TCP/UDP data transfer. Controller reads commands from a command file, then it asks the DBS daemons to start data transfer experiments, and after receiving results from the daemons, the DBS controller saves them into the local files. <h4>dbsd: DBS daemon (Measuring Host)</h4> The DBS daemon is a daemon program that is launched on the experimen- tal hosts. It sends and receives network traffic according to the commands instructed by the DBS controller. <h4>dbs view: DBS viewer</h4> The DBS viewer is a program for analysis data which is gathered by the DBS controller. It draws graphs to reveal the transitions of sequence numbers, changes of throughput, changes of delay times or other performance indexes. If measuring host has only one network interface card, traffic of command/result and measured traffic are transferred on the same network. This may influence the measurement results. To avoid this influence, DBS controls command/result and measured traffic are not transferred at the same time. DBS implementation assumes that clocks on all the hosts participating to the benchmark are synchronized. 4.1.2 Automation Commands of execution are driven by a command file. Format of the command file is as follows. When multiple data streams are transferred in the same test, many configurations should be written in the same file. # First Configuration { sender {’configurations of sender’; ’sending traffic pattern {s}’;} receiver {’configurations of receiver’;’receiving message pattern {s}’;} ’configuration of the connection’; } CESNET technical report number 18/2003 4 # Second Configuration { sender {’configurations of sender’; ’sending traffic pattern {s}’;} receiver {’configurations of receiver’;’receiving message pattern {s}’;} ’configuration of the connection’; } . Format of ’sending traffic pattern s’ pattern { data size, message size, interval, wait time; data size, message size, interval, wait time; . ; } Figure 2: Patern parameters meaning The traffic is modeled as a sequence of data chunks called ”frames”. The size of each frame may vary. Each frame may consist of several messages. A single message is defined that it can be transferred in a single UDP datagram. If a frame is longer than the UDP maximum transfer unit (64KB), the frame is split into several messages. Between frames, there is a time gap called ”wait time” which implies the application overhead. The preparation time of each frame can be treated as this wait time. Moreover, this model controls the frame intervals. This frame interval can be used for modeling of an application level rate control. Command File Sample CESNET technical report number 18/2003 5 # Sample file { sender { hostname = host1; port = 0; send_buff = 65535; recv_buff = 65535; mem_align = 2048; pattern {2048, 2048, 0.0, 0.0} } receiver { hostname = host2; port = 20001; recv_buff = 65535; send_buff = 65535; mem_align = 2048; pattern {2048, 2048, 0.0, 0.0} } file = test1; protocol = TCP; start_time = 0.0; end_time = 30; send_times = 2048; } See http://www.kusa.ac.jp/ yukio-m/dbs/dbs man.html for more information about constructing command file. 4.1.3 Settings possibilities send/receive buffer size modification setting the TCP no delay option specify the starting time of data transfer for each connection specify frame size and inter-frame time gap specify test duration sending complicated traffic patterns CESNET technical report number 18/2003 6 Through ”tcp trace” function, several TCP internal information such as TCP/IP pseudo-headers, TCP congestion window size, round trip time, retransmission timeout and other values in the TCB structure can be obtained. 4.1.4 Availability See http://www.ai3.net/products/dbs1 for details of precise OS versions sup- ported, and for download of the source code. Current implementation supports BSDI BSD/OS, FreeBSD, NetBSD, Linux, mkLinux, SunOS, IRIX, Ultrix, Digital UNIX, NEWS OS, HP-UX. 4.1.5 Required Environment C language compiler, UNIX-style socket API support. 4.1.6 RFC 2544 compatibility User defined frame format No Support Verifying received frames Checks only sequence numbers of received frames. Bidirectional traffic It could be made of two single traffics between specified hosts running at the same time in the opposite direction. Setting inter-frame time gap Full Support 4.1.7 References Performance and Control of Network Systems, Proceedings of SPIE, Volume 3231, November 1997(English) ”DBS: a powerful tool for TCP performance evaluations”2 Informating Processing