Paper, PDF, 2.07MB

MTAPerformanceComparison: sendmailvs.postfixon*BSD

BradKnowles SeniorConsultantforSnow,BV [email protected]

http://www.shub-internet.org/brad/papers/mtacomparison/

• Goal • Meta Information – Hardware Used – Software Tested – Tools Used – Methodology • Test Results • Conclusions

• Show you what it looks like to do MTA performance tuning – On as many *BSD platforms as I could • Focus on the process, not the numbers or expected/desired outcome – You get to see (most) everything, warts and all

• Basically, chapter 3 from the book sendmail Performance Tuning by Nick Christenson – See http://www.jetcafe.org/~n pc/book/sendmail/

• Hardware Used • Software Tested • Benchmark Tools • Methodology

• NetBSD – Twinhead “Twinstation 5G” (Sun SPARCstation 5 clone) – Not UltraSPARC, but the ancient SPARC 5 • OS: NetBSD 1.6-RELEASE • CPU: microsSPARC-II @ 110MHz • RAM: 32MB real, 384MB virtual • NIC: On-board “Lance” 10Base-T Ethernet & SBus QuadFastEthernet

• FreeBSD – Compaq Armada 4131T • OS: FreeBSD 4.6.2-RELEASE • CPU: Pentium 133 • RAM: 48MB real, 384MB virtual • NIC: Asanté FriendlyNET AL1011 “Prism2” 802.11b WiFi PCMCIA • HD: 10GB IBM Travelstar 20GN – 4200 RPM – 12ms avg. seek

• MacOS X (Part 1) – PowerBook G3 “Pismo” • OS: MacOS X 10.2.1 • CPU: PowerPC G3 @ 400MHz • RAM: 1GB real, 2GB virtual • NIC: Apple AirPort 802.11b WiFi • HD: 48GB IBM Travelstar 40GH – 5400 RPM – 12ms avg. seek

• MacOS X (Part 2) – PowerBook G4 (DVI) • OS: MacOS X 10.2.2 • CPU: PowerPC G4 @ 800MHz • RAM: 1GB real, 2GB virtual • NIC: Apple AirPort 802.11b WiFi • HD: 40GB Toshiba MK4018GAS – 4200 RPM – 12ms avg. seek

• Sendmail 8.12.6 – ftp://ftp.sendmail/org/pub/sendmail/sendmail.8.12.6.tar.gz • Postfix 1.1.11 – http://www.postfix.org/ftp-sites.html • Exim 4.10 – Tried every way I could think of, but could not get it configured to relay back to an IP address or port on the same machine

• Disk Benchmarking – PostMark from Network Appliance • http://www.netapp.com/tech_library/3022.html – Bonnie++ from Russell Coker • http://www.coker.com.au/bonnie++/ • SMTP Benchmarking – smtp-source/smtp-sink from Wietse Venema • Included with postfix distribution – Postal from Russell Coker (with smtp-sink) • http://www.coker.com.au/postal/

• Get disk subsystem baseline by performing benchmarking – For each platform & filesystem configuration • PostMark – See http://www.shub-internet.org/brad/FreeBSD/postmark.html • Bonnie++ – Configured to test similar parameters » File sizes, numbers, & subdirectories according to the four PostMark tests

• For FreeBSD, we only test turning on or off “soft updates” – We do not test (turned on by default) • vfs.vmiodirenable • dirprefs • dirhash – For data on all the possible combinations of the above, see “Recent Filesystem Optimisations in FreeBSD” at http://www.usenix.org/events/usenix02/tech/freenix/dowse.html – For comparison data of Journaling vs. Soft Updates, see “Journaling Versus Soft Updates: Asynchronous Meta-data Protection in File Systems” at http://www.usenix.org/publications/library/proceedings/usenix2000/ general/seltzer.html 2002-11-16 Copyright©2002byBradKnowles 16 Methodology

• For MacOS X – We test only • HFS+ – Apple’s Extended Hierarchical File System • MFS – Memory-based filesystem created using RamBunctious 2.0

• For MacOS X – We do NOT test • UFS – IMO, Apple broke it while porting HFS+ to Mach – Many applications do not recognize it, in addition to MacOS 9 itself – “Classic” requires that boot device be HFS+ and MacOS 9 must be installed before MacOS X • HFS+ with Journaling – Too new, introduced with MacOS X Server 10.2.2 – Initial reports indicate ~20% performance loss

• For MacOS X – Does not have a native “poll.h” or “libpoll” (required by postal) • Install manually from source http://www.clapper.org/software/poll/ • Install using “fink” – MacOS X package/port manager based on Debian “dselect” – Unable to test Bonnie++ • Need further assistance from the author to find out why it is crashing

• Perform SMTP benchmarking – For each hardware & software configuration, test • Test direct source->sink connection – Demonstrate maximum possible performance • Test source->relay->sink – Show relay performance

– Note • Testing only relaying, not local delivery • Test generated on, relayed through, and terminated on “loopback” network – Eliminate all question of network or NIC performance 2002-11-16 Copyright©2002byBradKnowles 20 TestResults

• Chart data only – PostMark table data available from http://www.shub-internet.org/brad/FreeBSD/postmark.html – Bonnie++ table data not yet available – smtp-source/smtp-sink data not yet available • `hostname` must be legal (I.e., does not end with “.”) – Postal data not yet available

• Dumb “Driver” Scripts – My “runpostmark” script available at http://www.shub-internet.org/brad/runpostmark – My “runbon” script available at http://www.shub-internet.org/brad/runbon – My “runsmtp” script available at http://www.shub-internet.org/brad/runsmtp

2002-11-16 Copyright©2002byBradKnowles 22 TestResults: PostMarkSampleRun % cd /var/tmp % postmark PostMark v1.5 : 3/27/01 pm>set number 1000 pm>set transactions 50000 pm>show Current configuration is: The base number of files is 1000 Transactions: 50000 Files range between 500 bytes and 9.77 kilobytes in size Working directory: /var/tmp Block sizes are: read=512 bytes, write=512 bytes Biases are: read/append=5, create/delete=5 Using Unix buffered file I/O Random number generator seed is 42 Report format is verbose. pm>run

2002-11-16 Copyright©2002byBradKnowles 23 TestResults: PostMarkSampleRun Creating files...Done Performing transactions...... Done Deleting files...Done Time: 497 seconds total 492 seconds of transactions (101 per second)

Files: 26014 created (52 per second) Creation alone: 1000 files (500 per second) Mixed with transactions: 25014 files (50 per second) 24868 read (50 per second) 24880 appended (50 per second) 26014 deleted (52 per second) Deletion alone: 1028 files (342 per second) Mixed with transactions: 24986 files (50 per second)

90 PostmarkTest#1:1000Files&50kTransactions 8,000.00 2,500 80 7,000.00

70 2,000 6,000.00 60 5,000.00 1,500 50 EastDataRead 4,000.00 DataWritten

KB/Sec Trans/Sec 40 Trans/Sec West 1,000 3,000.00 North 30 2,000.00 20 500 1,000.00 10 0.00 0

UFS TMPFS 0 ODS/R0 ODS/R5 P3+DPT U5+UFS U5+Log NFS/F330NFS/F630 P3/Linux 2xP3+UFS PPro+SOFT 2xP3+SOFT2xP3+MFS U5+TMPFSPBG3/HFS+PBG3/MFSPBG4/HFS+PBG4/MFS 1st Qtr 2nd Qtr 3rd Qtr 4thCompaq+UFSQtrCompaq+Soft Platform

90 PostmarkTest#2:20kFiles&50kTransactions 3,000.00 700 80

600 2,500.0070

500 2,000.0060

400 50 EastDataRead 1,500.00 DataWritten

KB/Sec Trans/Sec

Trans/Sec West 40 300

1,000.00 North 30 200

500.0020 100 10 0.00 0

UFS TMPFS 0 ODS/R0 ODS/R5 P3+DPT U5+UFS U5+Log NFS/F330 NFS/F630 P3/Linux 2xP3+UFS PPro+SOFT 2xP3+SOFT 2xP3+MFS U5+TMPFSPBG3/HFS+ PBG3/MFSPBG4/HFS+ PBG4/MFS 1st Qtr 2nd Qtr 3rd Qtr 4th QtrCompaq+UFSCompaq+Soft Platform

90 PostmarkTest#3:20kFiles&100kTransactions 2,500.0080 700

600 2,000.0070 60 500

1,500.00 400 50 EastDataRead DataWritten Trans/Sec KB/Sec 300 West 1,000.0040 Trans/Sec North 30 200 500.00 20 100

100.00 0

TMPFS P3+DPT U5+Log ODS/R0 ODS/R5 P3/Linux U5+UFS 0 NFS/F330 NFS/F630 2xP3+UFS 2xP3+MFS U5+TMPFS PPro+SOFT 2xP3+SOFT PBG3/HFS+ PBG3/MFSPBG4/HFS+PBG4/MFS Compaq+UFSCompaq+Soft 1st Qtr 2nd Qtr Platform 3rd Qtr 4th Qtr

90 PostmarkTest#4:20kFiles,100kTransactions,&100subdirectories 6,000.00 1,400 80

1,200 5,000.0070

1,000 4,000.0060

800 c 50 e EastDataRead 3,000.00 DataWritten

KB/Sec Trans/Sec

Trans/S West 40 600

2,000.00 North 30 400

1,000.0020 200 10 0.00 0

0 U5+UFS U5+Log P3/Linux 2xP3+UFS 2xP3+SOFT 2xP3+MFS U5+TMPFS PBG3/HFS+ PBG3/MFS PBG4/HFS+ PBG4/MFS 1st Qtr 2nd Qtr 3rd Qtr 4thCompaq+UFS QtrCompaq+Soft Platform

90 PostmarkTest#1:1000Files&50kTransactions 3,500.00 1,200 3,317.76 80 3,174.40

3,000.00 1,000 70 980

2,500.0060 800

2,000.0050 East c e DataRead 1,638.40 600 WestDataWritten 40 1,566.72

KB/Sec Trans/Sec

1,500.00 Trans/S 485 492 North 30 400 1,000.0020

489.66 511.62 200 500.00 10 332.02 346.92 149 268.32 280.35 235.40 245.96 82 72

0.000 0 PBG3/HFS+1st Qtr PBG3/MFS 2ndPBG4/HFS+Qtr PBG4/MFS3rd Qtr Compaq+UFS 4thCompaq+SoftQtr Platform

90 PostmarkTest#2:20kFiles&50kTransactions 3,000.00 700 80 2,816.00666

600 2,500.0070

60 500 2,000.00

50 400 East c e DataRead 1,495.04 1,500.00 349 DataWritten 40 1,392.64 West

KB/Sec Trans/Sec 300 Trans/S North 30 1,000.00 200 20 737.57

500.00 10 100 187.09 154.75 125.63 82.00 29 99.14 36 73.51 66.57 38.95 15 23 0.000 0 PBG3/HFS+1st Qtr PBG3/MFS 2ndPBG4/HFS+Qtr PBG4/MFS3rd Qtr Compaq+UFS 4th Compaq+SoftQtr Platform

90 PostmarkTest#3:20kFiles&100kTransactions 2,500.00 700 80 2,334.72

613 70 600 2,000.00

60 500 1,628.16

1,500.00 50 400 East c e DataRead 40 1,177.60 WestDataWritten

KB/Sec Trans/Sec

310 Trans/S 300 1,000.00 North 30 819.89 20 200 500.00 10 100 129.65 135.72 90.20 94.42 100.21 31 33 42.68 61.35 69.72 15 22 0.000 0 PBG3/HFS+1st Qtr PBG3/MFS 2ndPBG4/HFS+Qtr PBG4/MFS3rd Qtr Compaq+UFS 4th Compaq+SoftQtr Platform

90 PostmarkTest#4:20kFiles,100kTransactions,&100subdirectories 2,500.00 700 80

2,129.92 600 70 574 2,000.00

60 500

1,484.80 1,500.00 50 400 East c e DataRead 40 WestDataWritten

KB/Sec Trans/Sec 1,015.93 300 Trans/S North 1,000.0030 277 708.74 20 200 500.00 10 100 110.68 93.79 77.21 65.43 24 28 38.81 55.63 40.19 57.61 14 13 0.000 0 PBG3/HFS+1st Qtr PBG3/MFS 2ndPBG4/HFS+Qtr PBG4/MFS3rd Qtr Compaq+UFS 4th Compaq+SoftQtr Platform

% bonnie++ -d /var/tmp/bonnie -n 1:10k:0 -m compaq-soft -r 48m Writing with putc()...done Writing intelligently...done Rewriting…done Reading with getc()...done Reading intelligently...done start 'em...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.02d ------Sequential Output------Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP compaq-soft 300M 2407 67 4584 39 2052 21 1948 53 3644 18 57.8 3 ------Sequential Create------Random Create------Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files:max /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 1:10:0 402 31 +++++ +++ 832 49 564 46 +++++ +++ 1746 99 compaq-soft,300M,2407,67,4584,39,2052,21,1948,53,3644,18,57.8,3,1:10:0,402,31, +++++,+++,832,49,564,46,+++++,+++,1746,99

450 60%

402 400 50% 50% 350 47% 340 331

300 40%

250 32% FilesCreated/Sec 31% 31% 30% %CPUBusy 200 181 23% % CPU Busy Files Created/Sec 150 150 20%

100 76 10% 50

0 0% compaq-ufs/1:10:0 compaq- compaq- compaq- compaq- compaq- ufs/20:10:0 ufs/20:10:0:100 soft/1:10:0 soft/20:10:0 soft/20:10:0:100 Platform-Filesystem/Test

800 50%

704 45% 69544% 700 43% 67043% 40% 600 35%

500 30% FilesRead/Sec 400 25% %CPUBusy % CPU Busy

Files Read/Sec 20% 300

15% 200 165 10% 10% 100 5%

0 0% compaq-ufs/20:10:0 compaq-ufs/20:10:0:100 compaq-soft/20:10:0 compaq-soft/20:10:0:100 Platform-Filesystem/Test

2,500 70%

2,06260% 60% 2,000 1,857 52% 53% 50% 48% 49%

1,500 40% FilesDeleted/Sec %CPUBusy 30% 1,000 % CPU Busy Files Deleted/Sec 25% 832 20%

500 10% 215 188 103 0 0% compaq-ufs/1:10:0 compaq- compaq- compaq- compaq- compaq- ufs/20:10:0 ufs/20:10:0:100 soft/1:10:0 soft/20:10:0 soft/20:10:0:100 Platform-Filesystem/Test

600 60% 564 57%

500 50% 48% 46%

400 40% 350 330 35% 33% FilesCreated/Sec 300 30% %CPUBusy

24% % CPU Busy

Files Created/Sec 201 200 20% 154

100 78 10%

0 0% compaq-ufs/1:10:0 compaq- compaq- compaq- compaq- compaq- ufs/20:10:0 ufs/20:10:0:100 soft/1:10:0 soft/20:10:0 soft/20:10:0:100 Platform-Filesystem/Test

90 6%

80 81 80 79 5% 5% 5% 5% 70

60 4%

50 46 FilesRead/Sec 3% %CPUBusy 40 % CPU Busy Files Read/Sec 30 2% 2%

20 1% 10

0 0% compaq-ufs/20:10:0 compaq-ufs/20:10:0:100 compaq-soft/20:10:0 compaq-soft/20:10:0:100 Platform-Filesystem/Test

2,000 120% 1,890 1,811 1,800 1,746

99% 100% 1,600

1,400 80% 1,200 FilesDeleted/Sec 1,000 60% %CPUBusy 53% 53%

800 % CPU Busy Files Deleted/Sec 42% 40% 600

400 20% 200 170 407% 397% 0 0% compaq-ufs/1:10:0 compaq- compaq- compaq- compaq- compaq- ufs/20:10:0 ufs/20:10:0:100 soft/1:10:0 soft/20:10:0 soft/20:10:0:100 Platform-Filesystem/Test

% sudo smtp-sink 127.0.0.1:25 1024 & % foreach I (1 2 3 4 5 6 7 8 9 10) foreach? /usr/bin/time smtp-source -l 10240 -f [email protected] \ -t [email protected] -m 1000 -s 512 127.0.0.1:25 foreach? end 8.06 real 0.22 user 1.65 sys 9.03 real 0.26 user 1.45 sys smtp-source: fatal: connect: Operation timed out 32.00 real 0.17 user 1.68 sys 13.08 real 0.19 user 1.43 sys 10.98 real 0.31 user 1.28 sys 9.02 real 0.24 user 1.52 sys 8.04 real 0.25 user 1.57 sys smtp-source: fatal: lost connection while reading server greeting 16.51 real 0.19 user 1.61 sys 10.99 real 0.19 user 1.53 sys 16.52 real 0.25 user 1.50 sys

250.00 9.00

8.05 8.00 208.29 7.58 200.00 7.00

6.00 154.16 150.78 150.00 5.00 Msgs/Sec. Std.Dev. 4.00 100.00

84.20 Standard Deviation Messages per Second 79.10 3.00 68.19

2.00 50.00 39.98

0.98 19.98 1.00 0.64 9.98 0.42 0.46 5.00 0.00 0.15 0.10 0.07 0.00 0.00 512 256 128 64 32 16 8 4 2 1 NumberofThreads

250.00 45.00 237.28

40.00 38.23 200.00 35.78 188.79 35.00 32.92 32.42 29.34 155.99 30.00 150.00 25.00 123.55 Msgs/Sec. 21.14 Std.Dev. 20.00 97.38 100.96 101.69 19.76 100.00 93.87 93.57 96.08 Standard Deviation

Messages per Second 79.20 15.00

10.00 50.00 9.34 39.83

5.10 5.00 2.58 2.56 1.24 0.00 0.00 16384 8192 4096 2048 1024 512 256 128 64 32 16 8 NumberofThreads

30.00 4.50 28.39 4.23 4.00 25.00 3.50

20.00 3.00

16.62 2.50 Msgs/Sec 15.00 Std.Dev. 2.00 Standard Deviation Messages per Second 10.00 9.00 1.50

1.00 5.00 0.50

0.17 0.00 0.02 0.00 4 2 1 NumberofThreads

% sudo smtp-sink 127.0.0.1:25 1024 & % postal -m 10k -p 64 -c 1 127.0.0.1 postal.user - time,messages,data(K),errors,connections,SSL connections 23:05,7651,41262,0,7715,0 23:06,7795,41656,0,7795,0 23:07,7847,42305,0,7847,0 23:08,7616,41528,0,7616,0 23:09,7670,41659,0,7670,0 23:10,7413,39922,0,7413,0 23:11,7765,41886,0,7765,0 23:12,7752,41960,0,7752,0 23:13,7423,39835,0,7423,0 23:14,7475,39697,0,7475,0 23:15,7458,39981,0,7458,0 23:16,7462,40332,0,7462,0 23:17,7281,39294,0,7280,0 23:18,7408,40022,0,7409,0 23:19,7595,40882,0,7586,0 23:20,7737,41991,0,7746,0 23:21,7646,41122,0,7646,0 ^C

% sudo smtp-sink 127.0.0.1:25 1024 & % postal -m 10k -p 128 -c 1 127.0.0.1 postal.user - time,messages,data(K),errors,connections,SSL connections 23:27,8283,43988,0,8411,0 23:28,8965,48165,0,8965,0 23:29,8952,48100,0,8952,0 23:30,8785,46803,0,8785,0 23:31,7948,42811,0,7933,0 Server timed out on read. Server timed out on read.

[ .. .. Deletia .. .. ]

Server timed out on read. Server timed out on read. 23:32,8016,43367,28,8059,0 23:33,8960,47748,0,8960,0 23:34,9054,48507,0,9022,0 23:35,8737,47175,0,8769,0 23:36,9057,48586,0,9057,0 23:37,8800,46722,0,8800,0 23:38,8933,48092,0,8933,0 23:39,8969,48155,0,8955,0 23:40,8984,48576,0,8998,0 ^C 2002-11-16 Copyright©2002byBradKnowles 45 TestResults: PostalSampleRun

% sudo smtp-sink 127.0.0.1:25 1024 & % postal -m 10k -p 256 -c 1 127.0.0.1 postal.user - Usage: postal [-m maximum-message-size] [-p processes] [-l local-address] [-c messages-per-connection] [-r messages-per-minute] [-a] [-b [no]netscape] [-[z|Z] debug-file] [-s ssl-percentage] smtp-server user-list-filename conversion-filename

Postal Version: 0.61

% postal -m 10k -p 32 -c 1 127.0.0.1 postal.user - time,messages,data(K),errors,connections,SSL connections Can't connect to 127.0.0.1 port 25. Can't connect to 127.0.0.1 port 25. Can't connect to 127.0.0.1 port 25. Can't connect to 127.0.0.1 port 25. Can't connect to 127.0.0.1 port 25.

160.00 400.00 145.77378.02 140.00 350.00 126.46 120.00 300.00

100.00 250.00

79.69 Msgs/Sec 80.00 200.00 Std.Dev. 164.94 60.00 150.00 Standard Deviation Messages per Second 39.92 40.00 100.00

19.97 20.00 50.00 9.98 27.51 10.06 0.00 3.67 2.74 0.00 128 64 32 16 8 4 NumberofThreads

• When testing real MTAs, you need to watch more than just the output from the testing program – Not unusual for MTAs to accept mail faster than they can process it, especially under heavy load • Also watch – Mail queue size • mailq | grep '[0-9][0-9]:[0-9][0-9]' | wc -l – CPU load • iostat 5 – Memory utilization • vmstat 5

• Make sure that you modify /etc/postfix/main.cf to be suitable – Need to boost • local_destination_concurrency_limit • default_destination_concurrency_limit • Check /etc/postfix/master.cf to make sure there are no artificial limits on processes like smtp, smtpd, bounce, cleanup, rewrite etc…

• Unfortunately, Apple broke iostat – Still mostly okay for what we want to watch (CPU user %, system %, and idle %) • Apple doesn’t even provide vmstat – vm_stat is not the same program, although it is similar

70.00 0.90 0.84 0.85 0.80 60.00 57.22 52.26 0.70 50.00 0.64 0.60 39.50 40.00 0.50 Msgs/Sec Std.Dev. 30.00 0.40 Standard Deviation Messages per Second 19.98 0.29 0.30 20.00 0.20 9.92 10.00 0.13 0.10 0.08 4.99

0.00 0.00 32 16 8 4 2 1 NumberofThreads

12.00 1.40 10.95 10.36 1.20 10.00 1.14

1.00 8.00

6.41 0.80 Msgs/Sec 6.00 Std.Dev. 0.60 Standard Deviation

Messages per Second 4.00 0.48 0.40 0.33 2.00 0.20

0.00 0.00 4 2 1 NumberofThreads

12.00 1.80

10.65 1.60 1.55 10.00 9.40 1.40

8.00 1.20

1.00 0.95 6.14 Msgs/Sec 6.00 0.94 Std.Dev. 0.80 Standard Deviation

Messages per Second 4.00 0.60

0.40 2.00 0.20

0.00 0.00 4 2 1 NumberofThreads

12.00 350.00 11.08

303.20 300.00 10.00

250.00 8.00 7.53

200.00 Msgs/Sec 6.00 Std.Dev. 150.00 Standard Deviation

Messages per Second 4.00 100.00

2.00 50.00

17.72 0.00 0.00 compaq-ufs compaq+soft Platform-Filesystem

20.00 20.00 18.59 18.00 18.19 18.00

16.00 16.00 14.89 14.36 14.00 14.00

12.00 12.00 Msgs/Sec 10.00 9.69 10.00 Std.Dev. 8.00 8.00 Standard Deviation Messages per Second 6.00 6.00 5.37 4.97 4.75 4.00 4.00

2.00 1.92 2.00 1.47 0.68 0.00 0.37 0.00 32 16 8 4 2 1 NumberofThreads

20.00 1.40 18.01 18.00 1.20 1.16 16.00

14.00 1.00

12.00 0.80 9.92 Msgs/Sec 10.00 Std.Dev. 0.60 8.00 Standard Deviation Messages per Second 6.00 5.36 0.42 4.97 0.38 0.40 4.00 0.20 2.48 0.20 2.00 1.25 0.12 0.10 0.00 0.00 32 16 8 4 2 1 NumberofThreads

6.00 35.00

31.04 5.02 30.00 5.00 4.56

25.00 4.00 3.92 3.65 20.00 Msgs/Sec. 3.00 Std.Dev. 2.49 15.00 14.08 Standard Deviation

Messages per Second 2.00 9.67 10.00

1.00 5.00 4.17 4.04

0.00 0.00 16 8 4 2 1 NumberofThreads

7.00 90.00 6.61 82.20 80.00 6.00 5.91 72.11 70.00

5.00 62.62 60.00 4.10 4.00 50.00 Msgs/Sec. Std.Dev. 3.00 2.81 40.00

2.35 Standard Deviation Messages per Second 30.00 2.00 20.37 20.00

1.00 10.93 10.00

0.00 0.00 16 8 4 2 1 NumberofThreads

• Problem – Local syslog can cause excessive load on busy mail system • Solutions – Configure syslog to write to logs asynchronously • Perhaps via a “–” (dash) before the filename – Run syslog on a central log server, across the network • Also allows you to perform more in-depth log processing, without interfering with mail services • Generally logs are much easier to manage centrally • Can also combine with other central syslog processing applications – E.g., Addamark Log Management System

• Problem – Classic syslog uses UDP, can lose 75% or more of packets sent across the network • Solution – Replace classic syslog with nsyslog, ssyslog, or syslog-ng and use TCP instead of UDP • Depending on your particular requirements for hardware, OS, environment, etc….

70.00 3.00 2.80 60.89 60.00 2.50 52.04 52.59 50.00 2.06 2.00

40.00 1.67 36.32 Msgs/Sec 1.50 1.41 Std.Dev. 30.00 26.78 1.23 Standard Deviation

Messages per Second 1.00 19.30 20.00 0.80

9.96 0.50 10.00 0.47 0.35 4.98

0.00 0.00 128 64 32 16 8 4 2 1 NumberofThreads

20.00 8.00

18.00 7.10 17.20 7.00 16.00 6.00 14.00 5.50 5.00 12.00 9.92 Msgs/Sec 10.00 4.00 Std.Dev. 8.00 3.00 Standard Deviation Messages per Second 6.00 5.24 4.97 2.00 4.00 1.76 2.50 1.00 2.00 1.24 0.36 0.00 0.10 0.12 0.00 32 16 8 4 2 1 NumberofThreads

6.00 30.00 5.64

5.00 24.92 25.00 4.46

4.00 20.00 3.63

Msgs/Sec. 3.00 15.00 Std.Dev. 2.30 11.43 Standard Deviation

Messages per Second 2.00 10.00 8.44

1.00 5.00

2.31 0.00 0.00 8 4 2 1 NumberofThreads

• This process can be difficult and time- consuming to get right, but it pays off • Graphs & charts can help a great deal – But they don’t tell the whole story • Software – Postfix is a lot faster than even I had thought – Wasn’t able to get sendmail tuned for the performance I’m confident it can deliver – Benchmarking tools (esp. Bonnie++ and postal) need more work to be useful

• Use the Scientific Method – Try to set aside personal bias – Trust only what you can prove – Process • Generate Hypothesis • Generate test for Hypothesis • Test Hypothesis – Prove or Disprove • If disproven, modify or generate new Hypothesis 2002-11-16 Copyright©2002byBradKnowles 65 RelatedWork

• Nick Christenson – sendmail Performance Tuning book http://www.jetcafe.org/~npc/book/sendmail/ – “Performance Tuning Sendmail Systems” slides http://www.jetcafe.org/~npc/doc/performance_tuning.pdf • Brad Knowles – “Sendmail Performance Tuning for Large Systems” slides http://www.shub-internet.org/brad/papers/sendmail-tuning/

• Matthias Andree http://www-dt.e-technik.uni-dortmund.de/~ma/postfix/vsqmail.html http://www-dt.e-technik.uni-dortmund.de/~ma/postfix/bench2.html • Nakamura Motonori http://www.kyoto.wide.ad.jp/mta/eval1/eindex.html • Ciprian Ascante http://www.benchmarks.dmz.ro/index.php?topic=LinMSB