Cloudharmony Performance Benchmark: Select High-Performing Public Cloud to Increase Economic Benefits

CloudHarmony Performance Benchmark: Select High-Performing Public Cloud to Increase Economic Benefits

White Paper

What You Will Learn • Google reported that a 400-milisecond delay in presenting search results, less than the blink of an eye, reduced the Even small differences in cloud performance can affect costs, number of searches per user by .59 percent customer satisfaction, and time to market. • AOL observed that users who experience the fastest page This white paper, intended for enterprises evaluating public loads view 50 percent more pages per visit than those who cloud services, summarizes the results of an independent experience the slowest page loads comparison of public cloud service performance conducted • When Shopzilla accelerated page load times from by CloudHarmony in March 2011: approximately seven to two seconds, revenue increased by 7–12 percent • Forty-four tests measured performance for CPU-intensive • AutoAnything, a web-based supplier of auto accessories tasks, disk I/O, memory I/O, interpreted-language and parts, measured customer satisfaction before and after programming, and encryption and encoding. improving web performance for page loads from 14 to • The tests compared the performance of a cloud compute seven seconds. Customers were 29 percent more likely to instance to a baseline compute instance operating on Intel buy on their first visit, and 38 percent more likely to return Xeon 5500-series processors. to the site • The Savvis Symphony Virtual Private Data Center (VPDC) performed at or near the top of the CloudHarmony CloudHarmony Benchmark: Empirical benchmark in all five categories. The service platform consists Comparison of Public Cloud Performance of Cisco UCS® B200 M2 Blade Servers using two Intel Xeon Savvis, a leading global provider of cloud infrastructure for 5570 4-core 2.93-Ghz processors and 96 GB RAM enterprises, asked CloudHarmony to conduct a benchmark In the Cloud, Performance Matters comparing the performance of leading public cloud providers. CloudHarmony, which is not affiliated with or funded by any Organizations of all sizes are moving some or all of their cloud provider, tested the following services: business computing to the cloud, attracted by favorable economics. One benefit is that usage-based billing lowers • Savvis Symphony Virtual Private Data Center (VPDC) costs compared to purchasing infrastructure that sometimes • Amazon EC2 sits idle. But not all clouds are equal, and the extent of the • IBM SmartCloud Enterprise savings depends upon infrastructure performance. That is, the more work the cloud platform can do in a unit of time, the • Terremark vCloud Express lower the costs. • OpSource Cloud

In addition, even tiny performance increases can stimulate • Rackspace incremental sales on eCommerce sites. Examples reported by • BlueLock 1 Microsoft include: • GoGrid

• Bing found that a two-second slowdown in search results • SoftLayer decreased revenue per user by 4.3 percent

1. Microsoft Tech-Ed, May 2011

White Paper

After assessing 44 performance metrics, including both CPU Results synthetic and real-world performance testing, CloudHarmony aggregated the results into five performance categories: What benchmark measured: Based on the Amazon EC2 Compute Unit (ECU), the • CPU, relevant to compute-intensive tasks such as Monte CloudHarmony Compute Unit Carlo simulations used in the financial services industry (CCU) compiles 19 different CPU benchmarks • Memory I/O, relevant to caching systems and memory stores Most relevant for: Compute-intensive tasks such • Disk I/O, relevant to database and web server as big-data analysis, email • Interpreted language processing, relevant to development routing, and multi-threaded in the cloud applications

• Encryption and encoding, relevant to security and Business impact: Lower computing costs and privacy protection faster application response times The tests revealed how much better or worse the compute instance performance compared to a baseline compute Baseline: Intel Xeon Processor 1.2 GHz instance. If a baseline score is 100, then a compute instance 2007 [Verifying model #] earning an aggregate result of 125 is 25 percent faster than the baseline. An instance earning an aggregate score of 75 is See company comparisons for CPU results in Figures 1 25 percent slower. and 2.

White Paper

40 Large Cong Cores Memory (GB)

Savvis 8 32 35 35 34 Amazon 16 23

IBM 16 32 30 Rackspace 8 30 27 Terremark 8 16 26 25 Medium Cong Cores Memory (GB) 23

Savvis 4 8 20 Amazon 1 3.75

IBM 4 8 15 Rackspace 4 4 13

Terremark 2 4 10 10 10 Small Cong Cores Memory (GB)

6 6 Savvis 1 2 5 5 4 4 Amazon 1 1.7 3

1 Figure 1. CPU Results IBM 2 2 0 Savvis VPDC Ranked Highest in Rackspace 4 2 Savvis VPDC AWS EC2 IBM SmartCloud Rackspace Terremark Enterprise vCloud Express CPU Performance for 8-, 4-, and Terremark 1 1 CCU 2-Core Configurations Large Cong Medium Cong Small Cong

14 1 Core Cong Cores Memory (GB)

Savvis 1 2 12 Amazon 1 3.75

IBM 2 2 10 Rackspace 4 2

Terremark 1 1 8

Figure 2. CPU Results – Single Core 0 Savvis VPDC More Than Doubled Savvis VPDC AWS EC2 IBM SmartCloud Rackspace Terremark Enterprise vCloud Express Performance of Next Best Performance CCU in Single-Core Configuration

White Paper

Disk I/O Results

What it measures: How fast compute instances read 160 from and write to disk

140 Most relevant to: Database and web server workloads GoGrid and Rackspace tested with local storage. All other vendors tested with o-instance storage such as SAN.

160 Business impact: Lower computing costs because

of faster execution; faster project 140 completion

120 Baseline (100 points): Dual-processor Intel Xeon Processor 98 E5506 2.13 GHz, with eight cores 100 91 total and four 15,000 RPM SAS drives in a RAID 1+0 configuration 80

See how each cloud provider ranked in disk results in 60 Figure 3.

0 Savvis VPDC AWS EC2 IBM SmartCloud Rackspace Terremark Enterprise vCloud Express IOP

OpSource had best IOP in large class with 198.26 and Small Medium Large best in medium class with 190.24. GoGrid had the best in the small class with 150.72.

Figure 3. Disk Results Savvis VPDC Ranked First in Large and Medium Configurations

White Paper

Memory I/O Results

Large Cong Cores Memory (GB)

What it measures: How fast the cloud server reads and Savvis 8 16

writes data to memory Amazon 16 23

IBM 16 32 Caching systems and memory-based Most relevant to: Rackspace 8 30

data stores Terremark 8 16

Medium Cong Cores Memory (GB) Business impact: Faster application response time Savvis 4 8

Amazon 1 3.75 Baseline (100 points): Dual Intel Xeon Sever 5504 Quad- IBM 4 8 Core with 2.00 GHz processors and Rackspace 4 4

48 GB DDR3 ECC RAM Terremark 4 8

Small Cong Cores Memory (GB) See how each cloud provider ranked in memory results in Savvis 1 2

Figure 4. Amazon 1 1.7

IBM 2 2

Rackspace 4 2

Terremark 1 1

160

145.0 145.7

140

120

100

0 Savvis VPDC AWS EC2 IBM SmartCloud Rackspace Terremark Enterprise vCloud Express MIOP

Large Cong Medium Cong Small Cong

Figure 4. Memory Results Savvis Outperformed Other Services for Memory I/O in 4-Core and Single-Core Configurations, Tying for 8-Core Configuration

White Paper

Interpreted Language Programming Results What it measures: Performance of four languages commonly used to develop server- 180 side software applications, including Java, Ruby, Python, and PHP 160

Most relevant to: Software development 140

Business impact: Faster time to market, more time for 120 quality testing 100 Baseline (100 points): Dual-processor Intel Xeon Processor 80 E5506 2.13 GHz (8 cores total) with four 15,000 RPM SAS drives 60 configured in hardware-managed

RAID 1+0 40

Figure 5 displays each cloud provider’s results in interpreted 20 language programming.

0 Savvis VPDC AWS EC2 IBM SmartCloud Rackspace Terremark Enterprise vCloud Express Lang

Test include SPECjvm, Ruby, Python and PHP Large Medium Small

Figure 5. Interpreted Language Programming Results Savvis VPDC Performed Faster for Programming Across Large, Medium, and Small Environments

White Paper

Encoding and Encryption Results What it measures: Speed of encoding and encryption 180 Most relevant to: Storing sensitive information in the cloud, including intellectual property, 160 healthcare records, and financial information 140

Business impact: Secure access and compliance with 120 privacy requirements 100 Baseline (100 points): Dual Intel Xeon Server E5504 Quad- 80 Core 2.00 GHz processors with 48 GB DDR3 ECC RAM 60

Figure 6 illustrates each company’s encoding and 40 encryption results.

0 Savvis VPDC AWS EC2 IBM SmartCloud Rackspace Terremark Enterprise vCloud Express Encode

Large Medium Small

Figure 6. Encoding and Encryption Results Savvis VPDC Ranked First for Encryption and Encoding

7 © 2012 Cisco and/or its affiliates. All rights reserved. About the Savvis Symphony VPDC the Savvis Symphony VPDC to extend or replace their own Infrastructure: Built on Cisco Unified data centers stand to gain the most in terms of cost savings, Computing System™ incremental revenue from a faster customer experience, and first-to-market advantage. Savvis Symphony VPDC is one of the industry’s first enterprise- class virtual private data center solutions, providing multi-tiered For More Information security and service and network profiles. The service is built To learn more about Savvis cloud services, visit: on Cisco® UCS B200 M2 Blade Servers featuring dual Intel www.savvis.com/cloud Xeon 5570 4-core 2.93-Ghz processors and 96 GB RAM.

Conclusion To learn more about the Cisco Unified Data Center solutions at the foundation of the Savvis Symphony VPDC, visit: Introduced in 2010, The Savvis Symphony VPDC performed www.cisco.com/go/udc at or near the top of all five test categories: CPU, disk I/O, memory I/O, interpreted-language programming, and To read the full benchmark report, call Savvis: encryption and encoding. Therefore, organizations that use 1-800-SAVVIS1.

Appendix: Benchmark Summary

CPU Memory Disk Encryption Language

Unit CCU MIOP IOP Encode Lang

Baseline Amazon EC2 ECU Dual Intel E5504 Dual Intel E5506 Dual Intel E5504 Dual Intel E5506 Config Quad core 2.00 2.13 GHz (8 cores) Quad core 2.00 2.13 GHz (8 cores) GHz processors with 4X15K RPM GHz processors with 4X15K RPM and 48 GB DDR3 SAS drives with and 48 GB DDR3 SAS drives with ECC ram hardware RAID 1+0 ECC ram hardware RAID 1+0

Test 19 7 7 7 4

Test c-ray, crafty, dcraw, espeak, CacheBench, Blogbench, Monkey Audio SPECjvm, Ruby, Names geekbench, graphics-magick, Geekbench, Bonnie++, Dbench, Encoding WAV Python, PHP hmmer, john-the-ripper- hdparm, Flexible IO Tester, To FLAC, WAV To (blowfisj, des, md5) mafft, RAMspeed, hdparm buffered MP3, WAV To Ogg, nero2d, openss, opstone-(svd, Redis Benchmark disk reads, IOzone, WAV To WavPack, svsp, vsp), sudokut, tscp, Stream, Threaded I/O FFmpeg AVI to unixbench Unixbench Tester NTSC VCD, GnuPG

Cisco and the Cisco Logo are trademarks of Cisco Systems, Inc. and/or its affiliates in the U.S. and other countries. A listing of Cisco’s trademarks can be found at www.cisco. com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1007R)