Document: 2013 Consumer Security Products Performance Benchmarks (Edition 4) Authors: M. Baquiran, D. Wren Company: PassMark Software Date: 26 April 2013 Edition: 4 File: totalprotectionsuites-may2013.docx 2013 Consumer Security Products PassMark Software
TABLE OF CONTENTS ...... 2 REVISION HISTORY ...... 4 REFERENCES ...... 4 EXECUTIVE SUMMARY ...... 5 OVERALL SCORE ...... 6 PRODUCTS AND VERSIONS ...... 9
INTERNET SECURITY SOFTWARE ...... 9 ANTIVIRUS SOFTWARE ...... 10 TOTAL SECURITY SOFTWARE ...... 10 PERFORMANCE METRICS SUMMARY ...... 11 INTERNET SECURITY SOFTWARE (HDD) – TEST RESULTS ...... 14
BENCHMARK 1 – BOOT TIME ...... 14 BENCHMARK 2 – SCAN TIME ...... 15 BENCHMARK 3 – USER INTERFACE LAUNCH TIME ...... 16 BENCHMARK 4 – MEMORY USAGE DURING SYSTEM IDLE ...... 17 BENCHMARK 5 – BROWSE TIME ...... 18 BENCHMARK 6 – INTERNET EXPLORER LAUNCH TIME ...... 19 BENCHMARK 7 – INSTALLATION TIME ...... 20 BENCHMARK 8 – INSTALLATION SIZE ...... 21 BENCHMARK 9 – REGISTRY KEYS ADDED ...... 22 BENCHMARK 10 – FILE COPY, MOVE AND DELETE ...... 23 BENCHMARK 11 – INSTALLATION OF THIRD PARTY APPLICATIONS ...... 24 BENCHMARK 12 – NETWORK THROUGHPUT ...... 25 BENCHMARK 13 – FILE FORMAT CONVERSION ...... 26 BENCHMARK 14 – FILE COMPRESSION AND DECOMPRESSION ...... 27 BENCHMARK 15 – FILE WRITE, OPEN AND CLOSE ...... 28 BENCHMARK 16 – PE SCAN TIME ...... 29 BENCHMARK 17 – FILE COPY DISK TO DISK ...... 30 ANTIVIRUS SOFTWARE (HDD) – TEST RESULTS ...... 31
BENCHMARK 1 – BOOT TIME ...... 31 BENCHMARK 2 – SCAN TIME ...... 32 BENCHMARK 3 – USER INTERFACE LAUNCH TIME ...... 33 BENCHMARK 4 – MEMORY USAGE DURING SYSTEM IDLE ...... 34 BENCHMARK 5 – BROWSE TIME ...... 35 BENCHMARK 6 – INTERNET EXPLORER LAUNCH TIME ...... 36 BENCHMARK 7 – INSTALLATION TIME ...... 37 BENCHMARK 8 – INSTALLATION SIZE ...... 38 BENCHMARK 9 – REGISTRY KEYS ADDED ...... 39 BENCHMARK 10 – FILE COPY, MOVE AND DELETE ...... 40 BENCHMARK 11 – INSTALLATION OF THIRD PARTY APPLICATIONS ...... 41 Performance Benchmark Page 2 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
BENCHMARK 12 – NETWORK THROUGHPUT ...... 42 BENCHMARK 13 – FILE FORMAT CONVERSION ...... 43 BENCHMARK 14 – FILE COMPRESSION AND DECOMPRESSION ...... 44 BENCHMARK 15 – FILE WRITE, OPEN AND CLOSE ...... 45 BENCHMARK 16 – PE SCAN TIME ...... 46 BENCHMARK 17 – FILE COPY DISK TO DISK ...... 47 TOTAL SECURITY SOFTWARE (HDD) – TEST RESULTS ...... 48
BENCHMARK 1 – BOOT TIME ...... 48 BENCHMARK 2 – SCAN TIME ...... 48 BENCHMARK 3 – USER INTERFACE LAUNCH TIME ...... 49 BENCHMARK 4 – MEMORY USAGE DURING SYSTEM IDLE ...... 49 BENCHMARK 5 – BROWSE TIME ...... 50 BENCHMARK 6 – INTERNET EXPLORER LAUNCH TIME ...... 50 BENCHMARK 7 – INSTALLATION TIME ...... 51 BENCHMARK 8 – INSTALLATION SIZE ...... 51 BENCHMARK 9 – REGISTRY KEYS ADDED ...... 52 BENCHMARK 10 – FILE COPY, MOVE AND DELETE ...... 52 BENCHMARK 11 – INSTALLATION OF THIRD PARTY APPLICATIONS ...... 53 BENCHMARK 12 – NETWORK THROUGHPUT ...... 53 BENCHMARK 13 – FILE FORMAT CONVERSION ...... 54 BENCHMARK 14 – FILE COMPRESSION AND DECOMPRESSION ...... 54 BENCHMARK 15 – FILE WRITE, OPEN AND CLOSE ...... 55 BENCHMARK 16 – PE SCAN TIME ...... 55 BENCHMARK 17 – FILE COPY DISK TO DISK ...... 56 INTERNET SECURITY SOFTWARE (SSD) – TEST RESULTS ...... 57
BENCHMARK 1 – BOOT TIME ...... 57 BENCHMARK 2 – SCAN TIME ...... 57 BENCHMARK 10 – FILE COPY, MOVE AND DELETE ...... 58 BENCHMARK 17 – FILE COPY DISK TO DISK ...... 58 DISCLAIMER AND DISCLOSURE ...... 59 CONTACT DETAILS ...... 59 APPENDIX 1 – TEST ENVIRONMENT ...... 60 APPENDIX 2 – METHODOLOGY DESCRIPTION ...... 61
Performance Benchmark Page 3 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
Rev Revision History Date
Edition 1 Initial version of this report. 31 August 2012
Edition 2 Added newly tested products. 2 November 2012
Edition 3 Added newly tested products. 15 January 2013
Edition 4 Added newly tested products. 24 April 2013
Ref # Document Author Date
O. Warner, 1 What Really Slows Windows Down (URL) 2001-2009 The PC Spy
Performance Benchmark Page 4 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
PassMark Software® conducted objective performance testing on fourteen (14) Internet Security software products, sixteen (16) Antivirus software products, and five (5) Total Security software products on Windows 7 Ultimate Edition (64-bit) SP1 between July 2012 and February 2013. This report presents our results and findings as a result of performance benchmark testing conducted for these consumer security products.
Subsequent editions of this report will include new products released for 2013 as they are made available. For more details on which versions were tested, please see the section “Products and Versions”.
Note that throughout this report, categories marked with “(HDD)” and “(SSD)” were tested with a Hard Disk Drive and a Solid State Drive used as the main boot drive respectively.
Testing was performed on all products using seventeen (17) performance metrics. These performance metrics are as follows:
Boot Time; Scan Time; User Interface Launch Time; Memory Usage during System Idle; Browse Time; Internet Explorer Launch Time; Installation Time; Installation Size; Registry Keys Added; File Copy, Move and Delete; Installation of Third Party Applications; Network Throughput (previously named “Binary Download Test”) File Format Conversion; File Compression and Decompression; File Write, Open and Close; PE Scan Time and; and File Copy Disk to Disk.
Testing in the SSD configuration was performed on five (5) Internet Security products using four (4) of the above metrics. These performance metrics are as follows:
Boot Time; Scan Time; File Copy, Move and Delete; and File Copy Disk to Disk.
Performance Benchmark Page 5 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
PassMark Software assigned every product a score depending on its ranking in each metric compared to other products in the same category.
In the following table the highest possible score attainable is 238; in a hypothetical situation where a product has attained first place all 17 metrics. Internet Security products have been ranked by their overall scores:
Product Name Overall Score
Webroot SecureAnywhere Internet Security 2013 215
Norton Internet Security 2013 168
Kaspersky Internet Security 2013 161
SourceNext Virus Security 157
AVG Internet Security 2013 146
Trend Micro Titanium Internet Security 2013 124
Trend Micro Virus Buster 2013 120
ESET Smart Security 6 108
Avast! Internet Security 7 107
G Data InternetSecurity 2013 102
F-Secure Internet Security 2013 101
Panda Internet Security 2013 100
BitDefender Internet Security 2013 92
McAfee Internet Security 2013 83
Performance Benchmark Page 6 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
In the following table the highest possible score attainable is 272; in a hypothetical situation where a product has attained first place in all 17 metrics. Antivirus products have been ranked by their overall scores:
Product Name Overall Score
Webroot SecureAnywhere AntiVirus 2013 241
Kaspersky Anti-Virus 2013 179
Norton AntiVirus 2013 165
AVG Anti-Virus Free 2013 158
Microsoft Security Essentials 155
avast! Pro Antivirus 7 145
AVG Anti-Virus 2013 143
Panda Antivirus Pro 2013 142
Panda Cloud Antivirus FREE 141
Quick Heal Pro Antivirus 2013 135
Avira Antivirus Free 2013 135
G Data AntiVirus 2013 132
Trend Micro Titanium 2013 116
ESET NOD32 Antivirus 6 114
Avast! Free Antivirus 7 112
McAfee Antivirus Plus 2013 96
Performance Benchmark Page 7 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
In the following table the highest possible score attainable is 85; in a hypothetical situation where a product has attained first place in all 17 metrics. Total Security products have been ranked by their overall scores:
Product Name Overall Score
Norton 360 68
Kaspersky PURE 2.0 57
Trend Micro Maximum Security 2013 48
Bitdefender Total Security 2013 43
McAfee Total Protection 2013 39
In the following table the highest possible score attainable is 20; in a hypothetical situation where a product has attained first place in all 5 (SSD) metrics. Internet Security products have been ranked by their overall scores:
Product Name Overall Score
Norton Internet Security 2013 16
Bitdefender Internet Security 2013 15
Trend Micro Internet Security 2013 12
Kaspersky Internet Security 2013 11
McAfee Internet Security 2013 6
Performance Benchmark Page 8 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
Unless otherwise specified, we have tested the full, retail release of the newest generation (2013 versions) of security products. The names and versions of products which were tested can be found below from each category table:
Release Manufacturer Product Name Product Version Date Tested Year
Symantec Corporation Norton Internet Security 2013 2012 20.2.0.19 Dec 2012
G Data Software AG G Data InternetSecurity 2013 2012 23.0.0.19 Jul 2012
Avast Software a.s. Avast! Internet Security 7 2012 7.0.1456 Aug 2012
SOFTWIN Bitdefender Internet Security 2013 2012 16.16.0.1349 Aug 2012
Kaspersky Lab Kaspersky Internet Security 2013 2012 13.0.1.4190 Aug 2012
Panda Security SL Panda Internet Security 2013 2012 18.00.01 Sep 2012
McAfee, Inc. McAfee Internet Security 2013 2012 12.0.637 Sep 2012
AVG Technologies AVG Internet Security 2013 2012 2013.0.2677 Sep 2012
Trend Micro Titanium Internet Security Trend Micro Inc. 2012 6.0.1215 Oct 2012 2013
Webroot SecureAnywhere Internet Webroot Inc. 2012 8.0.2.27 Oct 2012 Security 2013
F-Secure Corporation F-Secure Internet Security 2013 2012 12.71 Build 102 Dec 2012
ESET, spol. s r.o. ESET Smart Security 6 2012 6.0.308.0 Feb 2013
Trend Micro Inc. Trend Micro Virus Buster Cloud 2012 6.0.1215 Feb 2013
SourceNext SourceNext Virus Security Zero 2013 2012 12.2.0186 Feb 2013
Performance Benchmark Page 9 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
Manufacturer Product Name Release Year Product Version Date Tested
Symantec Corporation Norton AntiVirus 2013 2012 20.2.0.19 Dec 2012
G Data Software AG G Data Antivirus 2013 2012 22.0.2.25 Aug 2012
Avast Software a.s. avast! Pro Antivirus 7 2012 7.0.1456 Aug 2012
Microsoft Corporation Microsoft Security Essentials 2012 4.0.1526.0 Aug 2012
Kaspersky Lab Kaspersky Anti-Virus 2013 2012 13.0.1.4190 Aug 2012
McAfee, Inc. McAfee Antivirus Plus 2012 12.0.637 Oct 2012
AVG Anti-Virus 2013 2012 2013.0.2677 Oct 2012
Panda Security SL Panda Antivirus Pro 2013 2012 12.0.0.1 Oct 2012
Trend Micro Inc. Trend Micro Titanium 2013 2012 6.0.1215 Oct 2012
Quick Heal Quick Heal Pro Antivirus 2013 2012 14.00.7.0.0.1 Oct 2012
Avast Software a.s. avast! Free Antivirus 7 2012 7.0.1473 Oct 2012
Panda Security SL Panda Clout Antivirus FREE 2013 2012 2.0.1 Oct 2012
Avira Operations Avira Antivirus Free 2013 2012 13.0.0.2832 Nov 2012 GmbH & Co. KG
Webroot SecureAnywhere AntiVirus Webroot Inc. 2012 8.0.2.27 Oct 2012 2013
ESET, spol. s r.o. ESET NOD32 Antivirus 2012 6.0.308.0 Feb 2013
AVG Technologies AVG Free 2013 2012 2013.0.2793 Nov 2012
Manufacturer Product Name Release Year Product Version Date Tested
Symantec Corporation Norton 360 2012 20.2.0.19 Dec 2012
Trend Micro Inc. Trend Micro Maximum Security 2013 2012 6.0.1215 Oct 2012
Kaspersky Lab Kaspersky PURE 2.0 2012 12.0.1.288 Aug 2012
BitDefender BitDefender Total Security 2013 2012 16.16.0.1348 Aug 2012
McAfee, Inc McAfee Total Protection 2013 2012 16.1.140 Dec 2012
Performance Benchmark Page 10 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
We have selected a set of objective metrics which provide a comprehensive and realistic indication of the areas in which an antivirus may impact system performance for end users. Our metrics test the impact of the antivirus software on common tasks that end-users would perform on a daily basis.
All of PassMark Software’s test methods can be replicated by third parties using the same environment to obtain similar benchmark results. Detailed descriptions of the methodologies used in our tests are available as “Appendix 2 – Methodology Description” of this report.
This metric measures the amount of time taken for the machine to boot into the operating system. Security software is generally launched at Windows startup, adding an additional amount of time and delaying the startup of the operating system. Shorter boot times indicate that the application has had less impact on the normal operation of the machine.
All antivirus solutions have functionality designed to detect viruses and various other forms of malware by scanning files on the system. This metric measured the amount of time required to scan a set of clean files. Our sample file set comprised a total file size of 982 MB and was made up of files that would typically be found on end-user machines, such as media files, system files and Microsoft Office documents.
This metric provides an objective indication as to how responsive a security product appears to the user, by measuring the amount of time it takes for the user interface of the antivirus software to launch from Windows. To allow for caching effects by the operating system, both the initial launch time and the subsequent launch times were measured. Our final result is an average of these two measurements.
This metric measures the amount of memory (RAM) used by the product while the machine and antivirus software are in an idle state. The total memory usage was calculated by identifying all antivirus software processes and the amount of memory used by each process.
The amount of memory used while the machine is idle provides a good indication of the amount of system resources being consumed by the antivirus software on a permanent basis. Better performing products occupy less memory while the machine is idle.
It is common behavior for security products to scan data for malware as it is downloaded from the internet or intranet. This behavior may negatively impact browsing speed as products scan web content for malware. This metric measures the time taken to browse a set of popular internet sites to consecutively load from a local server in a user’s browser window.
Performance Benchmark Page 11 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
This metric is one of many methods to objectively measure how much a security product impacts on the responsiveness of the system. This metric measures the amount of time it takes to launch the user interface of Internet Explorer 8. To allow for caching effects by the operating system, both the initial launch time and the subsequent launch times were measured. Our final result is an average of these two measurements.
The speed and ease of the installation process will strongly influence the user’s first impression of the antivirus software. This test measures the minimum installation time required by the antivirus software to be fully functional and ready for use by the end user. Lower installation times represent antivirus products which are quicker for a user to install.
In offering new features and functionality to users, antivirus software products tend to increase in size with each new release. Although new technologies push the size limits of hard drives each year, the growing disk space requirements of common applications and the increasing popularity of large media files (such as movies, photos and music) ensure that a product's installation size will remain of interest to home users.
This metric aims to measure a product’s total installation size. This metric is defined as the total disk space consumed by all new files added during a product's installation.
A large registry increases a machine’s use of resources. This may negatively impact system performance, especially on much older machines. This test measures the amount of keys and values added to registry, after rebooting the test machines, following a successful product installation. Lower numbers mean that a product has added fewer keys during installation and had less impact on the registry.
This metric measures the amount of time taken to move, copy and delete a sample set of files. The sample file set contains several types of file formats that a Windows user would encounter in daily use. These formats include documents (e.g. Microsoft Office documents, Adobe PDF, Zip files, etc), media formats (e.g. images, movies and music) and system files (e.g. executables, libraries, etc).
This metric measures the amount of time taken to install and uninstall third party programs. The installation speed of third party applications may be impacted by antivirus behavior such as heuristics or real time malware scanning.
The metric measures the amount of time taken to download a variety of files from a local server using the HyperText Transfer Protocol (HTTP), which is the main protocol used on the web for browsing, linking and data transfer. Files used in this test include file formats that users would typically download from the web, such as images, archives, music files and movie files.
Performance Benchmark Page 12 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
This test measures the amount of time taken to convert an MP3 file to a WAV and subsequently, convert the same MP3 file to a WMA format.
This metric measures the amount of time taken to compress and decompress different types of files. Files formats used in this test included documents, movies and images.
This benchmark was derived from Oli Warner’s File I/O test at http://www.thepcspy.com (please see Reference #1: What Really Slows Windows Down). This metric measures the amount of time taken to write a file, then open and close that file.
All antivirus solutions have functionality designed to detect viruses and various other forms of malware by scanning files on the system. This metric measured the amount of time required to scan a set of PE (Portable Executable) files. Our sample file set comprised a total file size of 2.03GB and consisted of .exe (329MB), .dll (920MB) and .sys files (827MB).
This test measures the amount of time taken to copy files between two local drives. The data set comprised a total file size of 5.44GB, and the formats used included documents, movies, images and executables.
Performance Benchmark Page 13 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
In the following charts, we have highlighted the results we obtained for Norton Internet Security in yellow. The average has also been highlighted in blue for ease of comparison.
The following chart compares the average time taken for the system to boot (from a sample of five boots) for each Internet Security product tested. Products with lower boot times are considered better performing products in this category.
Trend Micro Internet Security 2013 29.5
Trend Micro Virus Buster 2013 29.8
Webroot Internet Security 2013 31.5
McAfee Internet Security 2013 32.1
Norton Internet Security 2013 33.4
ESET Smart Security 6 33.8
Kaspersky Internet Security 2013 34.1
Avast! Internet Security 7 34.7
Average 36.5
Panda Internet Security 2013 39.2
SourceNext Virus Security 39.7
Bitdefender Internet Security 2013 41.5
G Data Internet Security 2013 43.4
F-Secure Internet Security 2013 43.4
AVG Internet Security 2013 45.1
0 s 5 s 10 s 15 s 20 s 25 s 30 s 35 s 40 s 45 s 50 s
Performance Benchmark Page 14 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average time taken to scan a set of 6159 files (totaling 982 MB) for each Internet Security product tested. This time is calculated by averaging the initial (Run 1) and subsequent (Runs 2- 5) scan times. Products with lower scan times are considered better performing products in this category.
Norton Internet Security 2013 20.6
Webroot Internet Security 2013 21.4
G Data Internet Security 2013 22.5
Kaspersky Internet Security 2013 30.5
McAfee Internet Security 2013 34.9
Bitdefender Internet Security 2013 35.4
Trend Micro Internet Security 2013 40.1
Average 42.8
Trend Micro Virus Buster 2013 43.3
ESET Smart Security 6 47.3
AVG Internet Security 2013 49.4
SourceNext Virus Security 58.0
Avast! IS 7 58.8
Panda Internet Security 2013 60.1
F-Secure Internet Security 2013 77.0
0 s 10 s 20 s 30 s 40 s 50 s 60 s 70 s 80 s 90 s
Performance Benchmark Page 15 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average time taken to launch a product’s user interface. Products with lower launch times are considered better performing products in this category.
SourceNext Virus Security 120
Webroot Internet Security 2013 125
Norton Internet Security 2013 128
AVG Internet Security 2013 208
Trend Micro Internet Security 2013 259
Avast! Internet Security 7 300
Trend Micro Virus Buster 2013 345
F-Secure Internet Security 2013 415
G Data Internet Security 2013 483
ESET Smart Security 6 487
Bitdefender Internet Security 2013 508
Average 538
McAfee Internet Security 2013 799
Kaspersky Internet Security 2013 896
Panda Internet Security 2013 2455
0 ms 500 ms 1,000 ms 1,500 ms 2,000 ms 2,500 ms 3,000 ms
Performance Benchmark Page 16 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average amount of RAM in use by an Internet Security product during a period of system idle. This average is taken from a sample of ten memory snapshots taken at roughly 60 seconds apart after reboot. Products with lower idle RAM usage are considered better performing products in this category.
Webroot Internet Security 2013 3.4
Panda Internet Security 2013 15.5
Norton Internet Security 2013 15.5
Avast! Internet Security 7 16.4
Bitdefender Internet Security 2013 37.3
SourceNext Virus Security 39.6
Kaspersky Internet Security 2013 49.3
Average 58.6
Trend Micro Internet Security 2013 80.1
G Data Internet Security 2013 82.5
F-Secure Internet Security 2013 85.3
Trend Micro Virus Buster 2013 90.4
AVG Internet Security 2013 90.4
McAfee Internet Security 2013 98.2
ESET Smart Security 6 116.5
0 MB 20 MB 40 MB 60 MB 80 MB 100 MB 120 MB 140 MB
Performance Benchmark Page 17 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average time taken for Internet Explorer to successively load a set of popular websites through the local area network from a local server machine. Products with lower browse times are considered better performing products in this category.
Webroot Internet Security 2013 37.1
SourceNext Virus Security 41.2
Kaspersky Internet Security 2013 41.7
Trend Micro Internet Security 2013 43.2
ESET Smart Security 6 43.3
Norton Internet Security 2013 44.0
Avast! Internet Security 7 44.8
Panda Internet Security 2013 49.6
Average 49.7
Trend Micro Virus Buster 2013 51.4
G Data Internet Security 2013 55.4
F-Secure Internet Security 2013 55.8
AVG Internet Security 2013 55.9
McAfee Internet Security 2013 59.1
Bitdefender Internet Security 2013 73.0
0 s 10 s 20 s 30 s 40 s 50 s 60 s 70 s 80 s
Performance Benchmark Page 18 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average launch times of Internet Explorer after rebooting the machine for each Internet Security product we tested. Products with lower launch times are considered better performing products in this category.
SourceNext Virus Security 994
AVG Internet Security 2013 1037
G Data Internet Security 2013 1527
Kaspersky Internet Security 2013 1592
ESET Smart Security 6 1862
F-Secure Internet Security 2013 1932
Norton Internet Security 2013 1950
Average 2148
Webroot Internet Security 2013 2237
McAfee Internet Security 2013 2275
Bitdefender Internet Security 2013 2319
Trend Micro Virus Buster 2013 2402
Avast! Internet Security 7 2547
Panda Internet Security 2013 3196
Trend Micro Internet Security 2013 4202
0 ms 500 ms 1,000 ms 1,500 ms 2,000 ms 2,500 ms 3,000 ms 3,500 ms 4,000 ms 4,500 ms
Performance Benchmark Page 19 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the minimum installation time it takes for Internet Security products to be fully functional and ready for use by the end user. Products with lower installation times are considered better performing products in this category.
Webroot Internet Security 2013 5.5
Norton Internet Security 2013 64.6
SourceNext Virus Security 66.4
Trend Micro Internet Security 2013 150.6
Kaspersky Internet Security 2013 158.1
F-Secure Internet Security 2013 201.0
Trend Micro Virus Buster 2013 201.0
Average 261.8
AVG Internet Security 2013 271.3
G Data Internet Security 2013 301.2
ESET Smart Security 6 347.6
Bitdefender Internet Security 2013 364.2
Panda Internet Security 2013 377.6
Avast! Internet Security 7 463.6
McAfee Internet Security 2013 693.1
0 s 100 s 200 s 300 s 400 s 500 s 600 s 700 s 800 s
Performance Benchmark Page 20 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the total size of files added during the installation of Internet Security products. Products with lower installation sizes are considered better performing products in this category.
Webroot Internet Security 2013 64.4
F-Secure Internet Security 2013 240.5
Panda Internet Security 2013 325.7
McAfee Internet Security 2013 376.3
ESET Smart Security 6 383.0
Kaspersky Internet Security 2013 388.0
SourceNext Virus Security 454.0
AVG Internet Security 2013 481.2
Trend Micro Internet Security 2013 489.5
Average 492.6
Norton Internet Security 2013 529.2
Trend Micro Virus Buster 2013 596.4
G Data Internet Security 2013 726.4
Avast! Internet Security 7 745.7
Bitdefender Internet Security 2013 1095.7
0 MB 200 MB 400 MB 600 MB 800 MB 1,000 MB 1,200 MB
Performance Benchmark Page 21 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the amount of Registry Keys created during product installation for each Internet Security product tested. Products with lower key counts are considered better performing products in this category.
Webroot Internet Security 2013 825
SourceNext Virus Security 3600
F-Secure Internet Security 2013 4533
Trend Micro Internet Security 2013 4693
Norton Internet Security 2013 4834
Trend Micro Virus Buster 2013 4945
Avast! Internet Security 7 5357
AVG Internet Security 2013 5402
ESET Smart Security 6 6422
Average 6686
Bitdefender Internet Security 2013 6695
Kaspersky Internet Security 2013 6945
Panda Internet Security 2013 7560
G Data Internet Security 2013 9166
McAfee Internet Security 2013 22632
0 5000 10000 15000 20000 25000
Performance Benchmark Page 22 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average time taken to copy, move and delete several sets of sample files for each Internet Security product tested. Products with lower times are considered better performing products in this category.
Webroot Internet Security 2013 23.8
Kaspersky Internet Security 2013 24.9
Norton Internet Security 2013 25.7
SourceNext Virus Security 26.0
Avast! Internet Security 7 26.3
G Data Internet Security 2013 26.8
AVG Internet Security 2013 28.0
Panda Internet Security 2013 28.6
Bitdefender Internet Security 2013 28.7
Trend Micro Virus Buster 2013 30.0
ESET Smart Security 6 30.4
Average 30.6
Trend Micro Internet Security 2013 32.2
F-Secure Internet Security 2013 45.9
McAfee Internet Security 2013 51.7
0 s 10 s 20 s 30 s 40 s 50 s 60 s
Performance Benchmark Page 23 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average time taken to install 3 different third party applications for each Internet Security product tested. Products with lower times are considered better performing products in this category.
Trend Micro Virus Buster 2013 95.9
SourceNext Virus Security 97.0
Webroot Internet Security 2013 98.5
Panda Internet Security 2013 101.8
AVG Internet Security 2013 104.8
ESET Smart Security 6 108.5
Kaspersky Internet Security 2013 109.1
Norton Internet Security 2013 110.1
Avast! Internet Security 7 110.9
Average 114.2
F-Secure Internet Security 2013 120.0
Trend Micro Internet Security 2013 121.6
McAfee Internet Security 2013 130.7
G Data Internet Security 2013 141.0
Bitdefender Internet Security 2013 149.4
0 s 20 s 40 s 60 s 80 s 100 s 120 s 140 s 160 s
Performance Benchmark Page 24 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average time to download a sample set of common file types for each Internet Security product tested. Products with lower times are considered better performing products in this category.
SourceNext Virus Security 10.3
Webroot Internet Security 2013 10.3
Trend Micro Virus Buster 2013 11.8
Trend Micro Internet Security 2013 11.8
Kaspersky Internet Security 2013 12.2
Avast! Internet Security 7 12.8
Norton Internet Security 2013 13.8
AVG Internet Security 2013 15.0
Bitdefender Internet Security 2013 16.7
McAfee Internet Security 2013 19.7
ESET Smart Security 6 21.6
F-Secure Internet Security 2013 23.8
Average 33.0
G Data Internet Security 2013 79.2
Panda Internet Security 2013 202.2
0 s 50 s 100 s 150 s 200 s 250 s
Performance Benchmark Page 25 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average time it takes for five sample files to be converted from one file format to another (MP3 WMA, MP3 WAV) for each Internet Security product tested. Products with lower times are considered better performing products in this category.
AVG Internet Security 2013 75.6
SourceNext Virus Security 75.9
ESET Smart Security 6 75.9
Webroot Internet Security 2013 76.0
Kaspersky Internet Security 2013 76.1
Bitdefender Internet Security 2013 76.2
Norton Internet Security 2013 76.3
McAfee Internet Security 2013 76.4
G Data Internet Security 2013 76.5
Average 76.5
Panda Internet Security 2013 76.8
Trend Micro Internet Security 2013 77.1
F-Secure Internet Security 2013 77.4
Trend Micro Virus Buster 2013 77.4
Avast! Internet Security 7 78.0
74 s 75 s 75 s 76 s 76 s 77 s 77 s 78 s 78 s 79 s
Performance Benchmark Page 26 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average time it takes for sample files to be compressed and decompressed for each Internet Security product tested. Products with lower times are considered better performing products in this category.
AVG Internet Security 2013 79.8
Webroot Internet Security 2013 80.3
Norton Internet Security 2013 80.9
Kaspersky Internet Security 2013 82.2
Panda Internet Security 2013 84.2
Bitdefender Internet Security 2013 84.4
Avast! Internet Security 7 84.6
SourceNext Virus Security 85.8
F-Secure Internet Security 2013 85.9
G Data Internet Security 2013 86.1
Average 86.7
ESET Smart Security 6 87.3
McAfee Internet Security 2013 94.9
Trend Micro Virus Buster 2013 98.3
Trend Micro Internet Security 2013 99.8
0 s 20 s 40 s 60 s 80 s 100 s 120 s
Performance Benchmark Page 27 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average time it takes for a file to be written to the hard drive then opened and closed 180,000 times, for each Internet Security product tested. Products with lower times are considered better performing products in this category.
Webroot Internet Security 2013 15.3
AVG Internet Security 2013 16.4
Kaspersky Internet Security 2013 18.1
Panda Internet Security 2013 25.7
Norton Internet Security 2013 28.9
Avast! Internet Security 7 40.6
Trend Micro Virus Buster 2013 63.1
Trend Micro Internet Security 2013 71.1
McAfee Internet Security 2013 74.5
F-Secure Internet Security 2013 101.1
ESET Smart Security 6 125.1
Average 222.8
G Data Internet Security 2013 405.3
Bitdefender Internet Security 2013 1027.9
SourceNext Virus Security 1105.7
0 s 200 s 400 s 600 s 800 s 1,000 s 1,200 s
Performance Benchmark Page 28 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average time taken to scan a set of 6351 portable executable files (totaling 2076 MB) for each Internet Security product tested. This time is calculated by averaging the initial (Run 1) and subsequent (Runs 2-5) scan times. Products with lower scan times are considered better performing products in this category.
Kaspersky Internet Security 2013 35.6
G Data Internet Security 2013 68.0
Webroot Internet Security 2013 76.5
AVG Internet Security 2013 78.0
McAfee Internet Security 2013 80.2
Trend Micro Virus Buster 2013 89.3
Norton Internet Security 2013 92.5
Bitdefender Internet Security 2013 96.5
ESET Smart Security 6 108.8
Average 110.9
Avast! Internet Security 7 114.4
SourceNext Virus Security 115.4
F-Secure Internet Security 2013 191.9
Panda Internet Security 2013 198.0
Trend Micro Internet Security 2013 207.9
0 s 50 s 100 s 150 s 200 s 250 s
Performance Benchmark Page 29 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average time taken to copy a total of 8,501 files, with a total file size of 5.44GB files, from one local drive to another local drive for each Internet Security product tested. The test was performed 5 times, and the average of all 5 runs was taken as the result. Products with lower times are considered better performing products in this category.
Trend Micro Internet Security 2013 109.8
F-Secure Internet Security 2013 114.9
AVG Internet Security 2013 116.4
Webroot Internet Security 2013 118.2
Norton Internet Security 2013 122.2
Bitdefender Internet Security 2013 122.7
Kaspersky Internet Security 2013 124.6
G Data Internet Security 2013 125.4
Avast! Internet Security 7 125.8
Trend Micro Virus Buster 2013 131.8
Average 136.2
Panda Internet Security 2013 143.6
ESET Smart Security 6 147.8
SourceNext Virus Security 149.8
McAfee Internet Security 2013 254.1
0 s 50 s 100 s 150 s 200 s 250 s 300 s
Performance Benchmark Page 30 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
In the following charts, we have highlighted the results we obtained for Norton AntiVirus in yellow. The average has also been highlighted in blue for ease of comparison.
The following chart compares the average time taken for the system to boot (from a sample of five boots) for each Antivirus product tested. Products with lower boot times are considered better performing products in this category.
Trend Micro Antivirus 2013 28.1
ESET Antivirus 6 29.4
avast! Antivirus 2013 30.8
Webroot AntiVirus 2013 31.5
Norton AntiVirus 2013 33.4
McAfee AntiVirus 2013 33.5
Microsoft Security Essentials 4.0 33.9
Avira Antivirus Free 2013 34.2
Kaspersky Anti-Virus 2013 34.6
Panda Cloud Antivirus FREE 2013 34.7
avast! Free Antivirus 7 35.1
Average 35.4
Panda Antivirus 2013 36.9
Quick Heal Antivirus 2013 39.5
AVG AntiVirus Free 2013 41.4
AVG Anti-Virus 2013 42.2
G Data AntiVirus 2013 47.8
0 s 10 s 20 s 30 s 40 s 50 s 60 s
Performance Benchmark Page 31 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average time taken to scan a set of 6159 files (totaling 982 MB) for each Antivirus product tested. This time is calculated by averaging the initial (Run 1) and subsequent (Runs 2-5) scan times. Products with lower scan times are considered better performing products in this category.
G Data AntiVirus 2013 21.0
Norton AntiVirus 2013 21.6
Webroot AntiVirus 2013 22.8
McAfee AntiVirus 2013 28.5
AVG AntiVirus Free 2013 29.5
Quick Heal Antivirus 2013 37.3
Microsoft Security Essentials 4.0 44.4
Trend Micro Antivirus 2013 44.5
Average 46.3
Kaspersky Anti-Virus 2013 49.0
Panda Antivirus 2013 54.8
AVG Anti-Virus 2013 56.8
Avira Antivirus Free 2013 57.4
avast! Antivirus 2013 60.0
avast! Free Antivirus 7 62.9
ESET Antivirus 6 71.8
Panda Cloud Antivirus FREE 2013 79.0
0 s 10 s 20 s 30 s 40 s 50 s 60 s 70 s 80 s 90 s
Performance Benchmark Page 32 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average time taken to launch a product’s user interface. Products with lower launch times are considered better performing products in this category.
Microsoft Security Essentials 4.0 61
ESET Antivirus 6 92
Webroot AntiVirus 2013 126
Norton AntiVirus 2013 201
AVG AntiVirus Free 2013 225
avast! Free Antivirus 7 250
AVG Anti-Virus 2013 303
avast! Antivirus 2013 326
Quick Heal Antivirus 2013 438
Trend Micro Antivirus 2013 487
Average 612
Panda Cloud Antivirus FREE 2013 678
McAfee AntiVirus 2013 699
Kaspersky Anti-Virus 2013 784
Avira Antivirus Free 2013 853
G Data AntiVirus 2013 1126
Panda Antivirus 2013 3137
0 ms 500 ms 1,000 ms 1,500 ms 2,000 ms 2,500 ms 3,000 ms 3,500 ms
Performance Benchmark Page 33 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average amount of RAM in use by an Antivirus product during a period of system idle. This average is taken from a sample of ten memory snapshots taken at roughly 60 seconds apart after reboot. Products with lower idle RAM usage are considered better performing products in this category.
Webroot AntiVirus 2013 3.2
avast! Free Antivirus 7 11.0
avast! Antivirus 2013 13.1
Panda Cloud Antivirus FREE 2013 35.3
AVG AntiVirus Free 2013 38.1
Panda Antivirus 2013 46.0
G Data AntiVirus 2013 55.2
Avira Antivirus Free 2013 58.7
Norton AntiVirus 2013 59.1
Kaspersky Anti-Virus 2013 62.3
Average 66.7
Trend Micro Antivirus 2013 76.4
Microsoft Security Essentials 4.0 77.0
AVG Anti-Virus 2013 83.1
ESET Antivirus 6 107.4
Quick Heal Antivirus 2013 123.7
McAfee AntiVirus 2013 217.5
0 MB 50 MB 100 MB 150 MB 200 MB 250 MB
Performance Benchmark Page 34 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average time taken for Internet Explorer to successively load a set of popular websites through the local area network from a local server machine. Products with lower browse times are considered better performing products in this category.
Webroot AntiVirus 2013 34.2
AVG AntiVirus Free 2013 35.9
Panda Antivirus 2013 36.1
McAfee AntiVirus 2013 36.4
Microsoft Security Essentials 4.0 38.1
Kaspersky Anti-Virus 2013 41.5
ESET Antivirus 6 41.9
Panda Cloud Antivirus FREE 2013 43.3
avast! Antivirus 2013 44.4
Norton AntiVirus 2013 45.4
avast! Free Antivirus 7 48.6
G Data AntiVirus 2013 52.2
Trend Micro Antivirus 2013 56.7
AVG Anti-Virus 2013 58.1
Average 59.7
Quick Heal Antivirus 2013 113.6
Avira Antivirus Free 2013 229.4
0 s 50 s 100 s 150 s 200 s 250 s
Performance Benchmark Page 35 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average launch times of Internet Explorer after rebooting the machine for each Antivirus product we tested. Products with lower launch times are considered better performing products in this category.
AVG Anti-Virus 2013 1065
AVG AntiVirus Free 2013 1128
ESET Antivirus 6 1765
Kaspersky Anti-Virus 2013 1771
Panda Cloud Antivirus FREE 2013 1877
Avira Antivirus Free 2013 1940
Average 2048
Webroot AntiVirus 2013 2096
Quick Heal Antivirus 2013 2132
Panda Antivirus 2013 2201
G Data AntiVirus 2013 2291
avast! Antivirus 2013 2291
Norton AntiVirus 2013 2293
McAfee AntiVirus 2013 2356
Microsoft Security Essentials 4.0 2440
avast! Free Antivirus 7 2483
Trend Micro Antivirus 2013 2642
0 ms 500 ms 1,000 ms 1,500 ms 2,000 ms 2,500 ms 3,000 ms
Performance Benchmark Page 36 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the minimum installation time it takes for Antivirus products to be fully functional and ready for use by the end user. Products with lower installation times are considered better performing products in this category.
Webroot AntiVirus 2013 5
Norton AntiVirus 2013 72
Panda Cloud Antivirus FREE 2013 123
avast! Antivirus 2013 143
Kaspersky Anti-Virus 2013 146
Panda Antivirus 2013 167
Trend Micro Antivirus 2013 201
Avira Antivirus Free 2013 201
AVG AntiVirus Free 2013 220
Average 229
AVG Anti-Virus 2013 239
Microsoft Security Essentials 4.0 268
G Data AntiVirus 2013 305
avast! Free Antivirus 7 321
Quick Heal Antivirus 2013 344
ESET Antivirus 6 376
McAfee AntiVirus 2013 532
0 s 100 s 200 s 300 s 400 s 500 s 600 s
Performance Benchmark Page 37 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the total size of files added during the installation of Antivirus products. Products with lower installation sizes are considered better performing products in this category.
Webroot AntiVirus 2013 3.5
Microsoft Security Essentials 4.0 47.8
Avira Antivirus Free 2013 231.1
McAfee AntiVirus 2013 250.3
ESET Antivirus 6 317.3
Panda Antivirus 2013 321.2
Kaspersky Anti-Virus 2013 363.6
Panda Cloud Antivirus FREE 2013 372.6
AVG Anti-Virus 2013 396.1
avast! Antivirus 2013 423.8
Average 430.8
AVG AntiVirus Free 2013 479.5
Norton AntiVirus 2013 524.5
avast! Free Antivirus 7 529.2
Trend Micro Antivirus 2013 529.2
G Data AntiVirus 2013 690.6
Quick Heal Antivirus 2013 1413.0
0 MB 200 MB 400 MB 600 MB 800 MB 1,000 MB 1,200 MB 1,400 MB 1,600 MB
Performance Benchmark Page 38 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the amount of Registry Keys created during product installation for each Antivirus product tested. Products with lower key counts are considered better performing products in this category.
Webroot AntiVirus 2013 495
avast! Antivirus 2013 2719
Quick Heal Antivirus 2013 2969
ESET Antivirus 6 3303
Panda Cloud Antivirus FREE 2013 4028
Trend Micro Antivirus 2013 4541
avast! Free Antivirus 7 4837
AVG Anti-Virus 2013 4994
Avira Antivirus Free 2013 5045
AVG AntiVirus Free 2013 5076
Norton AntiVirus 2013 5251
Microsoft Security Essentials 4.0 5276
Average 5490
Panda Antivirus 2013 6359
Kaspersky Anti-Virus 2013 6363
G Data AntiVirus 2013 7071
McAfee AntiVirus 2013 19516
0 5000 10000 15000 20000 25000
Performance Benchmark Page 39 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average time taken to copy, move and delete several sets of sample files for each Antivirus product tested. Products with lower times are considered better performing products in this category.
Webroot AntiVirus 2013 24.3
Norton AntiVirus 2013 25.2
G Data AntiVirus 2013 25.6
Kaspersky Anti-Virus 2013 26.2
avast! Antivirus 2013 27.1
avast! Free Antivirus 7 27.3
Quick Heal Antivirus 2013 27.4
AVG Anti-Virus 2013 27.8
Panda Antivirus 2013 29.6
Average 29.8
AVG AntiVirus Free 2013 29.9
Panda Cloud Antivirus FREE 2013 30.3
Trend Micro Antivirus 2013 30.4
Microsoft Security Essentials 4.0 30.6
McAfee AntiVirus 2013 32.4
Avira Antivirus Free 2013 37.6
ESET Antivirus 6 45.9
0 s 5 s 10 s 15 s 20 s 25 s 30 s 35 s 40 s 45 s 50 s
Performance Benchmark Page 40 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average time taken to install a third party application for each Antivirus product tested. Products with lower times are considered better performing products in this category.
Panda Antivirus 2013 101.3
G Data AntiVirus 2013 101.4
Webroot AntiVirus 2013 102.1
Kaspersky Anti-Virus 2013 102.6
Avira Antivirus Free 2013 103.1
avast! Antivirus 2013 108.4
Microsoft Security Essentials 4.0 109.0
AVG Anti-Virus 2013 109.1
AVG AntiVirus Free 2013 109.2
Average 109.5
avast! Free Antivirus 7 110.0
Norton AntiVirus 2013 110.4
Quick Heal Antivirus 2013 112.2
Panda Cloud Antivirus FREE 2013 113.2
McAfee AntiVirus 2013 119.7
ESET Antivirus 6 120.0
Trend Micro Antivirus 2013 120.3
90 s 95 s 100 s 105 s 110 s 115 s 120 s 125 s
Performance Benchmark Page 41 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average time to download a sample set of common file types for each Antivirus product tested. Products with lower times are considered better performing products in this category.
Panda Cloud Antivirus FREE 2013 10.4
Microsoft Security Essentials 4.0 10.4
Webroot AntiVirus 2013 10.5
Quick Heal Antivirus 2013 10.8
Panda Antivirus 2013 11.2
Kaspersky Anti-Virus 2013 11.7
Trend Micro Antivirus 2013 12.2
Norton AntiVirus 2013 13.2
avast! Antivirus 2013 13.3
Avira Antivirus Free 2013 14.1
AVG AntiVirus Free 2013 14.6
AVG Anti-Virus 2013 14.7
avast! Free Antivirus 7 15.1
McAfee AntiVirus 2013 19.0
Average 21.9
ESET Antivirus 6 23.8
G Data AntiVirus 2013 144.7
0 s 20 s 40 s 60 s 80 s 100 s 120 s 140 s 160 s
Performance Benchmark Page 42 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average time it takes for five sample files to be converted from one file format to another (MP3 WMA, MP3 WAV) for each Antivirus product tested. Products with lower times are considered better performing products in this category.
Avira Antivirus Free 2013 75.7
Microsoft Security Essentials 4.0 75.8
AVG Anti-Virus 2013 75.8
Webroot AntiVirus 2013 75.9
Quick Heal Antivirus 2013 75.9
Panda Cloud Antivirus FREE 2013 76.0
Kaspersky Anti-Virus 2013 76.1
Panda Antivirus 2013 76.4
Norton AntiVirus 2013 76.5
G Data AntiVirus 2013 76.7
Average 76.7
ESET Antivirus 6 77.4
Trend Micro Antivirus 2013 77.5
AVG AntiVirus Free 2013 77.6
avast! Antivirus 2013 77.8
McAfee AntiVirus 2013 78.0
avast! Free Antivirus 7 78.3
74 s 75 s 75 s 76 s 76 s 77 s 77 s 78 s 78 s 79 s
Performance Benchmark Page 43 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average time it takes for sample files to be compressed and decompressed for each Antivirus product tested. Products with lower times are considered better performing products in this category.
G Data AntiVirus 2013 78.2
Webroot AntiVirus 2013 79.5
Norton AntiVirus 2013 80.3
Kaspersky Anti-Virus 2013 81.0
AVG Anti-Virus 2013 81.3
AVG AntiVirus Free 2013 84.7
Quick Heal Antivirus 2013 84.9
avast! Free Antivirus 7 85.1
Average 85.2
Panda Antivirus 2013 85.7
ESET Antivirus 6 85.9
Panda Cloud Antivirus FREE 2013 85.9
Microsoft Security Essentials 4.0 86.0
avast! Antivirus 2013 86.9
Avira Antivirus Free 2013 87.1
McAfee AntiVirus 2013 90.9
Trend Micro Antivirus 2013 99.7
0 s 20 s 40 s 60 s 80 s 100 s 120 s
Performance Benchmark Page 44 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average time it takes for a file to be written to the hard drive then opened and closed 180,000 times, for each Antivirus product tested. Products with lower times are considered better performing products in this category.
Webroot AntiVirus 2013 15.1
AVG Anti-Virus 2013 16.4
Kaspersky Anti-Virus 2013 17.9
AVG AntiVirus Free 2013 23.8
Quick Heal Antivirus 2013 24.3
Panda Antivirus 2013 24.8
avast! Free Antivirus 7 26.7
Avira Antivirus Free 2013 27.1
Norton AntiVirus 2013 28.3
Panda Cloud Antivirus FREE 2013 33.4
avast! Antivirus 2013 33.5
McAfee AntiVirus 2013 38.4
Trend Micro Antivirus 2013 72.0
Average 72.5
ESET Antivirus 6 101.1
Microsoft Security Essentials 4.0 271.8
G Data AntiVirus 2013 404.8
0 s 50 s 100 s 150 s 200 s 250 s 300 s 350 s 400 s 450 s
Performance Benchmark Page 45 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average time taken to scan a set of 6351 portable executable files (totaling 2076 MB) for each Antivirus product tested. This time is calculated by averaging the initial (Run 1) and subsequent (Runs 2-5) scan times. Products with lower scan times are considered better performing products in this category
Kaspersky Anti-Virus 2013 33.5
Quick Heal Antivirus 2013 51.5
McAfee AntiVirus 2013 66.1
G Data AntiVirus 2013 69.1
AVG AntiVirus Free 2013 71.3
Webroot AntiVirus 2013 76.9
Trend Micro Antivirus 2013 77.8
Norton AntiVirus 2013 90.9
Avira Antivirus Free 2013 93.5
Average 96.3
Panda Cloud Antivirus FREE 2013 111.8
Microsoft Security Essentials 4.0 112.3
avast! Antivirus 2013 115.5
ESET Antivirus 6 118.1
avast! Free Antivirus 7 118.6
AVG Anti-Virus 2013 143.8
Panda Antivirus 2013 190.9
0 s 50 s 100 s 150 s 200 s 250 s
Performance Benchmark Page 46 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average time taken to copy a set of files from one local drive to another local drive for each Antivirus product tested. The test was performed 5 times, and the average of all 5 runs was taken as the result. Products with lower times are considered better performing products in this category.
Microsoft Security Essentials 4.0 102.2
G Data AntiVirus 2013 111.2
Trend Micro Antivirus 2013 113.0
Kaspersky Anti-Virus 2013 115.1
AVG Anti-Virus 2013 117.3
Webroot AntiVirus 2013 117.9
Norton AntiVirus 2013 121.7
Avira Antivirus Free 2013 121.9
avast! Free Antivirus 7 123.9
AVG AntiVirus Free 2013 123.9
avast! Antivirus 2013 124.1
Average 130.3
Panda Antivirus 2013 135.4
Quick Heal Antivirus 2013 139.6
ESET Antivirus 6 143.7
McAfee AntiVirus 2013 166.7
Panda Cloud Antivirus FREE 2013 206.9
0 s 50 s 100 s 150 s 200 s 250 s
Performance Benchmark Page 47 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
In the following charts, we have highlighted the results we obtained for Norton 360 in yellow. The average has also been highlighted in blue for ease of comparison.
The following chart compares the average time taken for the system to boot (from a sample of five boots) for each Total Security product tested. Products with lower boot times are considered better performing products in this category.
McAfee Total Protection 2013 33.4
Norton 360 33.5
Trend Micro Maximum Security 2013 34.3
Average 34.4
Kaspersky PURE 35.0
Bitdefender Total Security 2013 35.9
32 s 33 s 33 s 34 s 34 s 35 s 35 s 36 s 36 s 37 s
The following chart compares the average time taken to scan a set of 6159 files (totaling 982 MB) for each Total Security product tested. This time is calculated by averaging the initial (Run 1) and subsequent (Runs 2-5) scan times. Products with lower scan times are considered better performing products in this category
Norton 360 21
Bitdefender Total Security 2013 42
Trend Micro Maximum Security 2013 43
McAfee Total Protection 2013 48
Average 49
Kaspersky PURE 89
0 s 10 s 20 s 30 s 40 s 50 s 60 s 70 s 80 s 90 s 100 s
Performance Benchmark Page 48 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average time taken to launch a product’s user interface. Products with lower launch times are considered better performing products in this category.
Norton 360 112
McAfee Total Protection 2013 327
Average 559
Trend Micro Maximum Security 2013 765
Bitdefender Total Security 2013 778
Kaspersky PURE 814
0 ms 100 ms 200 ms 300 ms 400 ms 500 ms 600 ms 700 ms 800 ms 900 ms
The following chart compares the average amount of RAM in use by a Total Security product during a period of system idle. This average is taken from a sample of ten memory snapshots taken at roughly 60 seconds apart after reboot. Products with lower idle RAM usage are considered better performing products in this category.
Norton 360 25.6
Kaspersky PURE 62.9
Trend Micro Maximum Security 2013 79.1
Bitdefender Total Security 2013 92.7
Average 103.4
McAfee Total Protection 2013 256.3
0 MB 50 MB 100 MB 150 MB 200 MB 250 MB 300 MB
Performance Benchmark Page 49 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average time taken for Internet Explorer to successively load a set of popular websites through the local area network from a local server machine. Products with lower browse times are considered better performing products in this category.
Norton 360 42.2
Kaspersky PURE 50.5
Trend Micro Maximum Security 2013 53.4
Average 57.6
McAfee Total Protection 2013 69.2
Bitdefender Total Security 2013 72.9
0 s 10 s 20 s 30 s 40 s 50 s 60 s 70 s 80 s
The following chart compares the average launch times of Internet Explorer after rebooting the machine for each Total Security product we tested. Products with lower launch times are considered better performing products in this category.
McAfee Total Protection 2013 1599
Norton 360 1853
Kaspersky PURE 1989
Average 2162
Bitdefender Total Security 2013 2238
Trend Micro Maximum Security 2013 3130
0 ms 500 ms 1,000 ms 1,500 ms 2,000 ms 2,500 ms 3,000 ms 3,500 ms
Performance Benchmark Page 50 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the minimum installation time it takes for Total Security products to be fully functional and ready for use by the end user. Products with lower installation times are considered better performing products in this category.
Norton 360 59
Kaspersky PURE 194
Trend Micro Maximum Security 2013 238
Average 292
Bitdefender Total Security 2013 294
McAfee Total Protection 2013 673
0 s 100 s 200 s 300 s 400 s 500 s 600 s 700 s 800 s
The following chart compares the total size of files added during the installation of Total Security products. Products with lower installation sizes are considered better performing products in this category.
Kaspersky PURE 292.3
McAfee Total Protection 2013 355.3
Norton 360 356.7
Average 540.2
Trend Micro Maximum Security 2013 549.2
Bitdefender Total Security 2013 1147.2
0 MB 200 MB 400 MB 600 MB 800 MB 1,000 MB 1,200 MB 1,400 MB
Performance Benchmark Page 51 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the amount of Registry Keys created during product installation for each Total Security product tested. Products with lower key counts are considered better performing products in this category.
Trend Micro Maximum Security 2013 5134
Norton 360 5539
Bitdefender Total Security 2013 6321
Kaspersky PURE 7961
Average 9651
McAfee Total Protection 2013 23302
0 5000 10000 15000 20000 25000
The following chart compares the average time taken to copy, move and delete several sets of sample files for each Total Security product tested. Products with lower times are considered better performing products in this category.
Norton 360 25.6
Bitdefender Total Security 2013 28.4
Kaspersky PURE 28.8
Trend Micro Maximum Security 2013 31.5
Average 34.2
McAfee Total Protection 2013 56.5
0 s 10 s 20 s 30 s 40 s 50 s 60 s
Performance Benchmark Page 52 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average time taken to install a third party application for each Total Security product tested. Products with lower times are considered better performing products in this category.
Norton 360 104.2
Bitdefender Total Security 2013 105.4
Kaspersky PURE 109.9
Average 112.0
Trend Micro Maximum Security 2013 113.7
McAfee Total Protection 2013 126.8
0 s 20 s 40 s 60 s 80 s 100 s 120 s 140 s
The following chart compares the average time to download a sample set of common file types for each Total Security product tested. Products with lower times are considered better performing products in this category.
Trend Micro Maximum Security 2013 11.9
Bitdefender Total Security 2013 13.5
Norton 360 13.6
Average 14.8
Kaspersky PURE 15.2
McAfee Total Protection 2013 19.8
0 s 5 s 10 s 15 s 20 s 25 s
Performance Benchmark Page 53 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average time it takes for five sample files to be converted from one file format to another (MP3 WMA, MP3 WAV) for each Total Security product tested. Products with lower times are considered better performing products in this category.
Bitdefender Total Security 2013 75.9
Kaspersky PURE 76.5
Average 76.6
McAfee Total Protection 2013 76.6
Norton 360 76.6
Trend Micro Maximum Security 2013 77.3
0 s 10 s 20 s 30 s 40 s 50 s 60 s 70 s 80 s 90 s
The following chart compares the average time it takes for sample files to be compressed and decompressed for each Total Security product tested. Products with lower times are considered better performing products in this category.
Kaspersky PURE 80.5
Norton 360 80.5
Bitdefender Total Security 2013 84.7
Average 89.4
Trend Micro Maximum Security 2013 97.4
McAfee Total Protection 2013 103.7
0 s 20 s 40 s 60 s 80 s 100 s 120 s
Performance Benchmark Page 54 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average time it takes for a file to be written to the hard drive then opened and closed 180,000 times, for each Total Security product tested. Products with lower times are considered better performing products in this category.
Kaspersky PURE 25.1
Norton 360 28.4
Trend Micro Maximum Security 2013 71.5
McAfee Total Protection 2013 81.6
Average 230.5
Bitdefender Total Security 2013 945.9
0 s 100 s 200 s 300 s 400 s 500 s 600 s 700 s 800 s 900 s 1,000 s
The following chart compares the average time taken to scan a set of 6351 portable executable files (totaling 2076 MB) for each Total Security product tested. This time is calculated by averaging the initial (Run 1) and subsequent (Runs 2-5) scan times. Products with lower scan times are considered better performing products in this category
Kaspersky PURE 68
McAfee Total Protection 2013 83
Norton 360 93
Average 93
Trend Micro Maximum Security 2013 106
Bitdefender Total Security 2013 114
0 s 20 s 40 s 60 s 80 s 100 s 120 s
Performance Benchmark Page 55 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average time taken to copy a set of files from one local drive to another local drive for each Total Security product tested. The test was performed 5 times, and the average of all 5 runs was taken as the result. Products with lower times are considered better performing products in this category.
Trend Micro Maximum Security 2013 109
Kaspersky PURE 116
Bitdefender Total Security 2013 124
Norton 360 141
Average 153
McAfee Total Protection 2013 276
0 s 50 s 100 s 150 s 200 s 250 s 300 s
Performance Benchmark Page 56 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
In the following charts, we have highlighted the results we obtained for Norton Internet Security 2013 in yellow. The average has also been highlighted in blue for ease of comparison.
The following chart compares the average time taken for the system to boot (from a sample of five boots)for each Internet Security product tested. Products with lower boot times are considered better performing products in this category.
Trend Micro Internet Security 2013 7.9
Norton Internet Security 2013 8.8
Kaspersky Internet Security 2013 9.2
Average 9.3
BitDefender Internet Security 2013 9.8
McAfee Internet Security 2013 10.7
0 s 2 s 4 s 6 s 8 s 10 s 12 s
The following chart compares the average time taken to scan a set of 6159 files (totaling 982 MB) for each Internet Security product tested. This time is calculated by averaging the initial (Run 1) and subsequent (Runs 2- 5) scan times. Products with lower scan times are considered better performing products in this category
Norton Internet Security 2013 10
BitDefender Internet Security 2013 13
McAfee Internet Security 2013 25
Average 25
Trend Micro Internet Security 2013 37
Kaspersky Internet Security 2013 42
0 s 5 s 10 s 15 s 20 s 25 s 30 s 35 s 40 s 45 s
Performance Benchmark Page 57 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
The following chart compares the average time taken to copy, move and delete several sets of sample files for each Internet Security product tested. Products with lower times are considered better performing products in this category.
Norton Internet Security 2013 7.4
BitDefender Internet Security 2013 9.0
Kaspersky Internet Security 2013 10.2
Trend Micro Internet Security 2013 11.5
Average 13.4
McAfee Internet Security 2013 29.0
0 s 5 s 10 s 15 s 20 s 25 s 30 s 35 s
The following chart compares the average time taken to copy a set of files from one local drive to another local drive for each Internet Security product tested. The test was performed 5 times, and the average of all 5 runs was taken as the result. Products with lower times are considered better performing products in this category.
BitDefender Internet Security 2013 84.6
Kaspersky Internet Security 2013 84.7
Trend Micro Internet Security 2013 94.6
Average 130.3
Norton Internet Security 2013 164.7
McAfee Internet Security 2013 222.7
0 s 50 s 100 s 150 s 200 s 250 s
Performance Benchmark Page 58 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
This report only covers versions of products that were available at the time of testing. The tested versions are as noted in the “Products and Versions” section of this report. The products we have tested are not an exhaustive list of all products available in these very competitive product categories.
While every effort has been made to ensure that the information presented in this report is accurate, PassMark Software Pty Ltd assumes no responsibility for errors, omissions, or out-of-date information and shall not be liable in any manner whatsoever for direct, indirect, incidental, consequential, or punitive damages resulting from the availability of, use of, access of, or inability to use this information.
Symantec Corporation funded the production of this report, selected the test metrics and list of products to include in this report, and supplied some of the test scripts used for the tests.
All trademarks are the property of their respective owners.
PassMark Software Pty Ltd Suite 202, Level 2 35 Buckingham St. Surry Hills, 2010 Sydney, Australia Phone + 61 (2) 9690 0444 Fax + 61 (2) 9690 0445 Web www.passmark.com
An electronic copy of this report can be found at the following location: http://www.passmark.com/tpsreport13
Performance Benchmark Page 59 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
For our testing, PassMark Software used a test environment running Windows 7 Ultimate (64-bit) SP1 with the following hardware specifications:
CPU: Intel Core i7 920 Quad Core @ 2.67GHz Video Card: nVidia GeForce 8800 GT Motherboard: Intel x58 Motherboard RAM: 6GB DDR3 RAM HDD : Western Digital 500GB 7200RPM Network: Gigabit (1GB/s) switch
Performance Benchmark Page 60 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
As with testing on Windows Vista, Norton Ghost was used to create a “clean” baseline image prior to testing. Our aim is to create a baseline image with the smallest possible footprint and reduce the possibility of variation caused by external operating system factors.
The baseline image was restored prior to testing of each different product. This process ensures that we install and test all products on the same, “clean” machine.
The steps taken to create the base Windows 7 image are as follows:
1. Installation and activation of Windows 7 Ultimate Edition. 2. Disabled Automatic Updates. 3. Changed User Account Control settings to “Never Notify”. 4. Disable Windows Defender automatic scans to avoid unexpected background activity. 5. Disable the Windows firewall to avoid interference with security software. 6. Installed Norton Ghost for imaging purposes. 7. Disabled Superfetch to ensure consistent results. 8. Installed HTTP Watch for Browse Time testing. 9. Installed Windows Performance Toolkit x64 for Boot Time testing. 10. Installed Active Perl for interpretation of some test scripts. 11. Install OSForensics for testing (Installation Size and Registry Key Count tests) purposes. 12. Disabled updates, accelerators and compatibility view updates in Internet Explorer 8. 13. Update to Windows Service Pack 1 14. Created a baseline image using Norton Ghost.
PassMark Software uses tools available from the Windows Performance Toolkit version 4.6 (as part of the Microsoft Windows 7 SDK obtainable from the Microsoft Website) with a view to obtaining more precise and consistent boot time results on the Windows 7 platform.
The boot process is first optimized with xbootmgr.exe using the command “xbootmgr.exe -trace boot – prepSystem” which prepares the system for the test over six optimization boots. The boot traces obtained from the optimization process are discarded.
After boot optimization, the benchmark is conducted using the command "xbootmgr.exe -trace boot -numruns 5”. This command boots the system five times in succession, taking detailed boot traces for each boot cycle.
Finally, a post-processing tool was used to parse the boot traces and obtain the BootTimeViaPostBoot value. This value reflects the amount of time it takes the system to complete all (and only) boot time processes. Our final result is an average of five boot traces.
Performance Benchmark Page 61 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
Scan Time is the time it took for each product to scan a set of sample files. The sample used was identical in all cases and contained a mixture of system files and Office files. In total there were 6159 files whose combined size was 982 MB. Most of these files come from the Windows system folders. As the file types can influence scanning speed, the breakdown of the main file types, file numbers and total sizes of the files in the sample set is given here:
File Extension Number of Files File Size
.dll 2589 490MB .exe 695 102MB .sys 332 23MB .gif 302 1MB .doc 281 64MB .wmf 185 2MB .png 149 2MB .html 126 1MB .nls 80 6MB .jpg 70 1MB .ini 59 2MB .ico 58 <1MB .mof 43 6MB .ax 39 4MB .xls 38 3MB .ime 35 5MB .drv 31 1MB .txt 31 1MB .chm 30 6MB .cpl 29 4MB .mfl 29 3MB .inf 26 2MB .hlp 22 3MB .imd 20 18MB .py 20 <1MB .msc 18 1MB .vbs 18 1MB .xml 18 1MB .rtf 16 62MB .ocx 16 4MB .tsp 14 1MB .com 14 <1MB .xsl 14 <1MB .h 13 <1MB .vsd 12 2MB .scr 12 2MB .aw 12 2MB .js 12 1MB .zip 11 25MB .lex 9 10MB .ppt 9 4MB .acm 9 1MB .wav 7 5MB Total 6159 982
Performance Benchmark Page 62 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
This scan was run without launching the product’s user interface, by right-clicking the test folder and choosing the “Scan Now” option. To record the scan time, we have used product’s built-in scan timer or reporting system. Where this was not possible, scan times were taken manually with a stopwatch.
For each product, five samples were taken with the machine rebooted before each sample to clear any caching effects by the operating systems.
In 2009, we noticed many more products showing a substantial difference between the initial scan time (first scan) and subsequent scan times (scans 2 to 5). We believe this behavior is due to products themselves caching recently scanned files.
As a result of this mechanism, we have averaged the four subsequent scan times to obtain an average subsequent scan time. Our final result for this test is an average of the subsequent scan average and the initial scan time.
The launch time of a product’s user interface was taken using AppTimer (v1.0.1006). For each product tested, we obtained a total of fifteen samples from five sets of three UI launches, with a reboot before each set to clear caching effects by the operating system. When compiling the results the first of each set was separated out so that there was a set of values for the initial launch after reboot and a set for subsequent launches.
We have averaged the subsequent launch times to obtain an average subsequent launch time. Our final result for this test is an average of the subsequent launch average and the initial launch time.
In some cases, AppTimer did not correctly record the time taken for UI launch. For instance, some applications would open their window and look like they were ready, but then continued to be unresponsive. Where the measurement from AppTimer appeared inaccurate, we have taken the time manually with a stop watch.
AppTimer is publically available from the PassMark Website.
The Perflog++ utility was used to record process memory usage on the system at boot, and then every minute for another fifteen minutes after. This was done only once per product and resulted in a total of 15 samples. The first sample taken at boot is discarded.
The PerfLog++ utility records memory usage of all processes, not just those of the anti-malware product. As a result of this, an anti-malware product’s processes needed to be isolated from all other running system processes. To isolate relevant process, we used a program called Process Explorer which was run immediately upon the completion of memory usage logging by PerfLog++. Process Explorer is a Microsoft Windows Sysinternals software tool which shows a list of all DLL processes currently loaded on the system.
We used a script in conjunction with HTTPWatch (Basic Edition, version 6.1) to record the amount of time it takes for a set of 106 ‘popular’ websites to load consecutively from a local server. This script feeds a list of URLs into HTTPWatch, which instructs the browser to load pages in sequence and monitors the amount of time it takes for the browser to load all items on one page. Performance Benchmark Page 63 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
For this test, we have used Internet Explorer 9 (Version 9.0.8112.16421) as our browser.
The set of websites used in this test include front pages of high traffic pages. This includes shopping, social, news, finance and reference websites.
The Browse Time test is executed five times and our final result is an average of these five samples. The local server is restarted between different products and one initial ‘test’ run is conducted prior to testing to install Adobe Flash Player, an add-on which is used by many popular websites.
The average launch time of Internet Explorer interface was taken using AppTimer. This test was practically identical to the User Interface launch time test. For each product tested, we obtained a total of fifteen samples from five sets of three Internet Explorer launches, with a reboot before each set to clear caching effects by the operating system. When compiling the results the first of each set was separated out so that there was a set of values for the initial launch after reboot and a set for subsequent launches.
For this test, we have used Internet Explorer 8 (Version 8.0.6001.18783) as our test browser.
We have averaged the subsequent launch times to obtain an average subsequent launch time. Our final result for this test is an average of the subsequent launch average and the initial launch time.
This test measures the minimum Installation Time a product requires to be fully functional and ready for use by the end user. Installation time can usually be divided in three major phases:
The Extraction and Setup phase consists of file extraction, the EULA prompt, product activation and user configurable options for installation.
The File Copy phase occurs when the product is being installed; usually this phase is indicated by a progress bar.
The Post-Installation phase is any part of the installation that occurs after the File Copy phase. This phase varies widely between products; the time recorded in this phase may include a required reboot to finalize the installation or include the time the program takes to become idle in the system tray.
To reduce the impact of disk drive variables, each product was copied to the Desktop before initializing installation. Each step of the installation process was manually timed with a stopwatch and recorded in as much detail as possible. Where input was required by the end user, the stopwatch was paused and the input noted in the raw results in parenthesis after the phase description.
Where possible, all requests by products to pre-scan or post-install scan were declined or skipped. Where it was not possible to skip a scan, the time to scan was included as part of the installation time. Where an optional component of the installation formed a reasonable part of the functionality of the software, it was also installed (e.g. website link checking software as part of an Internet Security Product).
Performance Benchmark Page 64 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
Installation time includes the time taken by the product installer to download components required in the installation. This may include mandatory updates or the delivery of the application itself from a download manager. We have noted in our results where a product has downloaded components for product installation.
We have excluded product activation times due to network variability in contacting vendor servers or time taken in account creation.
A product's Installation Size was previously defined as the difference between the initial snapshot of the Disk Space (C: drive) before installation and the subsequent snapshot taken after the product is installed on the system. Although this is a widely used methodology, we noticed that the results it yielded were not always reproducible in Vista due to random OS operations that may take place between the two snapshots. We improved the Installation Size methodology by removing as many Operating System and disk space variables as possible.
Using PassMark’s OSForensics we created initial and post-installation disk signatures for each product. These disk signatures recorded the amount of files and directories, and complete details of all files on that drive (including file name, file size, checksum, etc) at the time the signature was taken.
The initial disk signature was taken immediately prior to installation of the product. A subsequent disk signature was taken immediately following a manual update and system reboot after product installation. Using OSForensics, we compared the two signatures and calculated the total disk space consumed by files that were new, modified, and deleted during product installation. Our result for this metric reflects the total size of all newly added files during installation.
This test measures the amount of keys and values added to registry, after rebooting the test machine following a successful product installation. The test was conducted using PassMark’s OSForensics, to count the number of keys, errors and values that were added under HKEY_LOCAL_MACHINE and HKEY_USERS. The Create Signature feature is used to take a before and after signature of these folders, and the signatures are then compared so that the new keys can be identified.
We used a single script in testing Benchmarks 10-15. The script consecutively executes tests for Benchmarks 10- 15. The script times each phase in these benchmarks using CommandTimer.exe and appends results to a log file.
This test measures the amount of time required for the system to copy, move and delete samples of files in various file formats. This sample was made up of 812 files over 760,867,636 bytes and can be categorized as documents [26% of total], media files [54% of total] and PE files (i.e. System Files) [20% of total].
The breakdown of the main file types, file numbers and total sizes of the files in the sample set is shown in the following table:
File format Number Size (bytes) Performance Benchmark Page 65 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
DOC 8 30,450,176
DOCX 4 13,522,409
PPT 3 5,769,216
PPTX 3 4,146,421
XLS 4 2,660,352
XLSX 4 1,426,054
PDF 73 136,298,049
ZIP 4 6,295,987
7Z 1 92,238
JPG 351 31,375,259
GIF 6 148,182
MOV 7 57,360,371
RM 1 5,658,646
AVI 8 78,703,408
WMV 5 46,126,167
MP3 28 191,580,387
EXE 19 2,952,914
DLL 104 29,261,568
AX 1 18,432
CPL 2 2,109,440
CPX 2 4,384
DRV 10 154,864
ICO 1 107,620
MSC 1 41,587
NT 1 1,688
ROM 2 36,611
SCR 2 2,250,240
SYS 1 37,528,093
TLB 3 135,580
TSK 1 1,152
UCE 1 22,984
EXE 19 2,952,914
DLL 104 29,261,568
AX 1 18,432
CPL 2 2,109,440
CPX 2 4,384
DRV 10 154,864
Performance Benchmark Page 66 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
ICO 1 107,620
MSC 1 41,587
NT 1 1,688
ROM 2 36,611
SCR 2 2,250,240
SYS 1 37,528,093
TLB 3 135,580
TSK 1 1,152
UCE 1 22,984
Total 812 760,867,636
This test was conducted five times to obtain the average time to copy, move and delete the sample files, with the test machine rebooted between each sample to remove potential caching effects.
This test measured how much time was required to install and uninstall a third party application. For this test, CommandTimer.exe timed how long it took to install and uninstall the following applications on the test machine:
Firefox 3.6.3 (11,909 KB) (MSI File) Microsoft .NET 3.5 (34,121 KB) (MSI File) Steam (1,551 KB) (MSI File)
This test was conducted five times to obtain the average time to install/uninstall the above third party programs, with the test machine rebooted between each sample to remove potential caching effects.
This benchmark measured how much time was required to download a sample set of binary files of various sizes and types over a 100MB/s network connection. The files were hosted on a server machine running Windows Server 2008 and IIS 7. CommandTimer.exe was used in conjunction with GNU Wget (version 1.10.1) to time and conduct the download test.
The complete sample set of files was made up of 553,638,694 bytes over 484 files and two file type categories: media files [74% of total] and documents [26% of total]. The breakdown of the file types, file numbers and total sizes of the files in the sample set is shown in the following table:
File format Number Size (bytes)
JPEG 343 30,668,312
GIF 9 360,349
PNG 5 494,780
MOV 7 57,360,371
RM 1 5,658,646
AVI 8 78,703,408 Performance Benchmark Page 67 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
WMV 5 46,126,167
MP3 28 191,580,387
PDF 73 136,298,049
ZIP 4 6,295,987
7Z 1 92,238
Total 484 553,638,694
This test was conducted five times to obtain the average time to download this sample of files, with the test machine rebooted between each sample to remove potential caching effects.
This test measured how much time was required to convert five (5) different MP3 files into WAV files and subsequently, convert the same MP3 samples into a WMA files. The total size of the five (5) MP3s used was 25,870,899 bytes.
To encode the MP3 into another format, we used an application called ffmpeg.exe. The format conversion process was timed using CommandTimer.exe.
This test was conducted five times to obtain the average conversion speed between these formats, with the test machine rebooted between each sample to remove potential caching effects.
This test measured the amount of time required to compress and decompress a sample set of files. For this test, we used a subset of the media and documents files used in the File Copy, Move and Delete benchmark. CommandTimer.exe recorded the amount of time required for 7zip.exe to compress the files into a *.zip and subsequently decompress the created *.zip file.
This subset comprised 1,218 files over 783 MB. The breakdown of the file types, file numbers and total sizes of the files in the sample set is shown in the following table:
File Type File Number Total Size
.xls 13 9.23 MB
.xlsx 9 3.51 MB
.ppt 9 7.37 MB
.pptx 11 17.4 MB
.doc 17 35.9 MB
.docx 19 24.5 MB
.gif 177 1.10 MB
.jpg 737 66.2 MB
.png 159 48.9 MB
.mov 7 54.7 MB
Performance Benchmark Page 68 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
.rm 1 5.39 MB
.avi 46 459 MB
.wma 11 48.6 MB
.avi 46 459 MB
.wma 11 48.6 MB
Total 1218 783 MB
This test was conducted five times to obtain the average file compression and decompression speed, with the test machine rebooted between each sample to remove potential caching effects.
This benchmark was derived from Oli Warner’s File I/O test at http://www.thepcspy.com (please see Reference #1: What Really Slows Windows Down).
For this test, we developed OpenClose.exe, an application that looped writing a small file to disk, then opening and closing that file. CommandTimer.exe was used to time how long the process took to complete 180,000 cycles.
This test was conducted five times to obtain the average file writing, opening and closing speed, with the test machine rebooted between each sample to remove potential caching effects.
This test measures the on demand scan times of a file set comprised only of executable files (.exe, .dll and .sys files). We performed five scans of the sample file set, with a machine restart between each scan to remove possible caching effects. The time taken to scan the files is taken from an antivirus product’s scan logs, or where logs are not available, manually with a stopwatch. Scans were launched by right clicking on the folder to be scanned.
A breakdown of the sample file set is as follows:
File Type Number File Size of Files Sys Files 2174 329MB
Dll Files 2037 920MB
Exe Files 2140 827MB
Total 6351 2076MB
The final result is calculated as an average of the five samples.
This test measures the amount of time taken to copy files between two local drives. The data set comprised of 8,501 files with a total file size of 5.44GB, and the formats used included documents, movies, images and executables. A breakdown of the sample file set is given below: Performance Benchmark Page 69 of 70 Edition 4 26 April 2013
2013 Consumer Security Products PassMark Software
File Number File Size Extension of Files
.jpg 2903 588MB
.dll 773 25MB
.exe 730 197MB
.gif 681 63MB
.wav 430 260MB
.sys 501 79MB
.png 451 27MB
.mp3 333 2157MB
.wma 585 925MB
.docx 267 81MB
.avi 247 1079MB
.doc 160 57MB
.xls 329 132MB
.ppt 97 148MB
.zip 14 177MB
Total 8501 5995MB
A total of five runs of this test were performed, with a machine restart between each run. The time taken to copy files was measured and recorded by CommandTimer.exe. All the files were copied between a folder on the local drive and a 2nd folder on a different drive. Files were deleted from the 2nd drive once the copy was complete. The final result is calculated as an average of the five samples
Performance Benchmark Page 70 of 70 Edition 4 26 April 2013