Norton 360 for Gamers Game Optimization Testing Windows 10
June 2021
Document: Norton 360 for Gamers Game Optimization Testing (Edition 1) Authors: J. Han, D. Wren Company: PassMark Software Date: 22 June 2021 Edition: 1 File: Norton_Game_Optimization_Testing_2021_Ed1.docx Norton 360 for Gamers Game Optimization Testing PassMark Software
Table of Contents
REVISION HISTORY ...... 3 EXECUTIVE SUMMARY ...... 4 OVERALL SCORE ...... 5 PRODUCTS LIST ...... 6 TEST RESULTS...... 7
PHASE 1 ...... 7 PHASE 2 ...... 15 CONCLUSION ...... 17
SCENARIOS WHERE THERE WAS A BENEFIT ...... 17 SCENARIOS WHERE THERE WAS NO BENEFIT...... 17 DISCLAIMER AND DISCLOSURE ...... 18
DISCLAIMER OF LIABILITY...... 18 DISCLOSURE ...... 18 TRADEMARKS ...... 18 CONTACT DETAILS...... 18 APPENDIX 1 – TEST ENVIRONMENT ...... 19
TESTING MACHINE ...... 19 TESTING GAMES ...... 19 APPENDIX 2 –METHODOLOGY DESCRIPTION ...... 20
WINDOWS 10 IMAGE CREATION ...... 20 GAME BENCHMARK TEST ...... 20 TEST PHASE 1...... 20 TEST PHASE 2...... 21
Edition 1 Page 2 of 21
22 June 2021
Norton 360 for Gamers Game Optimization Testing PassMark Software
Revision History
Rev Revision History Date
Edition 1 Initial version of report 22 June 2021
Edition 1 Page 3 of 21
22 June 2021
Norton 360 for Gamers Game Optimization Testing PassMark Software
Executive Summary
PassMark Software conducted objective performance testing of the Game Optimizer (GO) feature in the Norton 360 for Gamers product.
The performance benchmark test consisted of two phases. The aim of the first phase was to test the effect of GO on various games, both on a clean install of Windows 10 (Phase 1.1) and also with background activity (Phase 1.2). The second phase (Phase 2) is to compare the Norton 360 for Gamers product with competing Anti-virus products, which also advertised similar game optimizing features.
To test the game optimization performance, we selected five common PC games and used the games built-in benchmarks to measure the minimum, maximum and average FPS (frames per second) metrics. We ran some background tasks in parallel with the games to test the Norton GO game optimizing performance and we used a synthetic CPU load tool to produce a constant CPU consumption when comparing the Norton 360 for Gamers with the competitors’ products.
We found that Norton GO improves frame rate performance in games in some scenarios. It is of particular benefit when there are high CPU usage activities running in the background during the game play and the CPU is creating a bottleneck. In comparison to similar anti-virus products, Norton 360 for Gamers performed the best in the product comparison. Further details are in the report below.
Edition 1 Page 4 of 21
22 June 2021
Norton 360 for Gamers Game Optimization Testing PassMark Software
Overall Score
PassMark Software assigned a score to each product we tested in the Phase 2 test, depending on its ranking of the average FPS metric compared to other products in the category.
In the following table the highest possible score attainable is 30; in a hypothetical situation where a product has attained first place in all 5 games. The table below shows the products ranked by their overall scores:1
Product Name Overall Score
Norton 360 for Gamers 30
Webroot SecureAnywhere AntiVirus for PC Gamers 20
ESET NOD32 Antivirus 17
McAfee Gamer Security 16
Kaspersky Total Security 11
Bitdefender Antivirus Plus 9
1 The Razer Cortex product was not included in the comparison as it did not contain any anti-virus functionality.
Edition 1 Page 5 of 21
22 June 2021
Norton 360 for Gamers Game Optimization Testing PassMark Software
Products List
The tested game optimization product names and versions are as follows:
Product Name Version
Norton 360 for Gamers 22.21.5.44
Razer Cortex 9.14.15.1361
McAfee Gamer Security 1.7.128
ESET NOD32 Antivirus 14.1.20.0
Bitdefender Antivirus Plus 25.0.21.78
Kaspersky Total Security 21.3.10.391 (b)
Webroot SecureAnywhere AntiVirus for PC Gamers 9.0.30.75
Edition 1 Page 6 of 21
22 June 2021
Norton 360 for Gamers Game Optimization Testing PassMark Software
Test Results
We performed two phases of testing. The results obtained from each phase test have been plotted on charts for ease of comparison.
Phase 1
In the first stage of the Phase 1 test, we ran the frame rate benchmarks on a clean install of Windows 10 without Norton GO installed and also with the Microsoft Defender turned off. Then the test was repeated with the Norton GO active to compare the two results.
No Background Task
46.82 Ghost Recon Wildlands 47.04
59.90 Sleeping Dogs 59.90
59.00 Forza Horizon 4 59.00
35.00 Shadow of the Tomb Raider 35.00
42.00 Far Cry 5 42.00
0 10 20 30 40 50 60 70 Average FPS
Norton GO not installed Norton GO installed
Note that the above test was ran with NVIDIA GeForce GTX 960 graphic card, and subsequently we replaced the card with NVIDIA GeForce RTX 2060 and used the new graphic card for the remaining of the tests. This was done to remove the GPU as a significant bottleneck.
Edition 1 Page 7 of 21
22 June 2021
Norton 360 for Gamers Game Optimization Testing PassMark Software
In the second stage of the Phase 1 test, we ran one or more third-party applications in parallel with the games.
The types of background tasks were: • Downloading games on Steam • Chrome loading Web pages • Video encoding with Handbrake • Game recording with Game DVR • Chrome Cleanup computer • CPU stress with GBCPUStressTest
These background activities produced different levels of CPU loads on the system while running games benchmarks.
We ran the background tasks benchmarks with no Norton GO product installed (Microsoft Defender was turned off). Then the test was repeated with the Norton 360 for Gamers installed to test the effect of game enhancement feature of the Norton GO. We also ran the benchmarks with no background tasks, Norton GO not installed, and the Microsoft Defender turned off to produce a baseline. The average percentage increases in the average FPS metric achieved by the Norton GO are shown in the table:
Shadow of the Forza Ghost Recon Background Tasks Far Cry 5 Sleeping Dogs Tomb Raider Horizon 4 Wildlands
Downloading games on Steam 30.57% 13.77% 3.38% 5.15% 11.76%
Chrome loading Web pages 0.35% 2.22% 0.86% 0.86% -0.21%
Video encoding with Handbrake 40.69% 35.68% 13.75% 30.43% 24.61%
Game recording with Game DVR 0.66% 0.69% 0.00% -0.54% 0.56%
Chrome Cleanup computer 5.88% -3.13% 2.31% 3.10% 1.92%
CPU stress with GBCPUStressTest 43.63% 19.09% 6.43% 10.75% 47.93%
The Average Percentage Increases in the Average FPS by Norton GO
Edition 1 Page 8 of 21
22 June 2021
Norton 360 for Gamers Game Optimization Testing PassMark Software
The details of the average FPS measured from each type of background task test are shown in the chars below:
Background Task 1 - Downloading games on Steam
98.37
Ghost Recon Wildlands 79.54
88.89
140.97
Sleeping Dogs 125.63
132.10
125.00
Forza Horizon 4 118.33
122.33
100.67
Shadow of the Tomb Raider 82.33
93.67
103.67
Far Cry 5 76.33
99.67
0 20 40 60 80 100 120 140 160 Average FPS
Baseline (No background task) Norton GO not installed Norton GO installed
Edition 1 Page 9 of 21
22 June 2021
Norton 360 for Gamers Game Optimization Testing PassMark Software
Background Task 2 - Chrome loading Web pages
98.37
Ghost Recon Wildlands 91.01
90.82
140.97
Sleeping Dogs 131.30
132.43
125.00
Forza Horizon 4 116.00
117.00
100.67
Shadow of the Tomb Raider 90.00
92.00
103.67
Far Cry 5 95.33
95.67
0 20 40 60 80 100 120 140 160 Average FPS
Baseline (No background task) Norton GO not installed Norton GO installed
Edition 1 Page 10 of 21
22 June 2021
Norton 360 for Gamers Game Optimization Testing PassMark Software
Background Task 3 - Video encoding with Handbrake
98.37
Ghost Recon Wildlands 75.61
94.22
140.97
Sleeping Dogs 99.90
130.30
125.00
Forza Horizon 4 106.67
121.33
100.67
Shadow of the Tomb Raider 71.00
96.33
103.67
Far Cry 5 68.00
95.67
0 20 40 60 80 100 120 140 160 Average FPS
Baseline (No background task) Norton GO not installed Norton GO installed
Edition 1 Page 11 of 21
22 June 2021
Norton 360 for Gamers Game Optimization Testing PassMark Software
Background Task 4 - Game recording with Game DVR
98.37 Ghost Recon Wildlands 94.47 95.00
140.97 Sleeping Dogs 135.27 134.53
125.00 Forza Horizon 4 120.00 120.00
100.67 Shadow of the Tomb Raider 96.33 97.00
103.67 Far Cry 5 100.67 101.33
0 20 40 60 80 100 120 140 160 Average FPS
Baseline (No background task) Norton GO not installed Norton GO installed
Edition 1 Page 12 of 21
22 June 2021
Norton 360 for Gamers Game Optimization Testing PassMark Software
Background Task 5 - Chrome Cleanup computer
98.37 Ghost Recon Wildlands 91.89 93.66
140.97 Sleeping Dogs 127.90 131.87
125.00 Forza Horizon 4 115.33 118.00
100.67 Shadow of the Tomb Raider 96.00 93.00
103.67 Far Cry 5 90.67 96.00
0 20 40 60 80 100 120 140 160 Average FPS
Baseline (No background task) Norton GO not installed Norton GO installed
Edition 1 Page 13 of 21
22 June 2021
Norton 360 for Gamers Game Optimization Testing PassMark Software
Background Task 6 - CPU stress with GBCPUStressTest
98.37 Ghost Recon Wildlands 63.74 94.29
140.97 Sleeping Dogs 123.67 136.97
125.00 Forza Horizon 4 114.00 121.33
100.67 Shadow of the Tomb Raider 80.33 95.67
103.67 Far Cry 5 68.00 97.67
0 20 40 60 80 100 120 140 160 Average FPS
Baseline (No background task) Norton GO not installed Norton GO installed
Edition 1 Page 14 of 21
22 June 2021
Norton 360 for Gamers Game Optimization Testing PassMark Software
Phase 2 We performed the second phase of the test with different GO products installed and compared the results to each other. We manually produced a fixed amount of CPU load on the system while launching the game’s benchmark mode. A third-party CPU stress tool GBCPUStressTest was used to ensure the CPU load was constant for all competitors tests.
In the following charts, we have highlighted the results we obtained for Norton 360 for Gamers in orange color and the average has been highlighted in blue. Products with higher average FPS values are considered better performing products.
Game 1 - Far Cry 5
Bitdefender Antivirus Plus 66.00
Kaspersky Total Security 68.67
ESET NOD32 Antivirus 69.67
Webroot SecureAnywhere AntiVirus for PC Gamers 71.33
Average 76.22
McAfee Gamer Security 84.00
Norton 360 for Gamers 97.67
0 20 40 60 80 100 120 Average FPS
Game 2 – Shadow of the Tomb Raider
Bitdefender Antivirus Plus 81.00
McAfee Gamer Security 82.67
Webroot SecureAnywhere AntiVirus for PC Gamers 84.33
Kaspersky Total Security 84.67
ESET NOD32 Antivirus 84.67
Average 85.83
Norton 360 for Gamers 97.67
0 20 40 60 80 100 120 Average FPS
Edition 1 Page 15 of 21
22 June 2021
Norton 360 for Gamers Game Optimization Testing PassMark Software
Game 3 – Forza Horizon 4
McAfee Gamer Security 112.67
Kaspersky Total Security 114.67
ESET NOD32 Antivirus 114.67
Bitdefender Antivirus Plus 115.00
Webroot SecureAnywhere AntiVirus for PC Gamers 115.67
Average 115.83
Norton 360 for Gamers 122.33
0 20 40 60 80 100 120 140 Average FPS
Game 4 – Sleeping Dogs
Kaspersky Total Security 117.83
Bitdefender Antivirus Plus 117.90
Webroot SecureAnywhere AntiVirus for PC Gamers 119.33
ESET NOD32 Antivirus 119.73
Average 122.44
McAfee Gamer Security 123.87
Norton 360 for Gamers 135.97
0 20 40 60 80 100 120 140 160 Average FPS
Game 5 – Tom Clancy's Ghost Recon Wildlands
Bitdefender Antivirus Plus 58.15
Kaspersky Total Security 59.18
McAfee Gamer Security 60.58
ESET NOD32 Antivirus 61.33
Webroot SecureAnywhere AntiVirus for PC Gamers 61.59
Average 66.02
Norton 360 for Gamers 95.30
0 20 40 60 80 100 120 Average FPS
Edition 1 Page 16 of 21
22 June 2021
Norton 360 for Gamers Game Optimization Testing PassMark Software
Conclusion
We found that Norton GO could help improve frame rate performance in games in some scenarios.
Scenarios where there was a benefit
GO is of particular benefit when there are high CPU usage activities running in the background during the game play and the CPU is creating a bottleneck. It is important to note that this benefit was limited to restoring lost performance due to background CPU load and there was no scenario where game performance exceeded the performance obtained with a clean install of Windows.
Scenarios where there was no benefit
GO’s method of functioning only acts on the CPU’s process scheduling (known as process affinity and process scheduling priorities). The theory of its functioning is that the full resources of the system can be given to the game, at the expense of any background tasks. So, in the following cases no benefit or limited benefit can be expected.
• CPUs with only one or two cores. You can’t reschedule background tasks to another core if there is no other core available. • CPUs with large numbers of cores. e.g. 8+ cores. In this case many CPU cores are likely already idle. So rescheduling achieves nothing. • Gaming PCs with a relatively slow video card in comparison to the CPU. In this case the video card is the bottle neck. • Gamers running very high resolution monitors with games at maximum quality settings. In this case the video card is more often the bottle neck. • PCs with no significant background activity. e.g. a clean install of Windows. • PCs with significant background activity, but that background activity is mostly placing load on the disk, internet or video card. If there is no CPU load then the CPU load can’t be optimized. • Games that are not identified as games by GO. For example rare games, or games that don’t run full screen. In this case GO may not activate. However we did not see this in our testing. All games tested were identified as games.
Edition 1 Page 17 of 21
22 June 2021
Norton 360 for Gamers Game Optimization Testing PassMark Software
Disclaimer and Disclosure
This report only covers the versions that are listed in the “Products List” section of this report.
Disclaimer of Liability
While every effort has been made to ensure that the information presented in this report is accurate, PassMark Software Pty Ltd assumes no responsibility for errors, omissions, or out-of-date information and shall not be liable in any manner whatsoever for direct, indirect, incidental, consequential, or punitive damages resulting from the availability of, use of, access of, or inability to use this information.
Disclosure
NortonLifeLock Inc. funded the production of this report. The products list, performance metrics, and the games list included in the report were selected by NortonLifeLock.
Trademarks
All trademarks are the property of their respective owners.
Contact Details
PassMark Software Pty Ltd Level 5, 63 Foveaux St Surry Hills, 2010 Sydney, Australia Phone + 61 (2) 9690 0444 Fax + 61 (2) 9690 0445 Web www.passmark.com
Edition 1 Page 18 of 21
22 June 2021
Norton 360 for Gamers Game Optimization Testing PassMark Software
Appendix 1 – Test Environment
Testing Machine
The test performed on a desktop computer with the following hardware specifications:
Motherboard: ASRock Fatal1ty Z170 Gaming K6 CPU: Intel Core i7-6700K @ 4.00GHz RAM: G.Skill DDR4-2133MHz 2x8GB Video Card: ASUS Strix GeForce GTX 960 4GB Video Card: Gigabyte GeForce RTX 2060 OC 6GB Main Drive (OS): Samsung 950 Pro M.2 NVMe 256GB 2nd Drive (Steam Library): Samsung 850 Pro SATA III 512GB Monitor1: Dell UltraSharp U2412M (1920 x 1200 at 59 Hz) Monitor2: Dell UltraSharp U2412M (1920 x 1200 at 59 Hz) Internet: 1Gbps Internet Connection OS: Windows 10 Pro Version 2004
We used dual monitors with extended display mode, one monitor for playing the games in full-screen mode and the other one for executing and monitoring the background tasks.
NVIDIA GeForce GTX 960 graphic card was used for the first stage of Phase 1 test and NVIDIA GeForce RTX 2060 was used for the rest of the tests. NVIDIA graphics card Game Ready Driver (GRD) version 466.27 WHQL was installed.
Testing Games
• Tom Clancy's Ghost Recon Wildlands • Far Cry 5 • Shadow of the Tomb Raider • Sleeping Dogs • Forza Horizon 4
These games were selected as they are relatively modern and also have a built-in benchmark mode that returns consistent results. The games were downloaded and installed using the Steam platform.
Edition 1 Page 19 of 21
22 June 2021
Norton 360 for Gamers Game Optimization Testing PassMark Software
Appendix 2 –Methodology Description
Windows 10 Image Creation
A bootable version of Macrium Reflect (Backup software) was used to create a “clean” baseline image prior to testing. The aim was to create a baseline image with the smallest possible footprint and reduce the possibility of variation caused by external operating system factors.
The baseline image was restored prior to testing of each different product. This process ensures that we install and test all products on the same, “clean” machine.
The steps taken to create the base Windows 10 image were as follows:
1. Installation and activation of Windows 10 Pro Edition. 2. Update Windows. 3. Install and configure third-party applications and games used for testing. 4. Disable Microsoft Defender. 5. Disable Windows Automatic Updates. 6. Create a baseline image of the boot drive using Macrium Reflect.
Game Benchmark Test
The average FPS was measured by running the games’ built-in benchmark test modes. The game’s video and graphic settings were configured as default (game application recommended settings) after entering each game for the first time and the settings were kept the same for all tests of all products.
Each test was conducted three times and the final result was calculated as an average of the three samples.
Test Phase 1
The aim of the phase 1 benchmark is to test the effect of Norton GO, and this test consists of two parts. The first part of the phase 1 test was conducted with no background tasks launched while playing games’ benchmark modes. The second part of the phase 1 test was run with one or more third-party applications running in the background with the game play.
Background Task 1 - Downloading games on Steam
This test was performed by enabling the “Allow downloads during gameplay” option from the Steam settings and starting a game download and install before launching the game benchmark mode.
Background Task 2 - Chrome loading Web pages
This benchmark was conducted by playing two YouTube videos and loading ten other websites on Google Chrome during the game play. The URLs are as follows:
URL https://youtu.be/f8w7hATXQCg https://youtu.be/ctObBrR0oN4 https://www.tradingview.com/chart/
Edition 1 Page 20 of 21
22 June 2021
Norton 360 for Gamers Game Optimization Testing PassMark Software
https://au.finance.yahoo.com/ https://www.google.com/finance/ https://coinmarketcap.com/ https://www.binance.com/en/markets https://www.news.com.au/ http://www.bom.gov.au/ https://news.google.com/topstories?hl=en-AU&gl=AU&ceid=AU:en https://www.reddit.com/ https://www.passmark.com/
Background Task 3 - Video encoding with Handbrake
Command line version of HandBrake (Version 1.3.3) tool was used to encode an MP4 format sample video file to MKV file format in the background when playing the game benchmarks.
Background Task 4 - Game recording with Game DVR
Before starting the game benchmark mode, the game recording was launched using the Windows 10 built-in tool Game DVR.
Background Task 5 - Chrome Cleanup computer
This test was performed using the Google Chrome’s Cleanup tool. Clean up computer process was launched from the Chrome Settings before starting the game benchmark.
Background Task 6 - CPU stress with GBCPUStressTest
A third-party CPU stress test tool GBCPUStressTest was used for this benchmark. The tool was configured to produce 50% of CPU load to the system.
Test Phase 2
The aim of the phase 2 test is to compare the Norton 360 for Gamers with its competitors’ products. GBCPUStressTest tool was used to produce the same amount of CPU load to all test runs of all products.
The products’ “Gaming Mode” was turned on manually if it was disabled by default.
Edition 1 Page 21 of 21
22 June 2021