INTELLIGENCE-LED TESTING www.SELabs.uk [email protected] @SELabsUK www.facebook.com/selabsuk blog.selabs.uk HOME ANTI- MALWARE PROTECTION APR - JUN 2017 SE Labs SE Labs CONTENTS Introduction 04 Executive Summary 05 1. Total Accuracy Ratings 06 2. Protection Ratings 08 3. Protection Scores 10 4. Protection Details 11 5. Legitimate Software Ratings 12 SE Labs tested a variety of anti-malware (aka ‘anti-virus’; aka ‘endpoint security’) products from a range of well-known vendors in an effort to 6. Conclusions 16 judge which were the most effective. Appendix A: Terms used 17 Appendix B: FAQs 18 Each product was exposed to the same threats, which were a mixture of Appendix C: Product versions 19 targeted attacks using well-established techniques and public email and web- Appendix D: Attack types 19 based threats that were found to be live on the internet at the time of the test. The results indicate how effectively the products were at detecting and/or protecting against those threats in real time. Document version 1. 0. Written 7th July 2017 02 APR - JUN 2017 • Home Anti-Malware Protection Home Anti-Malware Protection • APR - JUN 2017 03 SE Labs INTRODUCTION EXECUTIVE SUMMARY Can anti-malware be 100 per cent effective? There are a lot of threats on the web, and going online without Product names protection is very risky. We need good, consistently effective It is good practice to stay up to date with the latest version of your anti-malware products to reduce our risk of infection. And the chosen endpoint security product. We made best efforts to ensure ones included in this report look great – in fact, some score 100 that each product tested was the very latest version running with per cent. That means they stopped all the threats that we exposed the most recent updates to give the best possible outcome. them to, and didn’t block anything legitimate. For specific build numbers, see Appendix C: Product versions on page 19. SIMON EDWARDS Director But wait a minute! Those in the security industry know full Products tested well that there is no such thing as 100 per cent security. There WEBSITE www.SELabs.uk is always a way past every security measure, and this is as true PRODUCT PROTECTION LEGITIMATE TOTAL TWITTER @SELabsUK in the anti-malware world as with any other measures for ACCURACY ACCURACY ACCURACY EMAIL [email protected] threat protection. RATING RATING RATING FACEBOOK www.facebook.com/selabsuk ESET Smart Security 100% 100% 100% BLOG blog.selabs.uk This test includes some of the very best anti-malware products in Kaspersky Internet Security 100% 100% 100% PHONE 0203 875 5000 POST ONE Croydon, London, CR0 0XT the world, and pits them against prevalent threats, be they ones Norton Security 100% 100% 100% that affect hundreds of thousands of users worldwide, or those Bitdefender Internet Security 100% 99% 99% TESTING TEAM that could be used to target individuals and organisations. It’s a Avast Free Antivirus 89% 100% 96% Thomas Bean tough test, but a fair one. You could argue that any anti-malware Dimitar Dobrev AVG AntiVirus Free Edition 88% 100% 96% product worth its salt would score 100 per cent or thereabouts. Stefan Dumitrascu Trend Micro Internet Security 97% 92% 94% Gia Gorbold Microsoft Security Essentials 78% 100% 93% Magdalena Jurenko Products can score 100 per cent in our tests because we’re not Alexandru Statie choosing thousands of weird and wonderful rare pieces of Products highlighted in green were the most accurate, scoring 85 per cent or more for Total Accuracy. Those in Jon Thompson malware to test. Regular users are extremely unlikely to yellow scored less than 85 but 75 or more. Products shown in red scored less than 75 per cent. For exact percentages, Jake Warren encounter those in the real world. We’re looking at the threats see 1. Total Accuracy Ratings on page 6. Stephen Withey that could affect you. IT SUPPORT Danny King-Smith Our mission is to help improve computer security through testing, Chris Short both publicly and privately. We also want to help customers The endpoints were mainly effective at handling False positives were not an issue for most products choose the best products by publishing some of those test results. • • PUBLICATION general threats from cyber criminals… All endpoint solutions were good at correctly classifying But don’t forget that success today is not a guarantee of success Steve Haines All products were capable of handling public threats legitimate applications and websites. Six out of the eight tomorrow. It’s important to keep monitoring test results. Colin Mackleworth such as those used by criminals to attack Windows PCs products made no mistakes at all, and products that and install ransomware. blocked legitimate sites and programs did so sparingly. SE Labs Ltd is a member of the Anti-Malware Testing If you spot a detail in this report that you don’t understand, or Standards Organization (AMTSO) would like to discuss, please contact us via Twitter or Facebook. • …but targeted attacks posed more of a challenge • Which products were the most effective? Half of the products were very competent at blocking ESET, Symantec, Kaspersky Lab and Bitdefender SE Labs uses current threat intelligence to make our tests as While every effort is made to ensure the accuracy of the more targeted, exploit-based attacks. Products from achieved the best results due to a combination of their information published in this document, no guarantee is realistic as possible. To learn more about how we test, how we Bitdefender, ESET, Kaspersky Lab and Symantec ability to block malicious URLs, handle exploits and expressed or implied and SE Labs Ltd does not accept define ‘threat intelligence’ and how we use it to improve our tests, liability for any loss or damage that may arise from any (Norton) handled the targeted attacks comprehensively. correctly classify legitimate applications and websites. errors or omissions. please visit our website and follow us on Twitter. Simon Edwards, SE Labs, 7th July 2017 04 APR - JUN 2017 • Home Anti-Malware Protection Home Anti-Malware Protection • APR - JUN 2017 05 SE Labs SE Labs 1. TOTAL ACCURACY RATINGS Awards The following products win SE Labs awards: Judging the effectiveness of an endpoint security it from downloading any further code to the target. product is a subtle art, and many factors are at play In another case malware might run on the target for a when assessing how well it performs. To make things short while before its behaviour is detected and its easier we’ve combined all the different results from this code is deleted or moved to a safe ‘quarantine’ area for E La ● Kaspersky Internet Security report into one easy-to-understand graph. future analysis. We take these outcomes into account S bs when attributing points that form final ratings. ● ESET Smart Security The graph below takes into account not only each H ● Norton Security product’s ability to detect and protect against threats, For example, a product that completely blocks a threat E O but also its handling of non-malicious objects such as is rated more highly than one that allows a threat to run R ● Bitdefender Internet Security M web addresses (URLs) and applications. for a while before eventually evicting it. Products that APR-JUN 2017 A E ● Avast Free Antivirus W allow all malware infections, or that block popular A L Not all protections, or detections for that matter, are legitimate applications, are penalised heavily. N A ● AVG AntiVirus Free Edition TI M equal. A product might completely block a URL, which stops the threat before it can even start its intended Categorising how a product handles legitimate objects series of malicious events. Alternatively, the product is complex, and you can find out how we do it in might allow a web-based exploit to execute but prevent 5. Legitimate Software Ratings on page 12. E La S bs H Total Accuracy Ratings E ● Microsoft Security Essentials O 1218 R M ● Trend Micro Internet Security APR-JUN 2017 A E W A L N A TI M 812 TOTAL ACCURACY RATINGS dition Product Total Accuracy Total Award E Rating Accuracy (%) 406 o ee Kaspersky Internet Security 1218 100% AAA r sentials r y s k ESET Smart Security 1216 100% AAA ender Norton Security 1216 100% AAA f irus F t on osoft s t V r Bitdefender Internet Security 1208 99% AAA G a ernet Security ernet Security ernet Security end Mic ee Antivirus t t t aspers v V r r Avast Free Antivirus 1173 96% AAA A K Smart Security Nor F Anti T Mic Security E In ESET Security Bitde In 0 In A AVG AntiVirus Free Edition 1171 96% AAA Trend Micro Internet Security 1140 94% AA Total Accuracy Ratings combine protection and false positives. Microsoft Security Essentials 1128 93% AA 06 APR - JUN 2017 • Home Anti-Malware Protection Home Anti-Malware Protection • APR - JUN 2017 07 SE Labs SE Labs 2. PROTECTION RATINGS The results below indicate how effectively the products • Neutralised (+1) Rating calculations dealt with threats. Points are earned for detecting the Products that kill all running malicious processes We calculate the protection ratings using the threat and for either blocking or neutralising it. ‘neutralise’ the threat and win one point. following formula: • Detected (+1) • Complete remediation (+1) Protection rating = If the product detects the threat with any degree of If, in addition to neutralising a threat, the product (1x number of Detected) + useful information, we award it one point. removes all significant traces of the attack, it gains (2x number of Blocked) + an additional one point.