A SANS Whitepaper

Why Your Vulnerability Management Strategy Is Not Working—and What to Do About It

Written by Jake Williams Sponsored by:

April 2019 LookingGlass Cyber Solutions

Introduction

Vulnerability management is a critical component of a holistic information security program, but it has become so commonplace that most organizations barely pay attention to it anymore. Yet unpatched vulnerabilities continue to plague organizations. Every year, the headlines are full of breach examples that resulted from unpatched systems.

Much of the problem involves the maturity of vulnerability management programs. It’s 2019, and yet organizations are still trying to apply 2005’s state-of-the-art technology to their cybersecurity programs. The state of attacks is changing, and vulnerability management must change with it. While many organizations understand that annual scans aren’t enough, they still struggle with prioritizing resources to address vulnerabilities. If there’s one takeaway from this paper, it’s that “scanning and patching” isn’t the same as “robust vulnerability management.” The state of attacks is changing, and vulnerability management In this paper, we look at why vulnerability management solutions have not met expectations and how IT and security teams can better implement those must change with it. solutions to maximize value. We then discuss how to deal with the resourcing constraints that all vulnerability management programs encounter. Lastly, we discuss how cyber threat intelligence enables teams to make data-driven decisions about prioritizing limited resources.

©2019 SANS™ Institute The State of Vulnerability Management

A few short years ago, just running a vulnerability scanner that touched every machine and produced a thousand-page report was considered state of the art. While the vulnerability scan report enabled IT and security teams to identify (and hopefully correct) missing patches and insecure configurations, the same teams quickly discovered they were drowning in data. Often, the reports were simply too large to be actionable. Teams didn’t know where to start and frequently found themselves dedicating their limited resources to less important vulnerabilities—instead of the most critical vulnerabilities.

In 2017, WannaCry1 and NotPetya2 wreaked havoc on some of the biggest organizations in the world. Both attacks were fueled by the leaked exploit EternalBlue3—a vulnerability for which patches had been available for months. And yet both events made front-page news, even outside the cyber industry. That NotPetya was still able to exploit targets using EternalBlue more than a month after WannaCry highlights the challenges inherent in vulnerability management. Patching, even for critical vulnerabilities known to be exploited in the wild, remains a significant issue in many enterprises.

Then came the Equifax breach.4 Equifax suffered a highly publicized breach when attackers compromised a Struts vulnerability on a public-facing web server—a vulnerability that had been identified years earlier. How did Equifax miss patching this important vulnerability? In its own reports to the government, Equifax made it clear there were many independent issues with their vulnerability management program. Nevertheless, a vulnerability known to be exploited in the wild went unpatched on a critical server. Nobody accepted the risk of not patching the vulnerability—the state of the vulnerability management program saw to that. Obviously, running a vulnerability management program is hard, even for an organization with the resources of Equifax.

Some teams try to act on data by sorting vulnerabilities identified by the scanner ratings of high, medium, low and informational. But oftentimes this method still leaves them wanting more. A medium vulnerability on a system that processes protected health information (PHI) is probably more important than a critical vulnerability on a system responsible for printing shipping labels. Most vulnerability scanners, and in some cases even the people overseeing the scans, lack context about the business roles and particulars of the systems they are scanning, so effective prioritization remains difficult.

Today, it’s clear that vulnerability scan reports alone aren’t enough. Security teams need more context to act on this data appropriately. Key performance indicators (KPIs) such as “the percentage of critical vulnerabilities patched within 60 days,” while relatively easy to quantify, say little about the security of the organization. Senior management

1 WannaCry ransomware attack, https://en.wikipedia.org/wiki/WannaCry_ransomware_attack 2 “The Untold Story of NotPetya, the Most Devastating Cyberattack in History,” www.wired.com/story/notpetya-cyberattack-ukraine-russia-code-crashed-the-world 3 EternalBlue, https://en.wikipedia.org/wiki/EternalBlue 4 “Equifax Officially Has No Excuse,” www.wired.com/story/equifax-breach-no-excuse

Why Your Vulnerability Management Strategy Is Not Working—and What to Do About It 2 is also waking up to these inappropriate metrics. In breach after breach, vulnerability management teams have reported compliance with all program goals. Unfortunately, vulnerability management can’t be both working and not working at the same time. This is a classic case of “what gets measured gets managed,” but if you’re measuring the wrong things, you can be compliant without actually addressing the underlying problem the program was intended to solve. Organizations need to find KPIs that actually align to organizational risk. One example of such a KPI might be the mean time to patch a vulnerability being actively exploited by an adversary known to target the organization or ones like it.

The scan data presented today leaves organizations ill-equipped to even understand their vulnerability landscape. It is extremely difficult for an organization to just “patch everything”—but should that even be the goal?

Vulnerability Management Program Goals When discussing vulnerability management programs, it helps to consider the goal of any such program. All too often, organizations say that the goal of their program is to ensure that patches are applied. But that’s not really accurate. The real goal of any vulnerability management program should be to minimize risks to the organization introduced by vulnerabilities. While applying patches is one component of any such program, the patching activity is simply a means to the end of minimizing risk.

It is through this understanding of vulnerability management programs that the rest of the paper is framed. We discuss patching, but we must acknowledge that many organizations simply lack the capability to apply all applicable patches in a timely manner (whatever that means for the organization). Instead, this paper discusses vulnerability management as a holistic program designed to reduce overall risk.

Default Scanner Priorities Aren’t Enough Most vulnerability scanners simply report the priority of vulnerabilities INFORMATIONAL on a scale of ratings including informational, low, medium, high and critical (see Figure 1). LOW Organizations are largely left to prioritize their patching according MEDIUM to these seemingly arbitrary ratings. Unfortunately, these ratings are not appropriate for all (or even most) HIGH organizations. The default ratings that vulnerability scanners assign assume no compensating controls CRITICAL and are not informed by context. Figure 1. Vageries of Risks at Each Vulnerability State

Why Your Vulnerability Management Strategy Is Not Working—and What to Do About It 3 Remember, the goal of a vulnerability management program is not to patch vulnerabilities (though that is an important component of any program). Because the vulnerability scanner does not contextualize the criticality of the asset or the data potentially at risk, the threat model of the organization or compensating controls, these ratings are often woefully inaccurate at predicting actual risk.

Chained vulnerabilities are another potential issue. The default ratings The default ratings that vulnerability that vulnerability scanners assign consider each vulnerability in scanners assign consider each isolation. But attackers don’t target assets this way. They will use any vulnerability in isolation. But combination of vulnerabilities present to accomplish their attack. attackers don’t target assets this way. Consider an asset that has multiple low vulnerabilities: one that allows directory traversal on a trivial file transfer protocol (TFTP) server and another that allows an attacker to list filesystem contents remotely. Each of these considered independently is probably a low vulnerability. In order to request a file through TFTP, the attacker must know a valid filename. Listing filesystem contents by itself will not lead to successful exploitation either. However, if the attacker can use the filesystem vulnerability to enumerate filenames and then use the TFTP vulnerability to obtain their contents, this combination of vulnerabilities could result in total system compromise. But what if the system in the previous example also has an unpatched Two low vulnerabilities combined critical vulnerability? In most vulnerability management programs, the may represent a greater risk to the critical vulnerability would be assigned a higher priority for patching. organization than a single critical What if the critical vulnerability requires authenticated access to the vulnerability. system to be exploited? Or what if the configuration of the system prevents reliable exploitation of the critical vulnerability? By examining the complete context, the combination of two low vulnerabilities represents a greater risk to the organization than a single critical vulnerability. This type of scenario illustrates why it is so important to factor in more than just the default ratings when prioritizing patches.

Building an Effective Risk-Based Vulnerability Prioritizing the application of patches Management Program Mapping security controls to assets With more vulnerabilities being discovered than ever before, every organization needs to prioritize building an effective vulnerability management strategy. An effective strategy includes the following Threat modeling to understand attack chains components, as shown in Figure 2. Applying gap analysis to prioritize new security controls The remainder of this paper describes how organizations can Figure 2. Components of an Effective incorporate these four components to implement an effective Vulnerability Management Strategy vulnerability management program.

Why Your Vulnerability Management Strategy Is Not Working—and What to Do About It 4 Prioritizing the Application of Patches Today’s organizations are using more devices, operating systems and software than they did in the past. At the same time, management is clamoring for more automation, which is reducing the acquisition of new head count in many organizations, and even if the budget is available, in many markets security staff are in extremely short supply. As any systems administrator can affirm, the attitude of “just apply the patch and everything will be fine” is laughable given the complexity of today’s interconnected software systems and their myriad configurations.

This combination of factors leaves many organizations unable to To prioritize patching effectively, the patch all of the vulnerabilities discovered on any given scan. In organization must both understand fact, in resource-constrained environments, patching 100 percent of vulnerabilities shouldn’t even be a goal. Instead, organizations its own network and possess a level should work to prioritize the application of patches to the most critical of expert knowledge in vulnerabilities vulnerabilities. But how should patches be prioritized? Organizations and exploit development. must establish criteria which ensure that patch prioritization requires fewer resources than simply patching everything.

In the “Patch Prioritization Criteria” sidebar, we offer some questions to ask when prioritizing patches. Note that some information (such as whether an exploit is available on the Dark Web) may not be immediately actionable by some organizations (those lacking a cyber threat Patch Prioritization Criteria intelligence team, for instance). 1 How critical is the data processed on the vulnerable asset? The first four criteria rely on having a solid inventory of the software, assets, data and compensating controls 2 Can the vulnerable asset be used to pivot to sensitive data? present on the network. The last five criteria are 3 Will a compensating control prevent exploitation or mitigate organization agnostic and deal with the vulnerability an attack? itself. This again underscores the difficulty of prioritizing patches: To do so effectively, the organization must 4 Is the vulnerability present in a default configuration? If so, is that configuration present in our network? both understand its own network and possess a level of expert knowledge in vulnerabilities and exploit 5 Is the vulnerability being actively exploited in the wild? development. Implementing the Center for Internet Is an exploit for the vulnerability publicly available? Controls (CIS) Critical Controls #1 and #2 (hardware and 6 software inventory, respectively)5 will help practitioners 7 Is an exploit for the vulnerability available for private sale understand the impact of particular vulnerabilities on (for instance, on a Dark Web forum)? their network assets. 8 How difficult is the vulnerability to exploit? This understanding requires expertise in multiple different disciplines and is perhaps why so few 9 Are other organizations in our vertical being exploited with this vulnerability? organizations undertake the prioritization process with the required rigor. Some organizations rely on the ratings the vulnerability scanners provide, but as we discuss later in the paper, those ratings fail to provide the proper level of granularity to make good decisions about prioritization.

5 www.cisecurity.org/controls/

Why Your Vulnerability Management Strategy Is Not Working—and What to Do About It 5 Threat Modeling Threat modeling can be an extremely effective tool for building a comprehensive vulnerability management program. For example, target-centric threat modeling (shown in Figure 3) works by identifying the likely targets of an attacker, then listing the vectors by which an attacker would gain access to those targets. Eventually, compensating controls will be added in, but that step is performed at a later stage.

When considering targets, don’t list things like “domain admin” because in most intrusions, that is only a means to an end. Also, avoid the desire to add targets such as “database server.” Although attackers may target the database server, they do so not because it runs a database but because that database houses critical data. In this stage of the process, enumerate the actual end goals of an intrusion. For instance, targets might be merger and acquisition data, trade secrets or payroll information. 1. Identify likely 2. Map targeted data to 3. Create intrusion maps 4. Map controls to the After targets are enumerated, attacker targets. systems and software. detailing how attackers threat model. the hard work begins. With the will gain access. assistance of threat hunting Figure 3. Steps for Target-Centric teams, the organization should map how an attacker would likely gain access to Threat Modeling critical assets. In this step, map data to the specific systems where it resides (and the software on those systems). Take care to note systems that may not store critical data but show up as a stepping stone in many intrusion maps between initial access and asset compromise.

Lastly, map controls to the systems described in the threat model. For instance, a web server may use a web application firewall that mitigates SQL injection attacks. Be sure to consider both network and endpoint controls in this phase.

The value of threat modeling to prioritize vulnerabilities should be obvious. Armed with an up-to-date threat model, the organization can understand which systems identified in the scan report represent the largest risk and plan patching efforts appropriately. In many cases, it makes more sense to patch a low-rated vulnerability system that houses critical data than to patch a high-rated vulnerability on a system that doesn’t appear anywhere in the threat model. For similarly rated vulnerabilities, it makes more sense to prioritize patching a system that contains no critical data but appears on many intrusion maps than to patch the same system that contains critical data. While this decision may run contrary to conventional wisdom, the threat model enables the organization to make the appropriate patching prioritization decision.

Why Your Vulnerability Management Strategy Is Not Working—and What to Do About It 6 Mapping Controls to Assets Most organizations with a vulnerability management program also have a risk management program. Typically, the risk management program handles the inventory of compensating controls. All too often, the risk management program is run by a governance, risk and compliance (GRC) team and is not considered to be part of the vulnerability management process. This classification is a mistake.

Mapping compensating controls to the specific assets they protect allows By mapping compensating controls organizations to quickly understand their level of exposure when a new to the specific assets they protect, vulnerability is discovered. For instance, suppose that a new, exploitable organizations can quickly understand remote code execution (RCE) vulnerability is discovered in Microsoft their level of exposure when a new SQL (MS SQL). The vulnerability scanner discovers that the organization vulnerability is discovered. has 15 vulnerable servers. Each will require an outage window to apply the patch. Of course, the vulnerability scanner’s IP address has been whitelisted in host and network firewall configurations, so what the vulnerability scanner sees as “remotely exploitable” may not, in fact, be exploitable by an attacker.

Seven of the servers contain data the organization has deemed critical. But as noted earlier, the threat model doesn’t tell the whole story. Of the 15 vulnerable servers, eight only allow connections from one other specific middleware server (including six of the seven that contain critical data), and four are protected by a network firewall that only allows connections from a development subnet. This leaves three servers the organization should prioritize for patching first. Without mapping compensating controls to the assets they Figure 4. Mapping Compensating protect, poor patching prioritization decisions would have been made, potentially Controls to the Specific Assets leaving the organization exposed. See Figure 4. They Protect

Prioritizing patching is much more difficult (if not impossible) without considering compensating controls. Most organizations fail to accurately account for compensating controls in patching decisions, but that’s because they find the process too onerous. Almost all see the value in performing this activity.

Why Your Vulnerability Management Strategy Is Not Working—and What to Do About It 7 Gap Analysis As discussed previously, patching isn’t the only component of a mature vulnerability management program. With the appropriate compensating controls in place, vulnerable systems can be operated safely without applying a patch. This isn’t to say that leaving vulnerabilities unpatched is a recommended practice. The concept is most easily analogized to driving in the snow—if you drive at a reduced speed using tire chains (compensating controls), you can drive relatively safely. However, nobody would recommend making a trip in the snow (even with tire chains) if you can postpone the trip until the severe weather has passed.

Mature organizations incorporating a risk-based perspective to vulnerability management in their cybersecurity programs will be able to identify consistent limitations in compensating controls that force prioritization of patches. A careful analysis of the organization’s threat model will also identify points where compensating controls are lacking and a single unpatched vulnerability could lead to the compromise of critical data.

Case Studies

This section addresses case studies combining the factors described throughout the paper to achieve optimal outcomes in vulnerability management.

Case Study No. 1: Newly Discovered Vulnerability in a Java Framework

The organization has learned of a new critical vulnerability in a Java framework it uses. In response, the organization uses its chosen vulnerability scanner to run authenticated scans against all its servers. The vulnerability scanner shows that a vulnerable version of the framework is installed on five servers in the organization. The information security group recommends that IT patch all affected servers immediately. Unfortunately, when IT applies the framework patch on the dev instances of the servers, the applications using the framework begin exhibiting errors.

IT now asks the development teams to update their applications to be compatible with the new framework version, but unfortunately, a single development team is responsible for four of the five applications. Which application should the team prioritize first? By examining the threat model, the organization learns that only two of the four applications managed by the Just saying “patch everything” isn’t a viable team process critical data. The vulnerability can be triggered vulnerability management strategy. only by processing a specially crafted file.

While both applications process data from outside the organization, one of the applications has input files scanned by endpoint scanning software before being ingested by the application. After learning of this compensating control, it is clear which application the organization should update first.

Why Your Vulnerability Management Strategy Is Not Working—and What to Do About It 8 This case study highlights how just saying “patch everything” isn’t a viable vulnerability management strategy. External factors such as required code changes may limit the options available to the organization. Situations like this also highlight the desire for threat modeling and the mapping of controls to assets. When threatened with an unpatched critical vulnerability, any organization will attempt this exercise on the fly, only without the appropriate data. What should be a regular part of the vulnerability management program essentially becomes a fire drill.

Case Study No. 2: Information Disclosure of Remote Code Execution

In this week’s scan results, the organization notes that it has an unpatched RCE vulnerability rated critical on 15 internal middleware servers and an information disclosure vulnerability rated low on two internal web servers. Unfortunately, the patches for both vulnerabilities require a system restart to be applied.

Conventional wisdom indicates that the organization should patch the RCE vulnerabilities first. After examining the threat model, the organization discovers that all affected assets process the same sensitivity data. Compensating controls are not a factor in this particular case either. All servers are available internally, and attackers can exploit the vulnerabilities without authentication.

The organization turns to the threat intelligence team to learn more about the vulnerabilities. Neither vulnerability has a publicly available exploit. However, the organization learns a group that has actively targeted them in the past is actively exploiting the information disclosure vulnerability in the wild. The organization now knows what to prioritize first: the information disclosure above the remote code execution vulnerability.

Case Study No. 3: SMB Zero Day

Three years ago, the organization scrambled to deal with the announced server message block (SMB) vulnerability Badlock.6 Simultaneously, the organization was fighting an issue in two of its factories where they were having difficulties patching the computers that control their industrial equipment. MS08-067 (the vulnerability exploited by Conficker7) was making yet another appearance in these environments.

The organization obviously has concerns about the exploitation of MS08-067 affecting the legacy industrial applications availability. Also of extreme concern is the announcement of a new remotely exploitable SMB vulnerability affecting all versions of SMB, including Samba servers. As the organization considers its exposure to the newly announced vulnerability (for which details are not released), the security team determines that limiting SMB communications is the only way to mitigate the organization’s exposure. The team recommends blocking SMB communications between sites and even within sites by limiting communications over SMB to only those endpoints that actually require it.

6 Badlock, https://en.wikipedia.org/wiki/Badlock 7 “Conficker Worm Still Spreading Despite Being Nearly Ten Years Old,” www.cyberscoop.com/conficker-trend-micro-2017

Why Your Vulnerability Management Strategy Is Not Working—and What to Do About It 9 This recommendation, unfortunately, does not sit well with the organization’s IT architects. They’ve heard this recommendation before and have fought against it claiming that it will increase administrative overhead and possibly lead to business interruption. In the past, the organization has sided with the IT team and ignored the recommendations of the IT security team in this matter.

Since the last time the security team made the recommendation to limit SMB communications, however, they completed a threat model. They are also mapping controls to vulnerabilities. The team notes that while they don’t know how bad the newly announced Badlock vulnerability will be, they can show that multiple other vulnerabilities they’ve handled since developing the threat model would have been mitigated by limiting SMB communications. The security team enlists additional support from the incident response team, which notes the rash of MS08-067 infections they’ve been addressing. By identifying a consistent limitation in applying effective controls, the IT security team convinces the organization to finally adopt its recommendations. Just as IT predicted, there are a few minor business interruptions, but these mostly stem from a lack of device inventory and application dataflow mappings.

If the story stopped there, it would already be a win for the organization. But it doesn’t. A year later, WannaCry is released into the wild. Although the kill switch domain is quickly activated, this doesn’t help the organization’s industrial assets because they can’t reach the internet. One of the organization’s recent acquisitions has internet-exposed hosts that become infected. But because of the SMB lockdown on the intersite VPNs, none of the organization’s industrial controllers are compromised. These are the same types of machines that were affected by MS08-067 and definitely were not patched for MS17-010 (the vulnerability behind WannaCry).

The organization, like many, struggles to deal with MS17-010. It has some legacy devices that simply cannot be patched. While intersite SMB communications are largely restricted, intrasite SMB communications are mostly allowed. The security team reports that other attackers may try to use MS17-010, perhaps chaining it with other lateral movement techniques. They also scan to identify all remaining unpatched endpoints and recommend locking down any SMB communication to those hosts using private VLANs and access control lists. The organization’s network team reluctantly capitulates and makes the changes.

Two months later, NotPetya is released by Russian hackers targeting Ukrainian businesses. Unfortunately, one of the organization’s subsidiaries is in Ukraine and is immediately affected. However, even within the site, NotPetya is unable to move laterally because of the private VLANs that were implemented. The malware is also unable to spread between sites because of the VPN restrictions adopted earlier.

Because of the maturity of their vulnerability management program, the organization only sees NotPetya as a blip on the radar—a stark contrast to the horrific financial losses its peers experienced. Note that the organization didn’t fare better because it had a 100 percent patch rate for the vulnerability; it weathered the NotPetya attack because it had mapped controls to assets and could effect change within the organization.

Why Your Vulnerability Management Strategy Is Not Working—and What to Do About It 10 Conclusions and Recommendations

In this paper, we discussed how running a vulnerability scanner and telling IT to “patch everything” isn’t a successful strategy for vulnerability management. Prioritization of patches is clearly needed to maximize the efficacy of limited IT resources. But prioritizing patching is harder than most organizations realize.

Organizations need to implement a risk-based approach in How to Introduce Automation and Integration their programs, so they are doing more than just determining Criticality of the asset the criticality rating of a vulnerability (see Figure 5). They need to consider the criticality of the asset, whether exploits for the Any publicly available exploits for the vulnerability vulnerability are publicly available, how vulnerabilities may be chained on a single host to facilitate exploitation and how their How vulnerabilities may be chained on a compensating controls map to vulnerable assets. single host to facilitate exploitation

Unfortunately, this approach also requires the expenditure How compensating controls map to of resources, and many organizations have stagnated in vulnerable assets maturing their VM programs. We advise organizations to find Figure 5. Considerations for technology—or a partner with this technology—that assists them in maturing their Rating a Vulnerability programs and augments capabilities they may lack (such as cyber threat intelligence). While a mature, risk-based vulnerability management program takes resources to build, it pays for itself many times over.

Why Your Vulnerability Management Strategy Is Not Working—and What to Do About It 11 About the Author

Jake Williams is a SANS analyst, senior SANS instructor, course author and designer of several NetWars challenges for use in SANS’ popular, “gamified” information security training suite. Jake spent more than a decade in information security roles at several government agencies, developing specialties in offensive forensics, malware development and digital counterespionage. Jake is the founder of Rendition InfoSec, which provides penetration testing, digital forensics and incident response, expertise in cloud data exfiltration, and the tools and guidance to secure client data against sophisticated, persistent attacks on-premises and in the cloud.

Sponsor

SANS would like to thank this paper’s sponsor:

Why Your Vulnerability Management Strategy Is Not Working—and What to Do About It 12