Sponsored by Raytheon

Don’t Wait:

The Evolution of Proactive Threat Hunting

Independently conducted by Ponemon Institute LLC Publication Date: June 2016

Connect with us: #DontWaitHunt

Ponemon Institute© Research Report

Don’t Wait: The Evolution of Proactive Threat Hunting Ponemon Institute, June 2016

Part 1. Executive Summary

The purpose of the “Don’t Wait: The Evolution of Proactive Threat Hunting” survey, sponsored by Raytheon, is to examine how organizations are deploying managed security services to strengthen their security posture. The research also looks at the critical success factors, barriers and challenges to having a successful relationship with managed security services providers.

We surveyed 1,784 chief information security officers and other senior IT security leaders in North America, Europe, Middle East and Asia Pacific1 who are familiar with their organizations’ managed security service practices. Managed security services providers (MSSPs) are engaged by organizations to manage and strengthen their IT environment’s security by providing services including security information and event management (SIEM), network security management (NSM), endpoint detection and response (EDR), incident response, forensics and more.

Security tools such as anti-virus, firewalls, intrusion detection and sandbox technologies, are built upon the assumption that attackers adhere to a known set of tools and tactics. Today, while a majority of MSSPs focus on these traditional, reactive tools, some provide more advanced, proactive services. Proactive threat hunting services can effectively find sophisticated and damaging threats, including previously undetected attacks, and stop them before businesses suffer damage.

In this study, 56 percent of respondents use an MSSP and 22 percent say they plan to engage an MSSP in the future. Part 2 of this report provides analysis of the 56 percent who are engaged with a provider. In many cases, it is a serious security incident such as a data breach that motivates companies to engage an MSSP to strengthen their security posture.

A key takeaway is that organizations using MSSPs understand the primary benefits of leveraging external expertise. Eighty percent view MSS as essential, very important or important to their overall IT security Figure 1. Main reasons organizations use MSSPs strategy. Figure 1 shows the primary reason to have an MSSP is to improve security posture (59 percent). This is followed closely by the need to reduce the challenge of recruiting and retaining necessary talent (58 percent) and the lack of in-house security technologies (57 percent).

The following are the seven most salient research findings.

1. MSSPs help companies achieve a stronger security posture. With evolving cyber threats, organizations face the critical challenge of lack of expertise, personnel and resources. MSSPs are seen as filling these gaps to improve their security.

1 The countries represented in these regions are: United States, Canada, United Kingdom, Denmark, France, Germany, Netherlands, Brunei, Kuwait, Saudi Arabia, Oman, Qatar, UAE, India, Australia, Japan, Singapore and South Korea.

Ponemon Institute©: Research Report Page 1

Many organizations worldwide still typically wait until after a breach before the money is allocated to engage an MSSP. Two-thirds of organizations not currently using an MSSP say that the top trigger would be a significant data loss resulting from an IT security incident. A breach would confirm that the organization’s risk of compromise is high, so it becomes a priority.

2. A shift from reactive services to proactive services offered by providers and demanded by organizations is occurring but is still in the early stages. The lack of proactive threat hunting services could be contributing to the daily barrage of media headlines about data breaches in organizations worldwide. It highlights a need for organizations to be doing more to protect their networks from the most insidious threats. Currently, MSSPs offer cybersecurity assessment (39 percent), integration services (31 percent) and digital forensics and incident response (DFIR) engineering and/or assessment (28 percent). Only 16 percent say their MSS offers proactive threat hunting to find advanced threats based on behaviors and anomalies.

3. Interoperability with security intelligence tools such as SIEM is essential or very important. When asked what characteristics of MSSPs are essential or very important, the number one feature is high interoperability with the company’s security intelligence tools, such as SIEM (73 percent). Also critical are speedy deployment (65 percent), round-the-clock threat monitoring and management (63 percent), a tried and tested service offering (62 percent) and scalability of services (61 percent). Not as critical are compliance with data protection requirements (52 percent) and indemnification for service failures (36 percent).

Whether organizations use MSSPs or not, interoperability/integration between MSSP and the customer is top priority. Those currently not using one say it is difficult to find MSSPs that would support or integrate with their systems and requirements. Fifty-three percent list difficulty finding vendors strong in interoperability as the reason they choose not to outsource.

4. MSSPs provide insights about security events and a better understanding of the external threat environment. Sixty-five percent of respondents believe their MSSP leverages insight gained from monitoring a large number of security events from a global customer base and 53 percent say the MSSP helps to better understand the external threat environment through the collection and analysis of information on attackers, methods and motives. More than half (51 percent) say it effectively mitigates the risks after they are identified.

5. MSSPs have identified existing software vulnerabilities that are more than three months old. Fifty-four percent of respondents say their MSSPs identified exploits of existing software vulnerabilities greater than three months old, and 45 percent say exploits of existing software vulnerabilities less than three months old have been discovered. They also revealed Web-borne malware attacks (51 percent). New threats are often going undetected because typical providers are not actively identifying new threats but importing threats identified by industry into their toolsets.

6. Responsibility for relationships with MSSPs is shifting. Fifty-nine percent say responsibility for the MSSP is shifting from IT to the lines of business. Today, however, the IT (43 percent) or IT security professional (15 percent) owns their organizations’ relationships with MSSPs. This represents a trend that MSS services are not considered a commodity but a strategic element and competitive advantage companies can foster. One reason for this shift is that in many organizations the CEO and board of directors now have a responsibility to the shareholders to ensure that companies are protected.

7. A lack of visibility into the outsourcer’s IT security infrastructure is a barrier to successful outsourcing of security services. Fifty-one percent say a lack of visibility into the outsourcer’s IT security infrastructure is the main hindrance to a successful approach to outsourcing. Other barriers are inconsistency with the organization’s culture (49 percent) and turf or silo issues between the organization’s IT security operations team and the outsourcer (46 percent).

Ponemon Institute©: Research Report Page 2

Even though the lion’s share of information security leaders agree that MSS is an important part of their overall cybersecurity strategy, most of those are still focusing on basic commodities and ignoring proactive approaches such as threat hunting. Outsourcing to providers with highly trained experts will become a necessity as organizations mature their IT infrastructure and the MSSP partners improve technologies and approaches. The old adage of building higher walls has proven insufficient in the face of cyber threats that are more complex and sophisticated. The current, most effective security concept is detect, isolate and eradicate through an in-house team, a managed security service provider or a hybrid solution that includes both.

Part 2. Key findings

In this section, we provide a deeper analysis of the findings based on the 56 percent of respondents who say their organizations currently engage an MSSP. The complete audited findings are presented in the Appendix of this report. We have organized this report according to the following topics:

A. The role of MSSPs in helping companies achieve a stronger security posture B. Perceptions about the characteristics and benefits of MSSPs C. Governance of MSSPs D. Regional differences E. Industry differences

A. The role of MSSPs in helping companies achieve a stronger security posture

MSSPs help companies achieve a stronger security posture. Figure 2 shows that a lack of expertise, personnel and resources is a challenge that keeps respondents’ organizations from having an effective cybersecurity posture. MSSPs are seen as filling these gaps to strengthen security posture.

Figure 2. What challenges keep your organization from having a fully effective cybersecurity posture? On a scale of 1 = most challenging to 8 = least challenging

Lack of in-house expertise 1.95

Insufficient personnel 2.04

Insufficient budget (money) 2.05

Lack of collaboration with other functions 3.42

Not considered a priority 4.09

Management does not see cyber attacks as a 5.31 significant risk No understanding how to protect against cyber 6.69 attacks

Lack of clear leadership 7.37

1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00

Ponemon Institute©: Research Report Page 3

MSSPs have identified exploits of software vulnerabilities that are more than three months old. Figure 3 shows 54 percent of respondents say their MSSPs identified exploits of existing software vulnerabilities greater than three months old, and 45 percent say exploits of existing software vulnerabilities less than three months old were identified. They also revealed Web-borne malware attacks (51 percent). Thirty-one percent say they cannot determine the exploits or compromises that have been identified.

Figure 3. Which of the following exploits or compromises has your MSSP identified over the past 12 months? More than one choice permitted

Exploit of existing software vulnerability greater 54% than 3 months old Web-borne malware attacks 51% Exploit of existing software vulnerability less than 45% 3 months old Fraud 37%

Spear phishing 36%

SQL injection 26%

Botnet attacks 26%

DDoS 25%

Cross-site scripting 21%

Clickjacking 15%

Zero day attacks 14%

Ransomware 14% Advanced persistent threats (APT) / targeted 11% attacks Rootkits 10%

Cannot determine 31%

0% 10% 20% 30% 40% 50% 60%

Ponemon Institute©: Research Report Page 4

The shift from reactive services to proactive service is still progressing. Figure 4 shows MSSPs offer cybersecurity assessment (39 percent), integration services (31 percent) and digital forensics and incident response (DFIR) engineering and/or assessment (28 percent). Only 16 percent say their MSSP offers proactive threat hunting to find advanced threats based on behaviors and anomalies.

Figure 4. Does your MSSP include the following services? More than one choice permitted

Cybersecurity assessment 39%

Integration services 31%

Digital forensics and incident response (DFIR) 28% engineering and/or assessment

Proactive hunting to find advanced threats based 16% on behaviors and anomalies

0% 5% 10% 15% 20% 25% 30% 35% 40% 45%

Ponemon Institute©: Research Report Page 5

Managed security services support a stronger security posture. Figure 5 shows both the MSS offered today and what respondents believe are the most important. The biggest gaps are in monitoring or management of customer-deployed SIEM technologies (33 percent vs. 18 percent) and monitoring or management of advanced threat defense technologies (32 percent vs. 22 percent).

The services most often provided are: monitored or managed firewalls or IPSs (63 percent); managed vulnerability scanning of networks, servers, databases or applications (53 percent); NSM (47 percent); monitored or managed intrusion detection systems (41 percent) and DDoS protection (40 percent).

Figure 5. Services currently provided and considered most important More than one choice permitted

Monitored or managed firewalls or intrusion 63% prevention systems (IPSs) 67%

Managed vulnerability scanning of networks, 53% servers, databases or applications 58%

47% Network security management (NSM) 48%

Reporting associated with monitored/managed 41% devices and incident response 48%

Monitored or managed intrusion detection 41% systems (IDSs) 41%

40% Distributed denial of service (DDoS) protection 41%

Managed or monitored security gateways for 39% messaging or Web traffic 42%

Security analysis and reporting of events 38% collected from IT infrastructure logs 41%

Monitored or managed multifunction firewalls, or 32% unified threat management (UTM) technology 39%

25% Endpoint detection and response (EDR) 30%

Monitoring and/or management of advanced 22% threat defense technologies 32%

Monitoring or management of customer- 18% deployed SIEM technologies 33%

3% Other 4%

0% 10% 20% 30% 40% 50% 60% 70% 80%

Services provided today Most important services

Ponemon Institute©: Research Report Page 6

Figure 5 above shows that in many cases the most important services are also provided such as monitored or managed firewalls or IPSs (67 percent); managed vulnerability scanning of networks, servers, databases or applications (58 percent); reporting associated with monitored/managed devices and incident response, network security management (NSM) (48 percent) and managed or monitored security gateways for messaging or Web traffic (42 percent).

B. Perceptions about the characteristics and benefits of MSSPs

Interoperability with security intelligence tools such as SIEM is essential or very important. The No. 1 desired feature of MSSPs is high interoperability with the company’s SIEM (73 respondents), as shown in Figure 6.

Also critical are speedy deployment (65 percent); round-the-clock threat monitoring and management (63 percent); a tried and tested service offering (62 percent) and scalability of services (61 percent).

Figure 6. Essential or very important features of MSSPs Essential and very important responses combined

The MSSP has high interoperability with the company’s security intelligence tools such as 73% SIEM

The MSSP provides services that can be 65% deployed quickly

The MSSP offers 24/7/365 threat monitoring and 63% management

The MSSP has a mature (tried and tested) 62% service offering

The MSSP provides services that are scalable in 61% terms of the company’s regional scope and size

The MSSP demonstrates compliance with data 52% protection requirements

The MSSP provides indemnification for service 36% failures

0% 10% 20% 30% 40% 50% 60% 70% 80%

Ponemon Institute©: Research Report Page 7

MSSPs provide insights about security events and a better understanding of the external threat environment. Figure 7 shows 65 percent believe their MSSP leverages insight gained from monitoring a large number of security events from a global customer base and 53 percent say the MSSP helps to better understand the external threat environment through the collection and analysis of information on attackers, methods and motives. More than half (51 percent) say it effectively mitigates the risks after they are identified.

Only 30 percent say their MSSP uses advanced analytics to identify threats through behavioral or statistical anomalies in security events, IT logs, network traffic or endpoint activity and 28 percent say it provides incident response capabilities that include attack mitigation and forensic investigation services.

Figure 7. Perceptions about the information and activities provided by MSSPs Strongly agree and agree responses combined

Our MSSP leverages insight gained from monitoring a large number of security events 65% from a global customer base

Our MSSP helps us to better understand the external threat environment through the 53% collection and analysis of information on attackers, methods and motives

Our MSSP effectively mitigates the risks after 51% they are identified

Our MSSP uses advanced analytics to identify threats through behavioral or statistical 30% anomalies in security events, IT logs, network traffic or endpoint activity

Our MSSP provides incident response capabilities that include attack mitigation and 28% forensic investigation services

0% 10% 20% 30% 40% 50% 60% 70%

Ponemon Institute©: Research Report Page 8

C. Governance of MSSPs

Responsibility for relationships with MSSPs is shifting. Figure 8 shows 59 percent of respondents say responsibility for the MSSP is shifting from IT to the lines of business. Today, however, the IT (43 percent) or IT security professional (15 percent) owns their organizations’ relationships with MSSPs.

Figure 8. Who owns your organization’s relationship with its MSSPs?

CIO or CTO 43%

Business units (LOB) 16%

CISO or CSO 15%

Procurement 4%

Enterprise risk management 2%

Compliance officer 1%

No one function (shared ownership) 19%

0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%

A lack of visibility into the outsourcer’s IT security infrastructure is a barrier to successful outsourcing of security services. Figure 9 shows 51 percent say this is the main hindrance to a successful approach to outsourcing. Other barriers are inconsistency with the organization’s culture (49 percent) and turf or silo issues between the organization’s IT security operations team and the outsourcer (46 percent).

Figure 9. What do you see as the main barriers to successfully outsourcing IT security services? (Two choices permitted)

Lack of visibility into the outsourcer’s IT security 51% infrastructure Outsourcing is inconsistent with the 49% organization’s culture Turf or silo issues between the organization’s IT 46% security operations team and the outsourcer

Lack of executive-level support 16%

Compliance with privacy and data protection 13% requirements

Lack of leadership 12%

Compliance with internal policies and contractual 8% requirements

Insufficient proof points or measures of success 6%

0% 10% 20% 30% 40% 50% 60%

Ponemon Institute©: Research Report Page 9

D. Regional differences

In this section, we compare some of the research findings for the four regions represented in this global study: North America (NA), Europe (EU), Middle East (ME) and Asia Pacific (AP).

MSSPs get the highest ratings in the Middle East. Figure 10 shows 65 percent of companies in the Middle East view their MSSPs as essential (25 percent) or very important (40 percent) to their IT security strategy. In Europe, fewer companies say their MSSPs are essential or very important (50 percent).

Figure 10. How important is an MSSP to your organization’s overall IT security strategy?

45% 40% 40% 35% 34% 35% 33%

30% 28% 25% 25% 25% 23% 20% 20% 19% 18% 18% 20% 17%

15% 13% 11% 10% 6% 6% 5% 4% 5%

0% Essential Very important Important Not important Irrelevant

NA EU ME AP

Ponemon Institute©: Research Report Page 10

Reasons to engage an MSSP vary among global regions. Figure 11 shows companies in the Middle East (63 percent) and North America (60 percent) mainly use MSSPs to deal with the challenge of recruiting/retaining necessary expertise. In Europe, companies say the MSSP improves their security posture (63 percent). Sixty-one percent of respondents in Asia Pacific say the main reason is the lack of in-house technologies.

Figure 11. Reasons organizations use MSSPs More than one choice permitted

60% Challenge of recruiting/retaining necessary 55% expertise 63% 55%

56% 63% Improves our organization’s security posture 59% 59%

55% 49% Speed to deploy services 50% 55%

53% 55% Lack of in-house technologies 62% 61%

0% 10% 20% 30% 40% 50% 60% 70%

NA EU ME AP

Ponemon Institute©: Research Report Page 11

High interoperability with security intelligence tools is critical in all regions. Figure 12 shows the most important characteristic of an MSSP is interoperability with security intelligence tools such as SIEM in North America (73 percent), Europe (75 percent), Middle East (69 percent) and Asia Pacific (74 percent).

Compared to other features, compliance with data protection requirements is not considered as critical in North America (52 percent), Middle East (46 percent) and Asia Pacific (44 percent). The exception is Europe, where 63 percent say it is essential or very important.

Figure 12. Perceptions about MSSP characteristics Essential or very important responses combined

73% The MSSP has high interoperability with the 75% company’s security intelligence tools such as SIEM 69% 74%

64% The MSSP has a mature (tried and tested) 70% service offering 55% 56%

52% The MSSP demonstrates compliance with data 63% protection requirements 46% 44%

0% 10% 20% 30% 40% 50% 60% 70% 80%

NA EU ME AP

Ponemon Institute©: Research Report Page 12

Insight gained from monitoring a large number of security events from a global customer base is an important benefit of MSSPs. Figure 13 shows companies in the Middle East (61 percent), North America (54 percent), Europe (48 percent) and Asia Pacific (43 percent) believe their MSSPs effectively mitigate the risks after being identified.

Figure 13. Perceptions about the benefits of MSSPs Strongly agree and agree responses combined

66% Our MSSP leverages insight gained from 70% monitoring a large number of security events from a global customer base 63% 60%

57% Our MSSP helps us to better understand the external threat environment through the 51% collection and analysis of information on 55% attackers, methods and motives 49%

54% Our MSSP effectively mitigates the risks after 48% they are identified 61% 43%

0% 10% 20% 30% 40% 50% 60% 70% 80%

NA EU ME AP

E. Industry differences

A special analysis of the industries represented in this research reveals interesting differences.

Expertise is most important for financial services, and an improved security posture is most important for technology and software. Figure 14 shows 67 percent of IT leaders in financial services say they engage an MSSP mainly to help with the challenge of recruiting and retaining necessary expertise. Only 45 percent in the health and pharmaceutical industry say this is a main reason for using an MSSP.

To improve their security posture, 72 percent in the technology and software industry use an MSSP, followed by government (68 percent) and energy and utilities (53 percent).

Figure 14. The two main reasons for using an MSSP

80%

70%

60%

50%

40%

30% Retail 64%

20% 68%Government Retail 54% Financialservices67% Services 53%Services Technology & software 72% software & Technology Industrial64% manuf & Industrial64% manuf & Services 50%Services Energyutilities60%& Healthpharma60%& Financialservices54% Government 47%Government Energyutilities53%& 10% 61% software & Technology Healthpharma45%&

0% Challenge of recruiting/retaining necessary Improves our organization’s security posture expertise

Financial services most often face the barrier of a lack of visibility into the outsourcer’s IT security infrastructure. What prevents organizations from having a successful relationship with their MSSP? Figure 15 shows 64 percent of respondents in financial services face this barrier. Only 49 percent in the technology and software industry say this is a barrier.

Inconsistency with the organization’s culture is not as much of a barrier to achieving a successful relationship in all industries. With the exception of government, at 50 percent, the other industries have less than half of respondents citing culture as an issue.

Figure 15. The top two main barriers to successfully outsourcing IT security services

70%

60%

50%

40%

30%

20% Financialservices64% Retail 49% Services 52%Services Retail 48% Services 48%Services Government 50%Government Government 49%Government Industrial54% manuf & Healthpharma51%& 10% Energyutilities49%& Energyutilities49%& Industrial47% manuf & Financialservices44% Healthpharma43%& Technology & software 49% software & Technology 49% software & Technology

0% Lack of visibility into the outsourcer’s IT security Outsourcing is inconsistent with the infrastructure organization’s culture

Interoperability with SIEM and fast deployment are most important for financial services. Figure 16 shows 86 percent of respondents in financial services say high interoperability with SIEM and other security tools is critical as well as fast deployment (79 percent). Services (58 percent) and government (53 percent) are not as likely to believe fast deployment is essential or very important.

Figure 16. Importance of services offered by your organization’s MSSP Essential and very important responses combined

100%

90%

80%

70%

60%

50%

40%

30% Retail 73% Retail 69% Services 74%Services Financialservices86%

20% Financialservices79% Government 69%Government Services 58%Services Healthpharma71%& Energyutilities70%& Industrial65% manuf & Industrial65% manuf & Technology & software 73% software & Technology Energyutilities61%& Government 53%Government

10% Healthpharma59%& Technology & software 64% software & Technology 0% High interoperability with SIEM and other Provides services that can be deployed quickly security tools

Part 3. Conclusion

The old cybersecurity concept of defend, defend and defend is insufficient in the face of today’s sophisticated threats. The current, most effective security concept is detect, isolate and eradicate through an in-house team or with a managed security service provider. Proactive threat hunting becomes a “must” for organizations, as cyberthreats from cyber criminals, nation-states and other malicious actors become more difficult to detect and deflect.

The shift from reactive services to proactive service is occurring today. Even though the lion’s share of information security leaders agree that MSS is an important part of their overall cybersecurity strategy, most of those are still focusing on commodity prevention-based services and ignoring proactive security. Organizations need to reverse their strategies if they are to remain competitive in today’s global economy.

The concern for many companies is whether an MSSP will work seamlessly with their current systems and whether their data will be safe. The strongest MSSP is solution agnostic, allowing the data to stay with the client, providing actionable reports, building historical perspectives and proactively hunting for problems.

Insufficient personnel and lack of in-house experts are the top challenges to a robust security posture. Organizations without adequate numbers of elite cyber professionals or staff with the skills the company needs can turn to managed security service providers. As the cost of these employees increases, it will become more challenging to retain talent. A model is needed that supports this and grows staff as needed. Savvy organizations choose vendors able to offer proactive threat hunting.

To meet organizational challenges of keeping up with the latest technologies, accessing elite talent and staying ahead of emerging cyberthreats, a strong managed security service provider can serve as a trusted partner focused on meeting each customer’s unique needs. Savvy information security leaders don’t wait until their organization has become a victim.

Part 4. Recommendations

Information security leaders worldwide in all organization sizes and industries can take steps to understand how proactive threat hunting and traditional managed security services can help them achieve organizational objectives and reduce risk.

• Identify the organization’s specific needs and requirements and what can and should be done in-house or by outsourcing • Evaluate information security staff skill sets and team size • Consider the value added by commodity services against proactive threat hunting to find and mitigate the sophisticated and damaging threats in today’s rapidly evolving landscape • Evaluate the current vendor, if any, based on the following criteria: o Ability to integrate with existing technologies o Ability to do proactive hunting of malware and other advanced threats o Willingness to work with any new or existing technologies (product agnostic) • Ensure a new or current vendor can advise on the best solutions that fulfill their needs and requirements

Part 5. Methods

A sampling frame of 51,712 IT security practitioners in North America, Europe, the Middle East and Asia Pacific was selected to participate in this survey. To ensure reliability, the selected participants were familiar with their organizations’ managed security services. Table 2 shows 1,967 total returns. Screening and reliability checks required the removal of 183 surveys. Our final sample consisted of 1,784 surveys or a 3.4 percent response.

Table 2. Sample response Freq Pct% Sampling frame 51,712 100.0% Total returns 1,967 3.8% Rejected or screened surveys 183 0.4% Final sample 1,784 3.4%

Pie Chart 1 reports the industry classification of respondents’ organizations. This chart identifies financial services (17 percent) as the largest segment, followed by industrial/manufacturing (12 percent) and the government sector (12 percent).

Pie Chart 1. Primary industry focus 3% 2% 2% 3% 17% Financial services 4% Industrial/manufacturing Government 5% Services Health & pharmaceutical 6% 12% Retail Energy & utilities 6% Technology & software Consumer products Hospitality Communications 9% 12% Transportation Entertainment & media 9% 10% Other

Pie Chart 2 shows 70 percent of respondents were from organizations with a global headcount of more than 1,000 employees.

Pie Chart 2. Global employee headcount 6% 9%

13% Less than 500

500 to 1,000 21% 1,001 to 5,000

5,001 to 25,000

22% 25,001 to 75,000

More than 75,000

28%

Part 6. Caveats to this study

There are inherent limitations to survey research that need to be carefully considered before drawing inferences from findings. The following items are specific limitations that are germane to most Web-based surveys.

! Non-response bias: The current findings are based on a sample of survey returns. We sent surveys to a representative sample of individuals, resulting in a large number of usable returned responses. Despite non-response tests, it is always possible that individuals who did not participate are substantially different in terms of underlying beliefs from those who completed the instrument.

! Sampling-frame bias: The accuracy is based on contact information and the degree to which the list is representative of IT security practitioners who are familiar with their organizations’ managed security services. We also acknowledge that the results may be biased by external events such as media coverage. Finally, because we used a Web-based collection method, it is possible that non-Web responses by mailed survey or telephone call would result in a different pattern of findings.

! Self-reported results: The quality of survey research is based on the integrity of confidential responses received from subjects. While certain checks and balances can be incorporated into the survey process, there is always the possibility that a subject did not provide accurate responses.

Please contact [email protected] or call us at 800.877.3118 if you have any questions.

Ponemon Institute

Advancing Responsible Information Management

Ponemon Institute is dedicated to independent research and education that advances responsible information and privacy management practices within business and government. Our mission is to conduct high quality, empirical studies on critical issues affecting the management and security of sensitive information about people and organizations. As a member of the Council of American Survey Research Organizations (CASRO), we uphold strict data confidentiality, privacy and ethical research standards. We do not collect any personally identifiable information from individuals (or company identifiable information in our business research). Furthermore, we have strict quality standards to ensure that subjects are not asked extraneous, irrelevant or improper questions.