Securing Vmware

June 2008 Securing VMware

Analytics Report

As IT groups spawn new virtual machines at a breakneck pace, security is too often an afterthought. Can VMware’s dominance of the enterprise server virtualization market buy us some breathing room? By Joe Hernick

InformationWeek Analytics Reports InformationWeek Analytics | Securing VMware 2

TABLE OF CONTENTS

4 Author’s Bio 5 Executive Summary 6 Research Synopsis 7 Securing VMware: A Shifting Landscape 7 What’s Old Is New 9 Danger On The Horizon 10 Real Threat, But Few Real Answers 11 Enter The VMsafe 12 Fruits Of Their Labor 14 Who’s Responsible For Virtualization Security? 15 Road From Perdition 20 Appendix

TABLE OF CONTENTS

7 Figure 1: Primary Server Virtualization Platform 8 Figure 2: VMware ESX Hosts in Production 9 Figure 3: Virtualized Servers Per VM Host

10 Figure 4: Perception of VM Security Risk 12 Figure 5: Addressing Security Concerns in Virtualized Environments 13 Figure 6: Approaches to Change Management/VM Provisioning 14 Figure 7: VM Security Tool Deployment Plans 15 Figure 8: Hyperjacking Concerns 16 Figure 9: Security Patch Management 17 Figure 10: VM-specific Security Tool Production 18 Figure 11: Planned Security Spending for Virtualized Environments 20 Figure 12: Involvement in Security Initiatives 20 Figure 13: Involvement in IT Operations 21 Figure 14: Job Title 21 Figure 15: Company Revenue 22 Figure 16: Industry

22 Figure 17: Company Size

Joe Hernick has covered virtualization, storage, operating systems, voice, and other topics for InformationWeek, Network Computing, and other publications for seven years. Joe sits on the editorial advisory board for Dark Reading and is a member of the CAIS Commission on Technology. He has been involved in start- ups, training, consulting, and most recently was a technology services manager at a Fortune 100 insurance company, where his work involved OS rollouts for 63,000 desktops, Y2K readiness, call-center load balancing, automated pharmacies, new-site construction, old-site consolidation, and HIPAA compliance.

Joe currently manages InformationWeek’s Virtualization Test Lab, running VMware, Citrix Xen, Virtual Iron, Microsoft, and Parallels hosts. He holds a BA in Economics, a Master’s in Information Management and is a PMI-certified Project Management Professional.

Executive Summary: Our survey on the state of VMware security revealed some startling facts: Just four in 10 consider hyperjacking a realistic threat, and nearly half take a laissez faire approach to virtual machine provisioning and management. Some even let business units deploy VMs with no oversight, perhaps because 20% assert that VMs are safer than physical servers.

The reality—and a concept that many IT and business managers fail to grasp—is that a virtual server is still a server. A production VM, and its host, must be held to the same level of rigor as a comparable physical production server, with identical change management policies for approval, deployment, patching, and other processes. We’re not saying we’d turn back the virtualization tide, even if that were possible. The ability to abstract servers from the physical world to the virtual—P2V—and consolidate multiple legacy servers onto a smaller number of virtualization hosts is yielding signifi- cant financial and operational advantages, including a smaller attack surface and opti- mized performance. However, virtualization also creates management and security challenges not faced in legacy data center environments.

For now, there are few new security concepts required once you enter the virtualized world. Traditional best practices are just as important, if not more important, than VM-specific security toolsets. Still, any hypervisor needs to have security baked in from the beginning, not tacked on as an afterthought. Armies of attackers are no doubt working feverishly for the bragging rights that will come with being among the first to hyperjack a high-value server.

So are industry-leading virtualization vendors doing enough to keep us safe?

VMware currently dominates the enterprise-server virtualization market, though Microsoft is in hot pursuit. We’ll examine whether VMware’s VMsafe program—which provides APIs with hooks into the ESX hypervisor—will pay off for IT, and maybe even help keep Hyper-V at bay.

For this report, we interviewed security experts from VMware and VMsafe partner vendors and polled 423 business technology professionals to assess concerns over, and security strategies in place for, virtualized environments in real-world organizations. We talked to security professionals who support—and are critical of—the burgeoning virtualization-specific security market, and even had a chat with Simon Crosby, CTO of VMware competitor Citrix, regarding the state of virtualization security.

What he had to say about VMware’s security initiatives may just surprise you.

Research Synopsis Survey Name: InformationWeek Analytics VMware Security Survey Survey Date: May 2008 Region: North America Number of Respondents: 423

Purpose: To examine security concerns and practices for virtualized servers among business technology professionals.

Methodology: The InformationWeek Analytics VMware Security Study was fielded on the Web in May 2008. This report examines the responses of 423 business technology professionals.

The sample for this project was taken from the subscriber base of InformationWeek. The results of the survey were aggregated and analyzed by representatives of InformationWeek.

Securing VMware: A Shifting Landscape Each security vendor we interviewed for this report is focusing on product development for VMware. And all of those vendors also have plans for Hyper-V and/or Xen product development. Making like Switzerland between VMware and Microsoft is a rational move—a reality backed up by our survey of 423 business technology professionals. VMware is still the dominant player in server virtualization, with 56% of installations, most Infrastructure3/ESX. But our poll reflects the growing influence of Microsoft: 24% of respondents listed either Hyper-V or Virtual Server 2005 as their primary server virtualization platforms. Citrix XenServer took third, with 7%.

This is a far cry from estimates of 70% to 80% VMware ownership of the server virtualization landscape. An outlier? Maybe. While VMware has the longest track record and the broadest slate of product offerings, other vendors are racing to catch up. We expected Hyper-V to make a mark, but we must admit to being surprised by these results. Figure 1 Primary Server We want to be very clear on this point, because Virtualization Platform it informs our security recommendations: The Which virtual machine (VM) hosting/hypervisor system is virtualization market is still developing, with your organization's primary server virtualization platform? Hyper-V riding Windows Server 2008 into the data center this year, Citrix leveraging a large 45% VMware Infrastructure 3/ESX 52% Presentation Server installed base, and myriad Viruses boutique hypervisor vendors targeting niche 24% Microsoft (Virtual Server 2005 or Hyper-V) market segments. Before buying virtualization- specific security products, especially those that 7% Citrix XenServer hook into a particular VM infrastructure, make sure you know where you’ll be in a year or two. 2% Parallels/SWSoft/Virtuozzo For now, scrupulously applying the security les- 2% sons we all learned the hard way in the physical Oracle VM world should keep your virtual systems safe 1% while you plot a course—assuming you haven’t Novell SUSE/Xen gone VM wild. 1% Solaris Containers / Sun xVM WHAT’S OLD IS NEW 1% In our poll, when we asked about VM-specific Virtual Iron security plans, 39% said they don’t need special- 11% ized tools, opining that a VM is just another Other VMware server. 2% Other Well, yes and no. 3% None

The problem is, the ease with which we can cre- Data: InformationWeek Analytics VMware Security Survey ate and deploy virtual servers has gone to a few of 423 business technology professionals IT pros’ heads—provisioning a VM takes literally

VMwareSecurity 1 52% Viruses InformationWeek Analytics | Securing VMware 8

minutes. People who ought to Figure 2 know better are dispatching VMware ESX Hosts in Production VMs into the wild at a pace that How many VMware ESX hosts are in production in your organization? outstrips internal security review and audit procedures. 4% More than 100 Blame it on budget pressure, 17% customer demand, weak man- 11-50 agement toolsets, lack of VM- specific policies, the animal 76% Fewer than 10 attraction of sexy technology, good-old human foolishness, 3% running out of data center 51-100 space, or any combination of the above. The fact is, many organi- Data: InformationWeek Analytics VMware Security Survey of 423 business technology professionals zations today are running ESX shops by the seat of their pants.

And we don’t expect Hyper-V to help matters. But that’s another report.

An ESX host, at its VmostMwa basicreS conceptualecurity model, 2 is a single physical server concurrently running multiple virtual servers, also known as guests, or VMs. The host platform does this by essentially tricking each VM into believing it has sole access to all physical system resources. Operating systems like to think that they have exclusive one-on-one relationships with system memory, CPUs, I/O calls, and other nuts-and-bolts connectivity.To get around this, modern hypervisors operate between guest machines and the physical host, intercepting calls to hardware and delegating physical resources to the VMs as required. An ESX host becomes a self- contained environment, virtualizing servers’ access to RAM, disk storage, and the network.

Therein lies the risk: This self-contained bundle of servers and virtual networks is outside the sight of traditional enterprise security apps. All those expensive intrusion-detection and NAC systems, active traffic analyzers, and any other widget in your security kit will view an ESX host as a single physical server. Intrahost links, hypervisor-to-VM connections, and the virtual network will be invisible to traditional tools. Take patching, among most fundamental of server security functions: When we asked how distribution and installation of security patches is handled, 15% said “we do the best we can.” Just 23% have automated procedures for both hypervisor hosts and guest VMs. It’s likely that “best effort” is not going to measure up for long.

When asked how change management/VM provisioning is handled in their organizations, close to half (45%) said that VMs are created with no review process. A shocking 4% of respondents admitted that virtual machines could be deployed by anyone, including non-IT employees, without review. Just one in five has included VM-specific provisions in their formal review processes.

DANGER ON THE HORIZON Virtualization security risks fall into two primary categories: intra-host threats and hyperjacking.

It’s intra-host threats that make the security folks we spoke with most nervous. The primary concern is that a guested OS could be compromised via a known OS or application exploit, then other VMs on the same host could be targeted from the compromised VM. With all attack vec- tors occurring within the virtual network on the physical ESX host, traditional network security tools would fail to detect suspicious traffic, as long as attackers contain their actions to VMs on the same host as the compromised guest.

If an intra-host attack is the more likely exploit, the worst-case theoretical scenario in a virtual- 52% ized environment is hyperjacking, where an exploit leads to a compromised physical host server, Viruses allowing criminals full access to all guests on a given machine. In subverting the hypervisor, malicious software could disguise its presence from security tools that may reside not only on the physical network outside the host, but even within hosted-OS partitions or in any software layer above the hypervisor. The exploit situation is analogous to the threat of cloaked rootkits compromising a standalone server OS: If you successfully control the Figure 3 hypervisor, you can manipulate all packets traversing the hypervisor Virtualized Servers Per VM Host How many virtualized servers (VM guests, VMDKs) and are in a position to sample, on average does your organization run per VM host? redirect, or spoof any data traffic to 8% or from hosted VMs. Lacking a More than 20 security provision, guest OSes have 5% 16% no way of knowing they’re running 16-20 1 on a compromised platform. 9% 11-15 Remember that most OSes are not 41% natively VM-aware; guests operate 2-5 under the illusion that they’re run- 21% 6-10 ning as a 1x1 server on dedicated hardware. Newer server OSes, such Data: InformationWeek Analytics VMware Security Survey as Windows 2008 and some specific of 423 business technology professionals Xen-aware Linux builds, do a better job of leveraging hypervisor-emulated hardware than legacy platforms. Future desktop and server OSes will be designed from the ground up to run in virtualized environments, taking advantage of virtualization-tuned hardware and hypervisorsVMwa from reSmultipleecu vendors.rity 3 Until then, though, we need to be on guard.

Hyperjacking has been widely discussed for at least a year, and yet, incredibly, 36% of respondents to our survey had never heard the term. Just 8% say concern over hyperjacking keeps their organizations from storing sensitive data on VMs.

REAL THREAT, BUT FEW REAL ANSWERS We see why there’s concern among security experts—when you look at large-scale virtualization systems that enable 10, 50, even hundreds of hosted servers to run on a single hardware platform, the potential for loss of control and revenue is enormous. It’s in everyone’s best interests to maintain the integrity of the hypervisor while building in multiple fail-safes for hosted OSes, to ensure they’re communicating with an untainted hypervisor as a bridge to underlying hardware and external connections.

Moreover, bad news sells. Gartner made headlines predicting that a patch-worthy hypervisor vulnerability would be discovered in a mainstream product before the end of this year. Other analysts looking for air time are quick to point out the potential risk, offering proof-of-concept examples and whispering of doom and gloom.

We take Gartner’s prediction with a big grain—make that a pound—of salt. Any likely hyperjack exploit will require physical access to a host platform, analogous to a rogue employee installing a rootkit on a traditional server. Will it happen in the wild? Sure. Is it likely an ESX host in your secure data center will be “owned” by the end of 2008? We don’t think so, though of course there are no ironclad promises.

As we’ll state many times in this report, spending your time focusing on basic infosec policies 52% and practices will yield a safer VM environment, rather than fretting about hypervisor exploits. Viruses Chris Hoff, chief security architect at Unisys and author of the Rational Survivability blog, is on our side of the fence, suggesting that while hyperjacking is certainly a theoretical threat, companies would be wise to spend their defensive dollars addressing the risks actually facing them today. Virtualization and security vendors tend to agree, admitting that hyperjacking is a slim pos- sibility but recommending that we take a measured risk analysis and ROSI (return on security investment) approach toward Figure 4 allocating security dollars. Perception of VM Security Risk In your organization, how are VMs viewed with regards to security risk? We fully concur. Applying a healthy dose of common sense 13% to potential threats will yield I don't know the best return on your securi- 20% 3% Safer than ty investment, while reinforc- Much less safe physical servers than physical servers ing your legacy security infra- 9% structure and maximizing Less safe than physical servers staffing. Consider your current risk profile and potential zero- 55% As safe as day exposure as you follow physical servers traditional change control and security procedures for your Data: InformationWeek Analytics VMware Security Survey of 423 business technology professionals VM hosts and guests.

Frankly, it doesn’t matter if your company is looking at VMware, Microsoft’s Hyper-V,Citrix’s Xen offerings, or one of the smaller specialized players in the market to implement virtualization. Shoring up traditional data and infrastructure security will still yield a bigger short-term payoff vs. large investments in fledgling virtualization-specific security offerings.

For the 70% of respondents who say their organizations are earmarking funds for protecting virtualized environments, “soft” spending, such as devoting funds to updating change-management policies and training non-operational IT staff on VM concepts, will yield extended benefits. For something more concrete, larger IT organizations should review virtualization-specific offerings from your existing management and security vendors; as established software shops work with the VMsafe APIs, enterprise toolsets will gain visibility into the virtualized world. Contact a VMsafe partner to see what’s available for testing that will work with your infrastructure.

We’re not big fans of hiring consultants on a whim, a VMware partner or Microsoft expert can prevent missteps in VM deployment and security design. Unless you have a solid skill set and knowledge base in house, bringing in help to review your specific virtualization security strategy represents dollars well spent.

ENTER THE VMSAFE In February,VMware Want Tighter Security? announced an official API Get Really Small for ESX hypervisor security under its VMsafe program, With traditional operating systems tipping the gigabyte aiming to provide a common scales, hypervisors present far smaller targets for would-be toolset for monitoring traffic villains. But will hypervisors resist the bloat that has made through the hypervisor, making hosts a touch more traditional OSes so vulnerable? So far, VMware and other ven- transparent. The list of dors are striking a balance between adding functionality to the backers reads like a Who’s core hypervisor, providing APIs to interact with the host Who of security vendors and directly, and keeping the code base as small as possible. includes everyone from big That’s not secure enough for you? One alternative is the guns to virtualization secu- “bundled” hypervisor, where minimalism is taken to the rity specialists. extreme. VMware’s offering is ESXi 3.5, a 32 MB hypervisor The VMsafe initiative touted as the world’s smallest. VMware partners with OEMs to enables third-party security provide ESXi as a free, bundled virtualization option for budg- products to gain visibility, et-conscious customers. Thin hypervisors are also attractive using ties to the ESX hyper- to the ultra-security-conscious, offering solid performance visor, into the operation of a while minimizing vulnerabilities, thanks to the exceedingly virtual machine to identify minute potential attack surface. and eliminate malware, including viruses, trojans, and key-loggers. Security

vendors can leverage VMsafe to sniff out and eliminate malware that is undetectable on physical machines.

VMware’s pitch implies that VMs running in a VMsafe ESX host are actually safer than physical servers due to process monitoring and inspection techniques that can occur courtesy of the abstraction of guested operating systems. Visibility into hypervisor activity provides the oppor- tunity for revolutionary observation and analysis tools for security vendors. VMware goes on to say that VMsafe includes sharing an “open, interoperable, and cross-platform set of technologies with partners so they can provide innovative security solutions.”

Thing is, “open” in this case refers to compatibility of products within the VMware ecosystem. Still, applications from diverse third-party vendors should play well with others, as long as all apps are written to spec. We expect VMsafe to provide enterprise customers with better security, granularity, visibility, correlation, and scalability in virtual machine deployments.

Specifically,VMsafe integration allows third-party security tools to monitor VM memory pages and CPU states; enables filtering of packets within the virtual network, to both the hypervisor and intrahost connections between VMs; provides in-guest, in-process APIs for complete monitoring and control of process execution; and allows storage control for guest VM disk files.

Current VMsafe partners are Altor Networks, Apani, BigFix, Blue Lane Technologies, Catbird Networks, Cenzic, Check Point Software Technologies, Configuresoft, F5 Networks, Fortinet, Fortisphere, IBM, Imperva, Kaspersky Lab, McAfee, Montego Networks, Reflex Security, RSA, Secure Computing, Shavlik Technologies, Solidcore Systems, Sophos, Symantec, Third Brigade, and Trend Micro.

Figure 5 Addressing Security Concerns in Virtualized Environments How does your organization address security concerns in virtualized environments?

Utilize traditional infrastructure security tools (firewall, network monitoring, intrusion detection systems, antivirus, NAC, etc.) with no specific provisions for virtualization 54% 52% Utilize traditional infrastructure security tools (firewall, network monitoring, intrusion Viruses detection systems, antivirus, NAC, etc.) with virtualization modules or plug-ins 17% Utilize virtualization-specific security tools provided by virtualization vendor to manage security in virtualized environments in addition to traditional security tools 12% Utilize third-party security tools designed for VMs (virtual security appliance, IDS) to manage security in virtualized environments AND traditional environments 3% Utilize third-party virtualization-only security tools (virtual security appliance, VM-targeted IDS or intra-host monitor, etc.) to manage security in virtualized environments in addition to 2% traditional security tools No VM security provisions in place 12%

Data: InformationWeek Analytics VMware Security Survey of 423 business technology professionals

VMwareSecurity 5 InformationWeek Analytics | Securing VMware 13

52% FRUITS OF THEIR LABOR Viruses So, just how safe is VMsafe going to make us? We’ll have to wait and see. While a number of capable virtualization security products are already on the market, and the majority of those vendors are VMsafe partners, as of early June, Figure 6 VMware and its partner Approaches to Change companies had yet to pro- Management/VM Provisioning duce a security product to How is change management/VM provisioning handled in your organization? the VMsafe specifications. 4% But we should see some- VMs can be created by anyone in the thing soon: In every case, organization interviews with represen- with no review process tatives from VMware, 5% 20% VMs can be created Formal review Symantec, Fortisphere, by anyone in IT process with with no review process and Reflex Security sign-off on VM security concerns showed a strong, almost 36% obsessive, drive to get VM creation 35% managed Formal review process product to market. by IT operations with no VM specific administrators check list items; with no change i.e. creation of a new It’s encouraging that management VM is equivalent to review a physical server build VMware is partnering with a wide range of companies, from small start- Data: InformationWeek Analytics VMware Security Survey of 423 business technology professionals ups with niche, virtualization-centric security products to large, established vendors. We’ve been critical over the past few years concerning the absence of VM integration withV Mwaenterprise-levelreSecurity security 6 tools—and the seeming apathy of vendors in this regard.

In hindsight, it’s clear that established security players such as McAfee, Symantec, and Sophos were reluctant to invest heavily in potentially proprietary hooks for securing VMware, and it seems they were smart to hold off. VMsafe provides these vendors with a standard API, and customers with a level of confidence for investment. Still, 30% of survey respondents say they won’t make any virtualization security-specific purchase in fiscal year 2009. In fact, 58% will make only minor investments in virtualization security, planning to spend less than $50,000 on VM-specific tools over the next year.

That sounds about right to us. Unless you have a clearly identified risk associated with your ESX environments, such as a large public-facing Web farm, guest VMs purposely running a few out-of-date security patches due to application compatibility issues, or terrifyingly loose control policies, we recommend holding off on making large-scale virtualization security investments. A fiscally conservative and targeted approach makes sense until VMsafe is fully baked.

When will that be? Our best guess based on off-the-record vendor discussions is that 4Q08 will

bring a nice variety of VMsafe-branded offerings. Coincidently, this will also be after Hyper-V finally hits production and VMworld in September. Remember, all VMsafe partners are also working on products for Microsoft and Xen-based hypervisors in tandem with ESX development. What’s the No. 1 priority? Meeting the VMworld deadline. The second order of business 52% Viruses for these vendors is widening the customer base to Hyper-V installations and/or Citrix shops.

WHO’S RESPONSIBLE FOR VIRTUALIZATION SECURITY? Simon Crosby, CTO of Citrix, generated a lot of buzz this spring when he stated that he was not in the business of securing virtual environments. To be specific, Crosby said in an interview that, “Citrix is not a security vendor for guests of the virtualized infrastructure.”

Crosby’s comments have been lambasted, Figure 7 misquoted, and dissect- VM Security Tool Deployment Plans ed by the press, with Does your organization plan to deploy VM-specific security tools in the next 18 months? Hoff as one of the most vociferous detractors. 21% We already have We interviewed both 39% VM-specific security There's no need for in place for this report. Crosby VM-specific tools; a clarified that he and VM is just another server 20% his company are “mani- We're waiting for our acally” focused on primary security vendors to bundle VM-specific security—as, he adds, support 20% are all virtualization We're planning to purchase vendors with regard to VM-specific tools their respective hypervisors. But when it Data: InformationWeek Analytics VMware Security Survey of 423 business technology professionals comes to third-party security offerings for client OSes and dependent applications, Citrix is relying on the vitality of the open-source Xen community and the emergingVMwa virtualizationreSecu marketrity 7 for vetting. Citrix’s virtualization security responsibility does not extend to the inner workings of VMs.

As Crosby says, it is “not my job to fix every legacy flaw in Windows.”

Hoff finds fault with Citrix’s hypervisor-centric approach, perhaps not surprising given his somewhat argumentative world view. Crosby respectfully disagrees with Hoff and other critics. And so it goes.

The reality is, virtualization security will ultimately rest on three legs: The virtualization vendor, third-party security tool vendors, and in-house IT/security staff applying existing policies to VM environments and virtualized servers. Other opinions of VMware aside, in our discussions, Crosby’s view of the VMsafe program seemed to be one of genuine respect. He acknowledges that VMware’s more than five-year lead and market presence facilitated creation of the VMsafe

initiative, and VMware’s partner agreements Figure 8 and customer base have provided the ecosystem for virtualization security tools to take root. Hyperjacking Concerns Is your organization concerned about "hyperjacking," an attack where the host hypervisor is compromised, exposing Still, he couldn’t resist placing an open source all guested servers to potential attack or exploitation? spin on VMsafe’s APIs; Crosby says he would like to see fully open standard virtualization 8% security hooks that allow security vendors to Hyperjacking scares the heck out of our security 52% folks; we have policies that restrict certain data Viruses be platform agnostic. Where VMware has pub- from being hosted on VMs lishedViruses VMsafe guidelines, it would be safe to 32% say the company has a strong interest in keep- Hyperjacking is a likely risk that needs to be ing third-party vendors close at hand. addressed by VM-specific security tools 15% Hyperjacking can only occur if someone has Every VMsafe partner we spoke with is, first physical access to a VM host, so we're not worried and foremost, developing security products to 9% the VMsafe spec, targeting VMware customers. Hyperjacking is a theoretical attack with no And, as mentioned, all of them are covering real world risk their bases and working on at least one paral- 36% lel project to support their products on Never heard of hyperjacking Microsoft’s Hyper-V or Citrix Xen. Data: InformationWeek Analytics VMware Security Survey of 423 business technology professionals

For the moment, however, all hypervisors appear to be secure or readily patchable, a handful of virtualization security tools are on the market ... and the lion’s share of responsibility still rests in localVMwa hands.reSecurity 8

“The real problem of security in a virtualized world is not technical, it is organizational and operational,” says Hoff. “With the consolidation of applications, operating systems, storage, information, security, and networking—all virtualized into a single platform rather than being dis- cretely owned, managed, and supported by reasonably operationally mature teams—the biggest threat we face in virtualization is [the fact that] we now have lost not only visibility, but the clearly defined lines of demarcation garnered from the separation of duties we had in the non- virtualized world.”

ROAD FROM PERDITION Fortunately, there are actions IT can take now to get back that operational maturity. Regardless of your company’s hypervisor preference, a universally applicable question all operations managers should be asking is, “How does our organization address virtualization security?”

If the answer is, “We don’t,” you’ve got some work ahead of you. But you’re not alone. This atti- tude seems to play out in our survey data: 12% of respondents have no VM provisions in place at all, 17% rely on traditional infosec tools with VM-specific provisions or plug-ins, 12% use tools

provided by their VM vendors, and a slim 5% of early adopters have implemented third-party virtualization security toolsets.

The majority (54%) relies on traditional infrastructure security tools without any VM-specific provisions. So what can you do right now to mitigate the limitations of VMware management tools and SAN architectures?

Figure 9 >> Separate VMware’s service console from the primary network or management VLAN by plac- Security Patch Management ing it on a separate physical network, or isolate How does your organization manage distribution and installation of security pat management tools via VLAN segmentation. You ches? don’t want the wrong folks sniffing your packets 23% en route to service control. Automated tools patch hypervisor hosts 52% and guest VMs as soon as patches are available Viruses >> Recognize that you will migrate VMs from 25% Automated tools patch guest VMs, hypervisor host to host. Vmotioning makes sense in any patches are manually applied multihost ESX shop for reasons ranging from 29% adaptive resource scaling to production failover Manual patches are applied to hypervisor hosts and guest VMs and disaster recovery. Still, precautions are required. Of primary importance is to ensure that 3% Guest VMs are patched, hypervisor host servers and VMs involved in Vmotioning are hosts are not patched affiliated with trust zones, and that relationships 2% among trust zones are understood and clearly Hypervisor hosts are patched, guest VMs are not patched defined. “Zones of trust” are areas of the network or infrastructure where all components can be 15% trusted completely; in VM-speak, all virtual net- We do the best we can work components and guest server instances, for 3% We rarely apply security patches due example, expect to operate with the same securi- to application conflicts ty and environmental privileges. Data: InformationWeek Analytics VMware Security Survey of 423 business technology professionals Don’t make the surprisingly common mistake of migrating sensitive VMs to less-than-secure servers. And remember that Vmotion info is sent in the clear. Unless your organization plans to invest in hardware-based SSL encryption for Vmotion data paths over open wires, play it safe and restrict Vmotioning to a dedicated network. VMwareSecurity 9 >> A very good rule of thumb: Label everything, physical or virtual. Reinvent yourself as a type-A reference librarian with a top-of-the-line labeling gun. Clearly tag and record details of all physical and virtual servers, as well as all physical and virtual networks. In particular, have you taken a good look at your wiring closets or patch panels lately? This is a good moment for introspection. Unless you can identify source and destination points for every cable, a rat’s nest of a wiring closet adds an unneeded layer of confusion to virtual environments.

>> Assess the state of documentation for VLANs in your physical network, and review DHCP rule sets and ACLs. Are you sure all core and all edge switches are in synch with your VLAN plan? Once you have your cable plant and physical and logical networks reflected in your documentation, turn your attention to your virtual networks. You can trace a wire in the physical world, or dig into your core switch and dissect ACLs. Your network folks can easily set up mir- rored ports and play with sniffers to look for trouble. But do you have experience troubleshoot- ing connection issues in virtual networks? You will if your ESX networks and connections are clearly defined and documented.

52% >> Quis custodiet ipsos custodes? Remember to watch the watchers. Who really needs to be Viruses an admin in an ESX environment? Assess rights and privileges per ESX box and per virtual server, and limit both user and admin access to required VMs. Determine if local admins really need to tweak resource or other environmental variables for their respective VMs. If conflict or debate ensues, you can always ask if database administrators get to crack the cases of “their” physical servers to add memory or swap drives. Resist the urge to delegate full autonomy and control to application owners, Figure 10 line of business partners, even VM-specific Security Tool Production members of the IT department Does your organization have VM-specific security tools in production? without requisite skills and trust levels.

15% If your organization chooses to Yes grant management authority to users outside of the ESX admin team, at least minimize the num- 85% No ber of folks with full access to the service console in favor of Web access controls. As with rules and policies for your physical environment, always limit top Data: InformationWeek Analytics VMware Security Survey of 423 business technology professionals admin rights and privileges to a very select few.

>> Invest in VMware’s VirtualCenter management software. As mentioned many times in this and other reports,VsecurityMwareS concernsecurity in VMw 10are are really management concerns. Relying on the management tools in VirtualCenter reduces the potential for human error in virtualization security; VM administrators can take a page from standard operations and security practices by implementing and enforcing centralized policies across ESX hosts and VMs. Global policies can alert administrators to security and design concerns and/or simply restrict the creation of guest instances that violate organizational guidelines.

Global policies, quite simply, protect us from ourselves. VirtualCenter allows administrators to forgo the service console for the vast majority of tasks while providing the added organizational

security benefit of leveraging Active Directory for user authentications. VMware Infrastructure 3 installations managed via VirtualCenter can rely on an organization’s existing directory management tools to control administrator and user access to hosted environments. Note that the service console cannot directly access AD for user accounts. It can, however, rely on AD to vet a local user’s credentials. VirtualCenter automatically tracks all commands and changes implemented via its interface, yielding a centralized, solid audit trail in the advent of an issue. It’s well worth the price.

>> Reign in VM resource hogs. ESX, like all hypervisor platforms, allows administrators to 52% Viruses clearly allocate resources to specific guests. This strategy ensures that a high-needs VM with a production application receives preference over a low-priority VM running maintenance tools, for example. Shares and limits allow IT to determine and allocate the correct mix of resources from the host’s limited pool of CPU, memory, and storage.

In the event of a compromised Figure 11 or malfunctioning virtual machine, share and limit Planned Security Spending thresholds can also restrict the for Virtualized Environments resources granted to the offend- How much does your organization plan to spend next fiscal year to protect your virtualized environments? ing VM. If no upper bounds are in place on your host, either by design to match highly variable 30% 29% No additional Less than $5000 resource requirements across funding outside of existing VMs or due to a poorly imple- security budget mented VM, the misbehaving machine can quickly consume 5% an inordinate percentage of More than $100K 14% resources and negatively impact $5001-10000 other VMs. 7% 15% $50001-$100K $10001-50000

Data: InformationWeek Analytics VMware Security Survey Worse, if Vmotion is in use, a of 423 business technology professionals rogue VM could potentially spread havoc across multiple hosts, leaving a path of destruction in its wake. This is not a good thing and is easily prevented by implementing some basic guidelines. One rule of Vthumb:Mwa TreatreS ecuall VMsrity as 11 generally well- behaved toddlers who may occasionally misbehave, sometimes with disastrous results.

>> Remember that Root = God, VMware is based on Unix, and Unix is monotheistic.

Restrict root user privilege. No one should be logging in remotely to an ESX host as root. If you can, someone else can. Lock out all weak connection protocols on your hosts, then permit SSH for a very limited pool of admin user accounts. Remember that sudo grants rights to privileged

commands. Tell your admins that they can wield god-like power via sudo, then let them be demigods.

Root is god. God does not get logged. Regular accounts running sudo get logged. Root and su access does not get logged.

Did we mention that no admin should have root power in your ESX shop?

>> Bolster the firewall and keep the attack surface small. If you can’t run your shop on inte- grated hypervisors like ESXi 3.5 (see related story, Page 11), at least make every effort to limit hypervisor plug-ins and add-on services for ESX. Do you really need to run a generic SNMP trap on every virtualization host? There are more secure methods of performance monitoring available via the console.

Do you really need to open up an additional inbound port on the hypervisor host to allow access for a third-party management tool? ESX ships with damn near every network port locked down for a very good reason: You are placing many eggs in one basket. Every time you open an additional port, no matter what the justification, you are making your VMware installation less safe.

By default, ESX Server 3 maintains a firewall between the network and service console. This is, again, a good thing. Out of the box, the firewall is configured as high security to be as restrictive as possible, limiting all inbound and outbound traffic excepting ports 22, 80, 443, and 902. Dropping down to medium security starts with these four ports open inbound, no restrictions on outbound traffic, and the option to open up additional inbound ports.

In VMware parlance, low security equals no security, no firewall. Think long and hard about any proposal to weaken firewall settings below high security.

52% Viruses

Appendix

Figure 12 Involvement in Security Initiative To what degree are you involved in the security initiatives at your organization (inclusive of strategy, technology purchase, systems management, etc.)?

Highly involved Somewhat involved Neutral

Not very involved Not at all involved

10%

13%

39%

16% 52% Viruses 22%

Data: InformationWeek Analytics VMware Security Survey of 423 business technology professionals

Figure 13 VMwareSecurity 12 Involvement in IT Operations To what degree are you involved in IT operations at your organization?

Highly involved Somewhat involved Neutral

Not very involved Not at all involved

5% 9%

55%

14%

17%

Data: InformationWeek Analytics VMware Security Survey of 423 business technology professionals

Figure 14 Company Revenue What is the annual revenue of your entire organization?

14% Don't know/decline to say 24% 7% Less than $6 million Government/Non-profit

7% $5 billion or more 19% 7% $6 million to $49.9 million $1 billion to $4.9 billion 5% 6% $500 million to $999.9 million $50 million to $99.9 million 11% $100 million to $499.9 million

Data: InformationWeek Analytics VMware Security Survey of 423 business technology professionals

Figure 15 VMwareSecurity 15 Job Title Which of the following best describes your job title? 52% Viruses 18% 2% IT/IS staff COO/Operations management 18% 2% Director/Manager, IT or Infrastructure CTO 15% 2% CEO/President Vice President, non-IT 6% 1% Director/Manager, IT Operations CFO/Financial management

5% 1% Line of business management CSO (chief security officer)/Security management 4% 1% CIO Director/Manager, Telecommunications 4% 1% Consultant Security staff 4% 14% Vice President, IT or Infrastructure Other IT Director/Manager 3% Director/Manager, Network Systems

Data: InformationWeek Analytics VMware Security Survey of 423 business technology professionals

VMwareSecurity 14 52% Viruses

InformationWeek Analytics | Securing VMware 22

Figure 16 Company Size Approximately how many employees are in your organization?

16% 10,000 or more 28% Less than 50 5% 5,000-9,999

15% 1,000-4,999 8% 50-99

8% 20% 500-999 100-499

Data: InformationWeek Analytics VMware Security Survey of 423 business technology professionals

Figure 17 VMwaIndureSecustryrity 17 What is your organization's primary industry? 52% Viruses 11% 3% Consulting and Business Services Financial services/Securities and Investments 9% 2% Education Non-profit 9% 2% Government Utilities 9% 2% Healthcare/Medical Financial services/Banking 7% 2% IT Vendors Financial services/Insurance 6% 2% Manufacturing/Industrial, non-computer Logistics/Transportation

4% 2% Media/Entertainment Electronics

4% 2% Telecommunications/ISPs Financial services/Other 3% 2% Biotech/Biomedical/Pharmaceutical Insurance/HMOs 3% 13% Retail/E-commerce Other 3% Construction/Engineering

Data: InformationWeek Analytics VMware Security Survey of 423 business technology professionals

VMwareSecurity 16