Security and Risk Management Strategies Blog

Network Zoning and Virtualization

Blogger: Eric Maiwald

During the course of 2009, I have had a number of client conversations around virtualization and its impact on network zoning. The impetus for the questions normally has something to do with the huge potential cost savings an organization might achieve by virtualizing the servers within its data center.  The more servers that can be virtualized, the greater the cost savings potential and the greater the flexibility an organization will have to load balance and recover applications. It makes a lot of sense to investigate the uses of virtualization technology.

As usual, there are tradeoffs and when we talk about virtualizing the data center, the impact of virtualization on network zoning must be considered. Network zones are built to provide some level of risk mitigation and control. Systems are placed into different zones because organizations are concerned about the potential for an intruder to gain access to sensitive information. If a system does not need to be accessed from the Internet, why should the system be accessible? The perimeters of the network zones are made up of different security and network devices. Most often, a firewall is used to separate network zones.

Many organizations deploy virtualization within a security zone and therefore they do not rely on the virtual environment to control communication between zones. However, if an organization wants to create a virtual environment that includes applications that normally exist in different network zones (the DMZ and the internal zone for example), one of the following possibilities must be true:

Possibility #1: The virtual environment provides the same (or higher) level of risk mitigation and control previously provided by the network security devices (such as the firewall).

Firewalls have been specifically built as a security device. Most firewalls have undergone significant security testing and most organizations implement some level of change control and inspection around firewall rules and rule changes. In addition, firewalls start with the idea that what is not specifically allowed is denied.

We know that applications within a virtual machine can determine that they are running within a virtual environment. This is the first step in finding ways for malicious software to affect other virtual machines or the virtual environment itself. Virtual environments have been evaluated against the Common Criteria and have achieved EAL 4 in at least two cases. However, the evaluations were made against security targets instead of certified protection profiles (see the Common Criteria Evaluated Products List).

Perhaps the bigger issue is something that Trent Henry bloggedblogged about earlier this year, the auditor’s view of the virtual environment. Some regulations (notably PCI) require traditional network segmentation approaches to separate sensitive systems from other systems on the network. Auditors may not view the controls provided by a virtual environment as equivalent to the more traditional approaches which may then open the organization up to the risk of an audit finding or a vast increase in the scope of the audit. If virtual environments are only used within a zone, this is not an issue.

One other thing to think about is that virtual environments may actually decrease some risk. The idea of a homogeneous environment with centralized management may provide some risk benefits around configuration control and logging. These potential benefits should be balanced and traded off with the increase in risk associated with the loss of separation.

Possibility #2: The level of risk has been reduced (or incorrectly evaluated originally) so that the controls provided by the dedicated security device are not necessary.

Everyone makes mistakes and it is certainly possible that the original risks were not as high as the organization thought. Alternatively, some other event has happened (such as a change in the business model) or control has been implemented so that a dedicated network security device is no longer necessary. In fact, it may be that the use of network zones may no longer be necessary. If the dedicated network security device is no longer necessary, then using a virtual environment to house applications from multiple zones makes a lot of sense. The organization should be able to show the risk assessment that provides the evidence of the reduced risk and then they should implement the virtual environment to gain the cost savings.

Possibility #3: Management’s appetite for risk has increased and so they are willing to accept greater risk.

Things change and in this case management is willing to accept greater risk to achieve significant cost savings. This is a valid position for management to take assuming they understand the risks to the business not only from intruders but also from auditors and regulators. In this case, the organization’s management has been told of the risk and has determined that the cost savings outweigh the risks involved. Management has made a deliberate decision to accept the level of risk to the enterprise. The security team should make sure that all of the risks have been identified and explained to management and management should officially accept the identified risks. Once this is done, the virtualization can be implemented to achieve the cost savings.

Virtualization is a great new technology – one that holds out a huge potential for cost savings in the data center. I’m sure that in the future, we will find additional benefits to the technology. Just because it is a “hot technology” does not mean that we should implement it everywhere. Implement it when you need to but be aware that nothing is ever free. Make sure to understand the tradeoffs associated with virtualization and do the risk analysis.

Pesky Virtual Environments

Blogger: Trent Henry

Burton Group’s Catalyst Conference is in the rear-view mirror now, but we continue to get questions from clients about some of the talks we gave. One that I delivered was called “Security, Audit, and Compliance in Virtual Environments.” (If you’re dying to see me actually wander the stage in full-motion video, check it out at Catalyst Replay: www.catalystreplay.com [pay site].)

Generally speaking, auditors are in the early stages of figuring out the implications of virtual environments, whether VMware, Citrix XenServer, Solaris Containers, or even mainframe LPARs (although they tend not to worry about mainframes—IT auditors have history and comfort with them). PCI DSS is causing the first set of issues, because of two requirements. First, “Implement only one primary function per server.” Virtual machines muddy the interpretation of this:

 

Some QSAs take a strict interpretation, and don’t feel that any application processes should coexist on a physical host. Others believe the hypervisors offer a meaningful level of isolation, and therefore applications running in separate guest OSes are not problematic. The PCI Security Standards Council has created a special interest group to clarify guidance here, and virtualization vendors have joined the group to weigh in. The second troubling requirement is “install and maintain a firewall,” but more on this in a moment.

As I spoke with auditors and clients about virtualization security, a number of questions arose. None of these has been consistently addressed by the industry, but we could be at loggerheads soon:

How do we separate systems and limit audit scope with perimeters and zones?

How are servers hardened from attack and kept up-to-date?

How is storage protected in easily-replicated virtual machines?

How are privileged user access and activities controlled (especially with hypervisor admin users who arguably have uber-root control)?

How do we monitor virtual systems?

System separation is a particularly galling concern, because the notion of “firewall” can be quite different in a virtual machine than on a copper-connected network. Is a virtual security appliance sufficient? Can host-based firewalls serve in lieu of network ones, especially to protect peered hosts that share intra-VM traffic on the same physical server (that is, only the virtual switch is used for communication)? Must we employ software that taps hypervisor privileged APIs like VMsafe or Xen Introspection? Or should we backhaul traffic from virtual switches to traditional network firewalls? (Yikes.)

A final consideration is something I call “controls on the move.”

 

The beauty of virtualization is the flexibility it provides, especially in moving workloads from underutilized machines to help with power management (green computing, and all that…) But notice in the graphic that when we rely on a virtual security appliance to protect guests, in-motion workloads can cause some problems of control inconsistency. Namely, if not every machine in a cluster is configured with the same security controls (e.g., IDS, firewall, monitoring), a guest OS could find itself dynamically moved and effectively stripped of some of its protection: That makes both auditors and security teams unhappy. The answer is to ensure good cluster configuration management—that all machines have a common set of protections. But not every vendor or every enterprise has managed that. An alternative is to focus on host-based controls, so each guest OS has its own security detail no matter where it travels. But this has issues, too, not the least of which is licensing cost.

There’s more to discuss, but this is a starting point. The bottom line is that virtualization is a wonderful technology but has to be approached with balance: audit and security, not only flexibility, are important.

Virtualization Security

Topic: Virtualization Security

Bloggers: Dan Blum, Eric Maiwald, and Drue Reeves

Within Burton Group some industry conversation will often spark an exchange of ideas that span traditional research specialty areas. In this case the hot topic du jour is virtualization security and its impact on data center and security architecture. I’ve captured the essential elements of this conversation in the following series of comments.

Dan Blum, Burton Group principal analyst in security and risk management, kicked off the conversation with:

Interesting discussion with VMWare today. It occurs to me that what VMWare is trying to do with virtualization security is a critical part of architecture for the dynamic data center. Virtualization security is also essential to cloud architecture for most or all layers of the cloud stack.

I’ll replay the discussion for you a bit.

As you know, in our security architecture we rate air gap as the highest surety separator, followed by the dedicated network firewall. Any  “virtual” (VM, VLAN) or other separation mechanism is in a lower class of surety.

For an organization’s private data center it may be wise from a surety perspective to separate/isolate the credit card database (or other high value resource) onto its own dedicated server(s) behind a dedicated network firewall.

For a cloud service provider, however, separating dedicated networks and servers breaks the multi-tenant economies of scale; virtualization is usually the way to go. On the other hand, VMWare argues that maybe private data center = dynamic data center = internal cloud.
I’m skeptical that the virtualization economics are as compelling for a internal cloud = dynamic data center as they are for a public cloud (you should be able to afford more dedication/surety).
However, there could be applications where separating/isolating the high value part demands not just one dedicated server, but many, creating a stronger economic argument to trump surety. Anyway, we agree that it is for the good of the industry to “raise the bar” for surety of virtual network and virtual server separation.

“Raising the bar” for virtual separation is much more than just deploying a virtual firewall. That’s just the start. The “secure cloud strength” or “dynamic data center strength” virtual security architecture would need:

Controlled data center network with outer perimeter and IDS
Physical and virtual server operations and configuration standards, hardening, integrity monitoring
Asset management such that virtual workloads are identified, tracked, managed and protected offline
Distributed policy infrastructure (correct policies maintained by vetted personnel in a framework of role-based administration with separation of duty)
Virtual firewalls on each server enforce the distributed policy, correctly identifying the policy that goes with each workload
Audit to ensure standards maintained

Burton Group’s research director for Data Center Strategies, Drue Reeves responds with:

VMware’s primary cloud architecture focus is at the Infrastructure as a Service (IaaS) layer, so it’s understandable for them to talk about internal clouds. (BTW – Internal/private clouds do exist, if not only for the reason of being able to easily move things to a public cloud. However, Bechtel found it cost-effective to move to an internal cloud model).The rest of the cloud (Platform as a Service -- PaaS, Software as a Service -- SaaS) has a larger architecture in terms of usage and consumption models.

Burton Group’s research director for Security and Risk Management, Eric Maiwald also adds his comments to Dan’s initial thought provoking post:

Keep in mind that the necessary security may involve more than one security objective (confidentiality, integrity, availability, use control, and accountability). The objectives may conflict.

The reason I bring this up is because confidentiality and integrity objectives may push an enterprise toward separation. However, availability objectives may push the enterprise toward distributed, highly connected systems. The use of virtualization may help with availability while negatively impacting confidentiality and integrity.

Dan responds to Drue’s post:

When complex virtual zoning schemes (with all the moving parts I indicated in the previous message) are implemented in a large dynamic data center, integrity and confidentiality may be improved. However, glitches in the complex configuration will hurt all the objectives from time to time and raise costs all the time.

Much easier if there’s no need for security. I know – let’s just pretend to have virtualization security!

Drue responds to Eric’s post:

It’s all about risk management, right? We highlight the advantages and risks, then let IT organizations make the business call.

Actually, I think cloud computing may help in some ways (and hurt in others). Data centers will be transformed into critical applications kept internally, while less critical apps are moved to the cloud.

Eric responds to Drue:

Overall, I think cloud will probably help with availability – providing redundancy and alternative sites (not to mention the better availability within a first class data center) that enterprises don’t want to pay for.

Confidentiality, integrity, use control, and accountability are a harder sell. In the macro sense, these are all degraded initially as you have more people who you may not know with potential access to the data. Assuming that the cloud vendors take appropriate steps to keep the external bad guys at bay, you still have to worry about these insiders. Different vendors may do things to control these insiders but it is still something external to the enterprise that “owns” the data.

Drue closes out this discussion with:

Completely agree.

That’s why it looks like internal clouds will keep critical apps, and non-critical apps are candidates to move externally…thereby creating that “air gap” we talk about.

Interesting.

Stay tuned as we continue this discussion at Burton Group’s Catalyst conference in San Diego, July 27-31, 2009. 

What Does It Mean to be a "Virtualization Security" Solution?

Blogger: Pete Lindstrom

It happens with every new technology and/or security fad (think wireless, handhelds, NAC, DLP) – people want to make their solution buzzworthy and so work hard to explain why they are in this “product category” that isn’t really a product category and have some unique characteristics that differentiate them from the crowd of others making the same claims at the same time (try being an analyst amidst these frenzied marketing messages ;-)). The latest buzzworthy term is virtualization security.

The first thing to recognize when evaluating your security vendors’ support for virtualization is that virtualization is designed to be transparent. That means that virtual machines (VMs) running traditional host operating systems are intended to look and feel just like physical OSes and therefore any security solution running on the host will almost certainly run on a corresponding VM (please let me know if you find an agent that doesn’t). You get this automatically. (Congratulations! ;-)).

From a network perspective VM transparency also applies, though network-based solutions must be able to run on OSes that have been ported to the virtual environment. Ironically, the network security solutions that were developed using proprietary hardware and software (typically to meet performance needs) are going to be much more difficult to port to a VM. Software-based solutions that operate on network traffic and security appliances that sit on Linux or FreeBSD, for example, should port over nicely as well.

The only reason any of this matters is to assess the speed for which you need a security solution and the availability of solutions that support that need. There is nothing wrong with having a solution that is virtualization-agnostic, but it is useful to know how unique the solution actually is if it is a differentiator in your evaluation process. In addition, in the near-term security solutions should be able to leverage the characteristics of virtualization for beneficial features such as patching a VM that is not in service or isolating a host agent from the host it is monitoring. That is essentially the VMsafe story.

As vendors pick up and elaborate on their virtualization messages, it may be helpful to have a set of questions to ask them in support of your evaluation. Here are some basic ones (Note: most of these are VMware-oriented since almost all security solutions are starting with VMware integration):

1. What virtualization platforms do you support? If they say “all of them” that is your first indicator that this is a strategy and not a solution.
2. Is your solution running on physical memory (i.e. at the hypervisor level) or is it using virtual memory (in its own VM)?
3. Did you have to rewrite code to integrate into the virtual environment? If so, what components required this? (This is a higher-level question that consumes a lot of the following questions).
4. Does your solution leverage the VMsafe API? On other platforms, does it have access to CPU, memory, network, and file system operations of the physical host?
5. Can your solution track VMs that leverage VMotion across physical hosts?
6. How does your solution identify a VM (e.g. by MAC or IP address, by VM ID, etc.)?
7. Can your solution integrate with Virtual Center or other management platform to take actions specific to VMs?
8. Are you managing configurations (patch/vuln mgt), encrypting communications, “inline” network security (NIPS or firewall), or providing some other security capability?

It is worth noting that there are other security solutions that leverage “virtualization” that sometimes mean something different – for example, many network security vendors are providing virtualization on their network devices to provide for more separation options. Also, there are companies like Phoenix Technologies’ HyperCore or Neocleus that will add virtualization capabilities that leverage hardware virtualization from Intel and AMD.

Five Immutable Laws of Virtualization Security

Blogger: Pete Lindstrom

There are many opinions about the impact of virtualization on an enterprise security posture. A recent article in InformationWeek (http://www.informationweek.com/shared/printableArticle.jhtml?articleID=204301246) gives a good example of the variance in opinions. In reality, we can apply traditional security practices to virtualization to determine whether risk increases or decreases with new virtualization architectures. It shouldn’t be surprising that the increase or decrease in risk is predicated on the current architecture. Here are five laws to live by when evaluating your virtualization architectures.

[This content is excerpted from the Burton Group Overview, “Attacking and Defending Virtual Environments.”]

When combining the standard risk principles with an understanding of the use cases of virtualization, a set of immutable laws can be derived to assist in securing virtual environments:

Law 1: Attacks against the OS and applications of a physical system have the exact same damage potential against a duplicate virtual system.

The point here is that, for all intents and purposes, a VM acts like a physical system, and in most environments that means it can be attacked in exactly the same way. The suggestion that because VMs run in user mode and are therefore more secure is insignificant if an attack remains within the VM and has access to the VM’s content and programs as if it is in kernel mode. So, when comparing a physical system with an OS and applications to an identically configured VM, all existing attacks will work the same way. Importantly, many attacks occur in the application stack, often against sensitive information itself. Any data on the VM may be stolen, and if the VM has network access it may be used as a stepping stone to attack other systems just as a physical system might be so used. The only attack that may be original and different is one that breaks the virtualization mechanisms and compromises additional VMs.

It is important to note that although VM and non-VM attacks are largely the same, the ability to recover from an attack may be significantly increased with VMs, and so the response and recovery may be different.

Law 2: A VM has higher risk than its counterpart physical system that is running the exact same OS and applications and is configured identically.

This corollary to the first rule above introduces the effect of the additional attack surface from the hypervisor and VMM. Because the VMM monitors and responds to a VM, it becomes susceptible to attack itself. The same is true for hypervisors. Any addition of software on a platform necessarily increases its risk if all other things are equal (i.e., there are no attack surface reductions performed in conjunction with the migration). Of course, all other things aren’t equal, so it is important to recognize the basically negative impact of the virtual environment on risk and to pay more attention to offsetting that risk in other ways.

Law 3: VMs can be more secure than related physical systems providing the same functional service to an organization when they separate functionality and content that are combined on a physical system.

When two processes share the same memory space, an attack against one process can impact the other. That is, system ownership by an attacker trumps any other security. One of the ways to benefit from virtualization is to separate various functions into their own isolated operating environments. An even better way is to separate data as well. Either type of separation provides an opportunity to reduce the risk added by the virtualization software.

Law 4: A set of VMs aggregated on the same physical system can only be made more secure than its physical, separate counterparts by modifying the configurations of the VMs to offset the increased risk introduced by the hypervisor.

While separating resources reduces risk, aggregating resources will initially increase risk (Rule 2) and so must be reconfigured or hardened to attain the same level of risk or a reduced level (Rule 3). Turning off services, adding controls, and separating content can reduce that risk.

The most obvious examples here are the replacement of a physical demilitarized zone (DMZ) with one built in a virtual environment and the aggregation of multiple physical desktops (e.g., like the military aggregating classified and unclassified desktops) into VMs on the same physical machine. Once again these VMs must be hardened to a level more secure than their physical counterparts in order to meet a similar risk level.

Burton Group’s current view is that subzones of a DMZ (or any other zone of trust) may be consolidated on a physical system yet separated by virtualization technology provided the risk of a breach is considered low. However, Burton Group recommends that virtualization not be used in a massive or systematic way to mingle production resources across zones of trust (e.g., between the DMZ and the trusted zone). Virtualization used in such a way could easily aggregate risks to medium or high, well exceeding the surety level of software VM isolation, which must be assumed at this time to be low.

Law 5: A system containing a “trusted” VM on an “untrusted” host has a higher risk level than a system containing a “trusted” host with an “untrusted” VM.

Broadly, “trusted” systems are managed and have protection controls deployed, while “untrusted” components may or may not be managed but are generally considered to have weaker protection and are therefore more susceptible to attack.

In American football, the general concept of “getting low” while blocking is an advantage to the blocker. In attacking systems, the same is true. Attacks at lower levels have higher risk than those at higher levels. This is primarily due to the ability to spoof or trick higher-level programs into believing assertions about trust and authenticity.

This rule is particularly important with Type 2 virtualization (OS-hosted virtual machine) because the host OS has a ready attack surface. Because client-side architectures involve both of these scenarios, it is important to recognize and understand the differences. Given this rule, it is important for deployments of trusted VMs in untrusted environments to consider the implications and harden the VM image accordingly.

[Addendum: During additional discussion within Burton Group, the question was raised, "Is this a vote against using virtualization?" On the contrary, we feel that virtualization has great benefit. It's just that some vendors and observers in the industry incorrectly and blithely say that virtualized environments improve security. There are many cases this isn't true--as discussed above--and it's important that practitioners think carefully about proper levels of protection as they move forward with such projects. Virtualization has security issues, but don't throw the baby out with the bathwater.]