Building security by design - a discussion of VMware App Defense

A truism about both personal and national defense is that cooperation and partnership provide more security than isolation and seclusion, yet for the longest time, just the opposite has been true when it comes to application and software security.

One of the early fears about virtual machines was that their reliance on shared server and hypervisor left virtualized applications vulnerable to attack from rogue VMs on the same system, particularly ones that might breach the security layer of the hypervisor itself.

Experience has shown that these worries were overblown as well-designed VM deployments have proven just as secure (or insecure, depending on the implementation) as their bare metal counterparts. Ironically, VMware, arguably the trend setter in VMs over the last decade, has been pursuing methods that make virtualized infrastructure and applications more secure than discrete systems. Most recently it offers that security through the AppDefense product recently announced at VMworld U.S.

Virtualization increases security granularity

VMware first started applying virtualization to security with its virtual network overlay product NSX, which can create arbitrarily granular network segments, so-called microsegmentation, each with unique security policies tailored to a particular application, user group or set of information governance controls.

NSX is uniquely positioned to provide such minute control due to its role as a network controller managed by a VM operations console (vCenter) responsible for system, application and user configurations along with overall security policy. As this NSX blog notes in highlighting the importance of local control over security enforcement,

The only way to meet the previously defined requirements is for a solution to have deep integration at necessary protection points, complete visibility and control of traffic flows, and a management and policy layer that is automated and flexible. NSX provides this level of security for all workloads within a data center, both physical and virtual. … NSX places security controls at the hypervisor level. To achieve the tightest control over traffic flows, it is essential to place the security closest to the workload, while still residing in a separate and secure trust zone based on the hypervisor.

With AppDefense, VMware applies the same design philosophy to applications running within the hypervisor.

Unlike network flows that are explicitly described, routed and controlled via TCP headers, application interactions aren't typically explicitly declared and managed by a central controller: communications between middleware application logic, a database and a front-end Web server are often only defined on a system analyst's white board and in the code itself, not a prescriptive centralized configuration. Such ambiguity makes it inherently harder to define the desired state of a multi-tier application and identify deviant behavior that could be the result of malware or other security threats.

Thus, an important first step to the AppDefense process is what company execs call 'learning the state of known good', rather than chasing observed bad actions.

AppDefense phases and features

AppDefense consists of three elements that:

1. Capture the configurations and intended state, dependencies and communications of applications in a VM environment. Although the VM management console knows about the operating environment and system-level requirements of various application images, AppDefense needs help from automation software like Ansible, Chef and Puppet to know each application's collection of software packages and internal configuration to build a manifest detailing the intended operating environment that is stored in a secure enclave within the ESX host that is inaccessible to guest VMs. But AppDefense also must understand the normal behavior of each application, which it learns by recording system interactions during a controlled training phase that feeds machine learning algorithms that produce a predictive model of baseline behavior.
2. Once learned, AppDefense moves into detect mode in which, like other security software, it monitors traffic and application behavior looking for deviations from the desired good state. Unlike traditional endpoint protection software, AppDefense has the advantage of operating within the hypervisor itself, not just as an agent within the OS. As Tom Corn, VMware's SVP of security products told me during a Q&A session, AppDefense can detect, but not block, anomalous behavior within the guest VMs using an OS kernel driver. Furthermore, the ESX hypervisor also locks memory pages of the driver itself to thwart attempts to defeat it.
3. Once an attack has been identified, as defined by a deviation from the intended good state, AppDefense kicks into respond mode in which it can apply a host of automated defenses at both the VM and, optionally should NSX also be in use, network levels. For example, the VM management console can suspend, shutdown, redeploy golden images or snapshot the current state of likely rogue application instances for future forensic analysis. Similarly, NSX can quarantine VMs to limit the spread of malware. AppDefense can also trigger other security products to help with threat response and mitigation. Initial software partners are Carbon Black (Collective Defense Cloud), (IBM (QRadar), RSA (NetWitness Suite) and SecureWorks (Cloud Guardian).

As Corn summarized in talking points repeated throughout VMworld appearances,

AppDefense delivers an intent-based security model that focuses on what the applications should do - the known good - rather than what the attackers do - the known bad. We believe it will do for compute, what VMware NSX™ and micro-segmentation did for the network; enable least privilege environments for critical applications.

Other virtualization-enabled security technique

Virtualization and application sandboxes (isolated runtime containers enforced by the OS) are routinely used to secure client systems.

For example, iOS restricts applications and their data to a restricted sandbox directory and are prohibited from accessing system resources except via public system interfaces, a security design that is a key reason why iOS devices are inherently resistant to malware and external attacks.

Client-side virtualization or containerization is deemed so effective a security measure that the NSA now recommends government agencies and business buy smartphones compliant with its Commercial Solutions for Classified Program (CSfC) standards that include iOS 9 and later, Blackberry, Samsung Knox and other platforms.

Bromium, founded by Simon Crosby, developer of the original Xen hypervisor, is the company best known for using virtualization to protect Windows PCs. Its scheme, using so-called micro-VMs that encapsulate and isolate each user-space application or process from the host OS, has proven impervious to attack and is popular with three-letter agencies, financial institutions and other security conscious businesses. Crosby told me that Bromium uses a technique similar to AppDefense's kernel driver to hook into the Windows OS. In fact, the company recently applauded VMware on validating the benefits of virtualization to security writing on its blog,

It’s increasingly clear that virtualization is becoming the new security superstar. Our recent experience at VMworld was wonderful proof as the virtualization community immediately realized the value of the Bromium approach. Bromium uses unique, multi-patented hardware-enforced isolation and least-privilege restrictions on all tasks running within micro-virtualized environments to create high-fidelity, low-exposure endpoints without relying on detection to prevent threats from the kernel on up. Unlike detection, which necessitates a never-ending and unwinnable race between attackers and defenders, virtualization-based security is sustainable for the long term.

My take

I have long believed that system level security by design, whether by OS-enforced app sandboxes or VM host-level introspection and threat mitigation, is the most effective means of endpoint protection.

The industry's sorry collective experience with an escalating set of threats, patches and detection measures has demonstrated the futility of post hoc security measures.

VMware is in a unique position to deliver such system-level security to enterprise applications given its position as the VM platform of choice for so many organizations.

Following on the growing success of NSX as a network security product, AppDefense is a promising extension into the realm of applications and the hypervisor.

Sadly, at VMworld the company provided no details on AppDefense's internal design and implementation, nor were pilot customers available for questioning or comments on implementation experiences. Thus, it's not yet possible to assess how effective AppDefense will be at preventing or limiting security breaches in an actual enterprise setting. These details will no doubt come and show both the product's benefits and limitations. Still, I believe that embedding security within VM runtime and management platform will prove much more effective than the whack-a-mole tactics used to date.