Google services are built with containers; now it’s sharing expertise on securing them

Containers have taken IT by storm as both infrastructure operators and application designers understand their many benefits over traditional VMs as a runtime environment. As Google Cloud Anthos demonstrates, containers can be used as a lift-and-shift environment to migrate legacy monolithic applications to cloud infrastructure, however the full power of containers is only realized when they are used as a vehicle for reusable code modules, aka application services, that are combined into modular applications. Known as a microservice architecture, the technique is so flexible, efficient and scalable that it has become the canonical cloud-native design pattern. Indeed, the terms “cloud-native” and “microservices architecture” are so often conflated that they have become synonymous.

The major problem with any such architectural paradigm shift Is a host of unintended consequences. Unfortunately, these often manifest themselves in unanticipated uses, inadvertently exposed system and application interfaces and accidental back doors into other systems, i.e. just the sort of security gaps and blind spots that deviously creative hackers can exploit for fun (lulz), vandalism or theft. As the AWS-Capital One incident demonstrated, which I detailed here, protecting monolithic cloud-hosted applications running as conventional VMs is hard enough, but the complexity is multiplied when applications are disaggregated into microservice components communicating via REST APIs and spread over one or more container clusters. Indeed, the security challenge is further magnified if third parties, outside the confines of a tightly controlled enterprise network, provide some of the services.

Thanks to Google, enterprise architects and security engineers can avoid learning the basics of container security the hard way, via mistakes and security breaches resulting in downtime and data loss. Instead, they can take a lesson from a pioneer of container technology and usage, Google, which has long used them as the runtime platform for the hundreds of applications and components powering its search engine, applications and infrastructure services. Google recently shared its model for cloud-native security via a whitepaper, BeyondProd: How Google moved from perimeter-based to cloud-native security, that details the design requirements and implementation for a hypothetical company using containers as its application runtime environment.

From BeyondCorp to BeyondProd

Google has a habit of explaining its security philosophy and implementation using an idealized organization that is modeled after Google’s internal architecture as the vehicle for a generalized security architecture. In BeyondCorp, the cloud giant explained a decentralized network security model that replaces hard and fast network boundaries enforced by firewall rules with a so-called zero-trust design that:

• Shifts network protection to the edge
• Replaces implicit mutual trust between systems with authenticated access for each transaction
• Replaces centralized network control and choke points with a distributed system of authenticated and encrypted connections between systems and users.

Originally called a deperimeterized network security model, Google evolved BeyondCorp into the basis for its service and user security design and processes. For both Google and the original advocates of deperimeterized security, the motivation is the same, which Google puts this way:

The perimeter is no longer just the physical location of the enterprise [data center], and what lies inside the perimeter is no longer a blessed and safe place to host personal computing devices and enterprise applications [microservices].

Google’s implementation has become familiar to anyone that uses its services, but particularly for those in organizations using Google’s enterprise services to centrally manage GSuite and mobile device security policies. For example, Google uses secure TLS (HTTPS) connections for all applications and requires two-factor authentication to an externally-facing network proxy when employees attempt to use applications from an untrusted external network like a coffee shop’s WI-FI. The proxy also certifies the authenticity of the user’s phone or laptop via a previously installed device certificate that is typically installed in a tamper-free hardware trusted platform module (TPM). Thus, access to applications and data depends on a combination of device and user credentials regardless of the network. Access from an internal corporate network or hotel Wi-Fi requires the same level of authentication.

Whereas BeyondCorp describes a regime of end-to-end user-to-machine security,  BeyondProd extends it into the realm of machine-to-machine communications in the form of microservice interactions.

End-to-end security in a containerized world

Google follows several principles in applying the BeyondCorp security model to container-based cloud-native applications.

1. Network perimeters should still be as secure as possible to eliminate or mitigate many attacks and block unauthorized traffic. Thus, securing the network edge remains necessary and beneficial, but it’s not sufficient.
2. There is no mutual trust between container services and all accesses are authenticated. Thus, when an individual container or service is compromised, an attacker can’t easily migrate to other services and systems in the same cluster or data center.
3. Container clusters use trusted machines with a local certificate and only accept authorized code from an authorized and authenticated user and trusted repository.
4. Infrastructure and code changes follow defined, automated and logged processes to improve reliability and aid troubleshooting.
5. Workloads sharing an OS are isolated on a shared VM using network controls and sandboxing technologies.

These principals are used to address the following cloud-native security requirements:

• Network controls at the perimeter, i.e. firewalls, are supplemented with zero-trust authenticated communications between services running as containers.
• Security controls based on fixed IP addresses are replaced with trusted machines with local TPM (trusted platform modules, a tamperproof hardware processor and memory) certificates and code from known and authenticated sources.
• Network (IP-based) controls are replaced with service-based identity and service-to-service authentication, which allows services to run anywhere in the environment as long as it meets the previous requirements (systems with trusted, authenticated certificates).
• Security policies are consistently applied and centrally managed across the environment (data center or cloud infrastructure).
• Service code deployments are standardized, authenticated and automated.

GCP provides several features and services that facilitate the implementation of a zero-trust microservice model such as an API gateway (Envoy, Traffic Director) for managing TLS termination and traffic, service mesh (Istio) for service-to-service authentication and encryption, Kubernetes admission controllers (Kritis) to enforce code deployment controls and secured nodes and a container sandbox in the GKE Kubernetes service. AWS and Azure container services offer many of the same features, with others available as open source code deployed separately.

Security remains a top priority and challenge

News of the latest data breach, ransomware attack or company plan to increase security spending is a daily occurrence. Containers are the next frontier in the cyber security battles, but one where the attackers have the advantage in technology depth and experience over most enterprises. Recent surveys indicate that IT leaders and app developers recognize the security problems containerized microservices present. Thus, Google’s BeyondProd architectural guidelines are both timely and necessary.

For example, a survey of 200 “senior technology decision makers in the US ” (titles like CIO, CTO, IT director/VP) in organizations with at least 1,000 employees sponsored by Kong, which make API management software targeting microservices, found that security was cited as a top priority by 40%, trailing by single digits only “improving efficiency” and “reducing cost.” Security was also named a chief attribute for new applications, in a virtual tie with availability. Unfortunately, it was also identified as the biggest challenge to adopting microservices, i.e. a cloud-native application design. An earlier survey on container adoption sponsored by Portworx, a provider of storage software for Kubernetes environments, found similar security concerns. When asked to cite the challenges when deploying containers, 51% cited security as one of their top-three, 11 points higher than any other issue.

My take

Containers are rapidly displacing VMs as the platform of choice for running enterprise applications. Still, once organizations get a taste of using them for legacy systems, it doesn’t take long before they are developing next-generation cloud-native applications that fully exploit the many benefits of containers and Kubernetes. However, by adopting a new model for designing, developing and deploying code, IT organizations embarking down the microservices path inevitably face unfamiliar terrain in the form of new infrastructure, management systems and security exposures. Getting the last one right is critically important given the costly and embarrassing ramifications of security mistakes.

In these early days of microservice adoption there is plenty of time for organizations to comprehensively address the challenges of microservice security by heeding Google’s design principles and technical advice. Indeed, as I discussed in my column Kubernetes is evolving into an enterprise-friendly platform, but challenges remain, organizations are hungry for solutions to the most pressing problems container-based applications and environments present, principally security, development tools and deployment automation.

The moat-and-castle approach to security, i.e. defending network and organizational perimeters, is dead. Google is leading the way to more robust enterprise security postures using a zero-trust model of end-to-end encryption, strong authentication and trusted hardware that is based on mature certificate and identity management standards. With BeyondProd, Google has extended the model to microservices, i.e. machine-to-machine transactions and communications, in a way the compliments its existing zero-trust system of user and data security. Any organization building microservice-based systems should internalize the concepts Google lays out and aggressively incorporate them from the outset, before letting sloppy security practices take root.