Container Security Maturity Model

Container Security Maturity Model2

Container Security Maturity ModelGetting security right at every stage of your containerization journey

As companies embrace containerized, cloud-native applications, they recognize that the need for security is as paramount as ever but struggle to secure these new technologies. Since everyone is learning the new stack, no one has a roadmap for how to apply security across the various stages of the containerization journey. Each stage introduces novel security challenges, and organizations must learn both the infrastructure and the security at the same time. Understandably, security needs evolve as companies move from developing their first containerized application to doing all new development in containers and managing thousands of microservices. The following maturity model will help organizations understand and successfully meet the security challenges that go along with adopting and expanding the use of containerized applications.

Container Security Maturity Model3

Maturity Phase

Application Scope:individual experimentation

Orchestration:none

Security focus:secure coding practices

Security tools:no dedicated tools

Application Scope:containerize an existing app

carve out a part of an existing application

create a new app

Orchestration:Kubernetes is recommended

Security focus:governance and policies

image scanning

vulnerability management

configuration management

Kubernetes-specific security

Security tools:container or Kubernetes security tooling

Application Scope:teams developing some apps in containers

some lift-and-shift to containerize some key legacy apps

Orchestration:Kubernetes is critical

Security focus:network segmentation

expanded vulnerability and config management

compliance

runtime security

Security tools:Kubernetes-native security

Application Scope:all new apps developed in containers

Orchestration:Kubernetes is critical

Security focus:automation

deep visibility into Kubernetes

service isolation

complexity management

Security tools:Kubernetes-native security

Application Scope:as many applications as possible are containerized

Orchestration:Kubernetes plus additional layers such as service mesh

Security focus:awareness of new threat vectors

additional infrastructure layers

Security tools:Kubernetes-native security

Organization-wide Adoption

STAGE 5

ExpansionSTAGE 4

Single App in Production

STAGE 3

OfficialProject

STAGE 2

Individual Initiatives

STAGE 1

Container Security Maturity Model4

Introduction: What’s at StakeThe consequences of a security incident are the same regardless of whether an application is containerized, running on virtual machines, or a legacy system. Security breaches can mean hefty fines, loss of customers’ trust, millions of dollars in lost revenue, and lost time and money spent to urgently patch the security hole. And no amount of after-the-fact security patches can retrieve sensitive data. We know security incidents are happening in containerized apps: 94% of respondents to our 2020 State of Container and Kubernetes Security survey had a container security incident in the past 12 months.

In the past 12 months, what security incidents or issues related to containers and/or Kubernetes have you experienced? (pick all that apply)

Detected misconfiguration

Security incident during runtime

Major vulnerability to remediate

None

69% 27% 24% 6%

Container Security Maturity Model5

Along with adopting containers, many organizations are also adopting DevOps principles and practices, and this shift is amplifying the changes to security. DevOps teams now share responsibility for security with central security specialists, and everyone is looking at how to “shift left” and apply security practices earlier in the software development life cycle. And all sides face a learning curve - DevOps is learning about security, and security teams are learning about containers and Kubernetes. Recent research from Microsoft detailing the Kubernetes Attack Matrix highlights the broad set of attacks organizations must defend against.

So the stakes are as high as ever, and companies are less prepared, given the steep learning curves. In many cases, companies simply don’t know what they don’t know — they aren’t fully aware of the security challenges particular to a containerized architecture, let alone how to proactively address them.

One common reaction is to simply try to apply the same security tactics that have always been used, but these new application architectures and development workflows require a different approach. So companies often lag one step behind in security as they move further along in their container maturity. We’ve developed this container security maturity model to help companies get ahead on security, so they can put in place the tools and practices essential to meeting not just the needs of their current stage but to continue serving them effectively as they progress on their container journey.

Stage 1: Individual initiatives

The first step on the containerization journey - like most infrastructure changes - is almost always an individual one. It involves developers learning to use containers on their individual machines, for example, for projects that may never go into production. At this stage, the applications are typically fairly basic and generally don’t involve collaboration between colleagues. Often the developer is simply looking to learn the new technology.

The Container Security Maturity Model

Individual Initiatives

Container Security Maturity Model6

At the individual initiative stage, developers might start experimenting with Kubernetes, depending on the scope of what they are trying to build.

This first step can last for months. While application developers can generally get up to speed with containers relatively quickly, moving from a side project to an officially sanctioned project that is prepared to meet the organization’s operational requirements for applications in production requires a big jump. This first stage does not require organizational buy-in, but moving to the next step requires having an officially sanctioned project to work on.

Security at Stage 1

When it’s just individual developers learning and testing out containers, there’s no critical need for dedicated security tools or any change in security practices, as long as the project isn’t destined for production. As always, developers should use secure coding practices, but other aspects of security are not as critical in Stage 1.

Nonetheless, this stage is a good time to start working towards a security-as-code mindset, in which security is declarative and automated and is a part of the development workflow rather than addressed after the app is built.

The Security Challenge

At this stage, security isn’t critical. However, if the organization intends to expand the containerization project beyond individuals playing around, it needs to ensure that everyone understands that the stakes and complexity related to security rise quickly.

Most companies get security right at this stage — mostly because there’s not much to get wrong. But the jump in security challenges between Stage 1 and Stage 2 is massive and catches many organizations off guard.

Container Security Maturity Model7

Stage 2: Officially sanctioned project

At this stage, containerization has become not just a side-project for individual developers but part of an official project, managed by a team or by a business unit. The group is creating an application (or containerizing an existing app) that is destined for production but is still in development. Most organizations follow one of three routes when developing their first containerized application: • Containerize an existing application• Carve out a part of an existing application to containerize• Develop a new applicaiton from scratch in containers

Regardless of which route the team takes, this step is essentially a proof-of-concept, to learn the benefits of cloud-native technology and to see how it works in a single application.

Image registries become a necessity once multiple individuals need to collaborate. As teams start to use image registries, they must think of protecting against any place where a vulnerability could be introduced into the application. Using a private image registry allows organizations to enforce policies around who can access images or repositories and to ensure that images come from a trusted source. Image registries can also provide vulnerability scanning. However, a compromised image in a registry is a common way attackers can get initial access to the application.

Most organizations use Kubernetes for orchestration as they prepare to deploy a containerized application in production. While this abstraction layer makes applications more resilient and easier to scale, Kubernetes adds complexity, introduces its own threat vectors, and has a steep learning curve for both application and platform developers. When containers run in Kubernetes, they’re packaged in replication units called pods. Pods can contain one or more containers, but all containers in the same pod will share the same resources and local network, so putting containers into separate pods is a best practice. Kubernetes can introduce vulnerabilities itself, and Kubernetes’ configurations greatly impact the overall security posture of the organization. Even more concerning, Kubernetes’ default configurations are not secure, so ignoring them will create massive security risk. The Kubeconfig file is another common way attackers can get initial access to the application. Break into Kubernetes, and you’ve gotten the keys to the kingdom.

OfficialProject

Container Security Maturity Model8

Security at Stage 2

Stage 2 on the maturity journey is where a gap often starts to develop between what companies are doing and what they should be doing. The root cause of this gap is a knowledge or skills gap. Often no one, from individual developers to security specialists who are used to legacy monolithic apps, fully understands all the potential security issues that can arise in containers and Kubernetes.

The second core problem at Stage 2 is a failure to plan for the future. Once organizations build internal expertise around containers and Kubernetes, they often move through the rest of the maturity stages quickly. One common mistake is to consider only those tools and security practices needed at the scope of this current stage rather than proactively planning for the broader set of functionality the organization will need as it becomes more mature and expands the containerization project to additional applications.

Companies must account for the following security considerations at this stage:

Kubernetes-specific security. Having security tools in place that are purpose-built for Kubernetes and understand Kubernetes’ worldview is essential to securing applications starting at this stage. Only a Kubernetes-specific solution will be able to provide native security for both containers and Kubernetes, looking at vulnerabilities and configurations of Kubernetes itself, along with those of containers.

Create organizational security policies and governance. Container security is not just a matter of tooling. While container security tools will help protect applications, organizations must also put in place policies that apply best practices for managing security across the container life cycle. Most companies will already have a set of security policies in place to govern secure application development and operations practices and will need to revise them to suit the needs of containerized applications to take into account the specific challenges of complex, distributed containerized applications.

Container Security Maturity Model9

Set up vulnerability management. As organizations work on a proof-of-concept application that is destined for production, they must start thinking about protecting against vulnerabilities that could be introduced into the environment. During the development process, the most important component of vulnerability management is image scanning. Some image registries, especially those hosted by public cloud providers, provide built-in image scanning. Standalone image scanning tools are also available. Different scanning tools vary in the granularity they provide, and companies shouldn’t assume that all image scanning capabilities are the same. Some might identify OS vulnerabilities and language-specific vulnerabilities but not be able to scan per layer or highlight packages that contain certain open source license types, for example.

Manage configurations. During this stage, DevOps and security teams need to pay attention to at least basic configuration management best practices, ideally based on the CIS Benchmarks developed by the Center for Internet Security. These configuration guidelines, for both Docker and Kubernetes, provide a good starting point for companies looking for best practices related to security. At this stage, teams will likely still be handling configuration management manually. Configuration management must span both containers and Kubernetes to successfully protect the full cloud-native stack, and tooling that automates configuration checks will improve security posture and reduce operational effort.

The Security Challenge

Managing security in a containerized application is fundamentally different from traditional applications. At this stage, the organization is still learning how to secure cloud-native applications. Most teams will recognize that they need to have an image scanning tool in place. But how often should they run scans? What are all the possible types of vulnerabilities that the image scanner needs to be able to detect? What should you do if you find a vulnerability? Teams often lack the expertise needed to answer those questions, which would inform the security policies. This knowledge gap is as much of a security problem as not having a good image scanner in place. This area shows the value of having out-of-the-box policies available in container and Kubernetes security tools.

Container Security Maturity Model10

The second part of the knowledge gap at this stage is a frequent inability to imagine the types of security challenges that loom as the company moves further along the maturity journey. When applied and managed correctly, security shouldn’t delay the organization’s progress on the container maturity journey. But to enable that rapid development pace, organizations must understand and plan for the security challenges ahead.

Stage 3: Deployment of a single application in production

Congratulations! Your first application is running in production.

Hopefully the organization was thinking ahead at the development stage and set up everything correctly to minimize potential security problems once the application is in production. Developers don’t always fully understand how an application will behave in production and how it will respond to different types of user behavior, and this is especially true of complex, distributed cloud-native applications. At the same time, the security risks increase dramatically as soon as an application is being used in production, especially if it’s an Internet-facing app, because now there’s real-world exposure.

At this stage, Kubernetes is a critical part of any containerized application. Using Kubernetes automates most of the operational tasks related to running containers, and organizations can’t run containerized applications without it at any scale. Kubernetes manages container scheduling, provides load balancing, conducts health checks, manages auto-healing and failovers, and automatically scales. Kubernetes is also a core part of managing the deployment and rollback process, provisioning storage, and configuring network communications between components of the deployments. While Kubernetes is essential at this stage, it adds an additional layer of complexity to the application stack and comes with its own security ramifications.

Single App in Production

Container Security Maturity Model11

Security at Stage 3

Harden the environment. To follow security best practices for both containers and Kubernetes at this stage, teams must: • Keep Kubernetes up to date• Lock down the API server and follow infrastructure best practices • Use role-based access control (RBAC) to restrict access to key components

Meet the application’s compliance needs. The compliance requirements depend on the application’s function as well as your industry, but at this stage you need to be aware of which compliance standards apply and ensure that the application complies with them.

Expand configuration management. At this stage, you’ll need to tighten your configuration management for both containers and Kubernetes.

For containers, you’ll need to do the following:• Ensure containers are configured to not acquire new privileges• Run containers as non-root users• Use a read-only root filesystem in the container• Use a secrets management tool rather than storing secrets in the image• Adjust the container’s privileges to ensure they are as restrictive as possible• Set limits to the container’s memory and CPU that will allow it to function as designed but do no more• Do not use the host network or process space

For Kubernetes, ensure the following:• Isolate resources using namespaces• Limit communications between pods• Use pod security policies to restrict what pods are able to do

Container Security Maturity Model12

Set up network segmentation. The specifics of how to manage network segmentation will depend on the application, but the principle behind network segmentation remains the same: each pod should be allowed to communicate only with the internal or external resources needed to perform its function and should be isolated from all other pods, both inside the same application and in other applications.

Set up runtime security. Once an application is in production, it’s important that you have the ability to detect when the application and/or individual components of the application are acting unusually, which can be a sign of a security incident. Make sure you can easily answer the following questions about your application at any time:• What processes is the container executing?• Do you have enough information to assess the risk for each container and prioritize a fix accordingly? • What does the active network traffic look like? Are any of the running policies overly permissive and allowing too large a blast radius?

The Security Challenge

Once an application goes into production, its risk profile may increase dramatically - its user base has grown, meaningful data could be at risk, and it could be open to access from outside the organization. Organizations will understandably continue to struggle with learning how to manage Kubernetes at scale, from how to manage load balancing to ensuring scaling is working properly. But that learning curve should not mean the organization overlooks or delays applying appropriate security processes. Organizations must simultaneously develop their skills in infrastructure and security management, with strong build and deployment best practices, or their security risk will only increase as the deployment grows.

Stage 4: Expansion

The first app was a success. Now the organization is containerizing more applications or scaling the initial application. These applications are all destined for production. The number of engineers and teams involved increases, and the team that worked on the pilot project can find itself mentoring other teams.

Expansion

Container Security Maturity Model13

Complexity becomes a major problem at this stage. Complexity creates challenges not only for security but also for incident response and compliance. Organizational governance policies are essential at this stage to ensure a standard workflow for developing, testing, deploying, and running applications. These policies need to include security guardrails that ensure security is handled consistently across the organization and to minimize the risk of individual errors.

Security at Stage 4

As organizations expand the footprint of their containerization initiative, the security challenges get exponentially more complex. Teams should be aware of the following areas to ensure security is as tight as possible at this stage.

Automation is essential. As companies deal with multiple clusters, it’s no longer feasible to manage configurations manually or to analyze runtime behaviors by sifting through raw event data. The use of automation is especially important in the context of organizational governance — automation tools are the only way to ensure that organization-wide security policies are consistently implemented.

More complicated compliance requirements. At this stage, it’s important to track which applications or individual microservices are subject to which compliance requirements — and to see if they are not meeting those requirements with on-demand compliance checks.

Service isolation and segmentation. The ability to appropriately isolate and segment traffic between services is crucial as the number of services grow and the compliance and security ecosystem becomes more complex.

The Security Challenge

Complexity increases exponentially as the landscape of containerized applications grows, and most teams have trouble keeping their security protocols up to date. Organizations must adjust governance policies

Container Security Maturity Model14

to reflect the realities of containerized applications as well as ensure that these policies are followed consistently throughout the organization.

Stage 5: Organization-wide adoption of containers

As organizations adopt containers for all new development and potentially also move existing applications into containers, they start to run up against new problems, in both infrastructure and security. At this stage, most organizations will be thinking beyond Kubernetes for orchestrating their services. They’ll be using technologies such as service meshes to manage how services interact with each other and to get visibility into individual services’ behavior and the communications between services.

Optimization and automation are critical at this stage. Organizations should be constantly looking for ways to streamline the development and operations process, simplify the interface with Kubernetes, and automate any process that should be done the same way every time.

Security at Stage 5

As more layers are added to the stack — such as Kubernetes and a service mesh — it becomes impossible to control configurations manually. Components simply have too many knobs to control manually, and automation becomes essential. The out-of-the-box configurations for both containers and Kubernetes are not secure by default, and companies need to rely on automation to ensure security best practices for their creation and configuration are followed consistently. The additional technology layers that come into play at Stage 5 also come with their own set of best practices for configuration, isolation, and vulnerability management.

At this stage, organizations should be well on their way to consistent, organization-wide application of the security-as-code practice, with policies built into the infrastructure, non-compliant assets being identified and rejected immediately and automatically, and application development pace very high.

Organization-wide Adoption

Container Security Maturity Model15

At this most evolved step of the container maturity journey, as all new development is done in containers, a second skills gap often opens up — everyone is stepping into the unknown. As teams integrate one or more service meshes, use more advanced Kubernetes concepts like custom resources and run across multiple environments, novel security challenges emerge. It’s hard for teams to anticipate what vulnerabilities might exist at this level or to know what they don’t know.

The Security Challenge

Every time a new layer or technology is added to the stack, it forces a new learning curve. At the same time, new threat vectors are likely to emerge as applications become increasingly complex. Organizations need to ensure the security tools they have in place can grow with them as they add these new technologies, because visibility and automated configuration management become exponentially more important as the complexity of the application suite increases.

No organization ever achieves a perfect security posture, but it is possible — and essential — to ensure security is as tight as possible, no matter where you are on your container journey. Many of the keys to getting security right aren’t about technology — they’re about structuring the organization and enabling collaboration in a way that values security expertise and encourages developers and security experts to work together. Here’s how.

Look to apply security earlier. Adopting a shift left approach and integrating security into the development process from the very beginning is critical when organizations become fully cloud native, but it’s useful at any stage of the maturity journey. Applying security sooner in the development process not only helps increase the application’s security but also reduces last-minute hiccups in the deployment process that can happen if a security vulnerability is found late in the game.

Get Security Right at Every Step

Container Security Maturity Model16

Collaborate with security. The security team should work collaboratively with the DevOps team and function more like internal consultants than a separate team. This collaboration is important at every stage of the development process. Security teams shouldn’t focus just on identifying what’s broken — instead, they should be internal trainers, building the DevOps teams’ understanding of security best practices. The organization’s goal should be to have security teams that are always catching novel security gaps and vulnerabilities, with previously identified problems no longer getting built or deployed.

Anticipate your future needs. Organizations should not wait until they have an application in production to implement security. Playing a constant game of catch-up will leave the organization exposed while teams evaluate tools and figure out what’s needed. Delaying security will also mean delaying the rollout of the company’s most innovative applications.

Adopt a proactive security stance. The best time to handle security is before a breach. No one hears about the organizations who do security well — they just quietly address security to prevent problems that would make headlines. Proactive security requires both an organizational commitment and the right infrastructure and tooling to support consistent, organization-wide application of security best practices.

Clearly communicate who is ultimately responsible for security. To be effective, security depends on a flexible approach to responsibility. Different types of applications might need different structures, but in every case, someone must be designated as the security lead. Shared responsibility for security can lead to a situation where no one feels responsible — and according to this article, that is exactly what happens when responsibility for security isn’t clearly defined.

Accurately assess the security needs across applications and infrastructure. In a perfect world, every system would have perfect security. The reality is that security needs change based on the details of each individual application. Public-facing applications present a greater risk than internal applications; highly regulated industries need more security than organizations in other sectors. Determine the level of security needed to achieve the risk abatement your organization demands.

Container Security Maturity Model17

About StackRox

The StackRox Kubernetes Security Platform’s Kubernetes-native architecture protects your application across build, deploy, and runtime as you progress on your container journey.

As your environment grows more complex and you depend on more automation, the StackRox platform will enable you to operationalize security in those more sophisticated environments and keep pace with the speed of DevOps.

Kubernetes-native security provides the following crucial benefits:

• Increased protection: The Kubernetes-native approach eliminates blind spots, providing staff withinsights into critical vulnerabilities and threat vectors.

• Reduced time and costs: Kubernetes-native security reduces the time and effort needed to implementsecurity. It also streamlines security analysis, investigation, and remediation using the rich contextKubernetes provides.

• Minimal operational risk: The Kubernetes-native approach taps into the scalability and resiliencynative to Kubernetes, avoiding operational conflict and complexity that can result from out-of-band security controls.

Ready to see StackRox in action?

Get a personalized demo tailored for your business, environment, and needs.