.svg){kind=logo}
Managing and governing Kubernetes clusters at scale can be complex.
You can simplify this complexity with Kubernetes policy engines like Kyverno or Open Policy Agent (OPA).
They empower you to define and enforce policies across your Kubernetes environment.
Combine them with a Kubernetes management platform like Palette, and you can make managing and governing your Kubernetes clusters consistent and scalable.
In this blog, we’ll explore how you can do just that.
Policies: the common ruleset for expected behavior
Before we get into the specifics of engines like Kyverno, we first need to discuss policies and what value they add.
Policies allow us to define the desired or monitored behavior expected of our users and applications. They allow us to put guardrails around user actions to require their actions to be governed by a common ruleset (policy).
In Kubernetes, policies enable us to create a set of rules as code and apply these rules consistently across many environment types.
In the world of cloud-native, Kyverno and OPA are two leading policy engines — for the purposes of this blog, we’ll focus on Kyverno, but here at Spectro we have several customers using OPA too.
What is Kyverno?
Kyverno is an open-source policy engine designed for Kubernetes, created by Nirmata. It is currently a CNCF Incubating project with over 5,000 Github Stars. It allows you to define policy as code in Kubernetes, making managing and controlling your Kubernetes resources easy.
In the past few years, Kyverno has evolved from managing policies within Kubernetes to supporting policies on objects outside of Kubernetes, such as cloud services, infrastructure as code, application authorizations, and many other use cases.
With Kyverno, you can create policies that validate, mutate (change), generate, or delete Kubernetes resources based on predefined rules and conditions. There are hundreds of examples available to get you started.
Kyverno Architecture High-Level
Policy enforcement via admission controller
One benefit of Kyverno is its ability to enforce policies at the admission controller level. The admission controller allows us to intercept requests to the API server after authorization and authentication, but before the object being created is persisted. Policies are therefore evaluated and applied before resources are created or modified.
By catching policy violations early in the process, Kyverno helps maintain your cluster's desired state and prevents misconfigurations from propagating. Policies can be created across any Kubernetes object.
Kyverno policies are written in YAML (JSON) format. You can define rules based on resource kinds, labels, annotations, and other criteria. For example, you could create a policy that ensures all containers have a specific label or a policy that only allows signed images from a specific container registry to be downloaded.
These rules can then trigger actions such as denying the creation of non-compliant resources, modifying resource configurations, or generating additional resources based on templates.
To get started with Kyverno, we can turn to its docs for installation instructions. Once the Helm chart is installed, we can create our first policy. The example below creates a simple policy that adds the label kyverno=cool to all pods.
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: add-labels
spec:
rules:
- name: add-labels
match:
any:
- resources:
kinds:
- Pod
mutate:
patchStrategicMerge:
metadata:
labels:
kyverno: cool
Once we apply this, all of our pods will have this label. While this example is simple, it shows that policies don’t have to be complicated to be useful. By leveraging resource selection, we could apply this only to specific pods, perhaps in a namespace. Our application could then use these labels for additional actions to achieve the desired outcome.
Open Policy Agent (OPA)
The Open Policy Agent, or OPA, is another policy engine. OPA has taken a different approach to policy as it began outside of the Kubernetes ecosystem, and attempts to unify the policy language across the entire cloud-native stack. It does this through a new language called Rego.
Rego extends Datalog, a well-understood decades-old query language, to support structured document models such as JSON. Rego queries can define policies that enumerate data instances that violate the system's expected state.
OPA is more of a general-purpose policy engine for policy requirements across different systems. Rego is a powerful language that focuses more on validating policy than on mutation, although mutation and generation can be achieved by integrating with other systems.
Open Policy Agent Kubernetes Architecture
Kyverno and OPA are both policy engines, but they approach how policy is enforced or applied differently.
Kyverno was written for Kubernetes and tends to have a lower learning curve because it uses Kubernetes native YAML syntax for policy definitions. Policies are managed as CRDs in the Kubernetes cluster and can be used to enforce or validate configuration elements. Kyverno can also support some cross-domain policies using the Kyverno CLI to integrate policies into your pipelines, for example.
Conversely, OPA is a powerful policy engine that can be used outside of Kubernetes domains, as this is where it began.
Scaling policy with Palette
Kyverno and OPA provide a solid foundation for policy management. Now we can add in the Palette platform to deploy these policies consistently at scale.
Palette simplifies the management and governance of Kubernetes clusters across multiple clouds and environments by using Cluster Profiles (you can think of them as blueprints).
Profiles allow you to define reusable sets of Kubernetes configurations and then deploy them consistently across your entire fleet of clusters. You can add all kinds of things to your profiles, including manifests, helm charts, and packs, and further customize through metadata and presets. This flexibility enables you to tailor your profiles to the specific needs of your clusters.
Both Kyverno and OPA can be added to a Palette Managed Kubernetes cluster as a pack.
When you add the Kyverno or OPA packs to your Cluster Profiles, you can streamline the deployment and management of policies across your Kubernetes infrastructure. Instead of spending hours manually applying policies to each cluster, you can create policy packs and include them in your Profile for automatic deployment to any cluster you create.
As we modify our policies over time, we can simply version our Cluster Profiles. Enabling version control allows us to iterate through what our policies should be to achieve our desired outcome.
This ensures that your clusters always comply with your defined policies and significantly reduces the chances of human error, giving you more time to focus on other critical tasks.
Real-world scenarios
Let's look at a few real-world scenarios where the combination of Kyverno and Palette's Cluster Profiles can be powerful:
1. Enforcing Security Best Practices: With Kyverno policies and Palette profiles, you can enforce security best practices across your Kubernetes clusters. For example, you can create policies that ensure all containers run with non-root privileges and restrict the exposure of sensitive ports.
2. Standardizing Resource Configurations: Consistency is essential when managing multiple Kubernetes clusters. With Kyverno and Palette profiles, you can standardize resource configurations across your infrastructure. For instance, you can define policies that enforce specific labels or annotations on resources or require resource limits and requests.
3. Automating Compliance and Governance: Compliance and governance requirements vary across organizations and industries. You can create policies that validate resource configurations against predefined standards, generate audit trails, and block non-compliant deployments.
With Palette and the addition of a monitoring profile, your policy engine can provide reporting on policy usage, consumption, violations, etc.
Conclusion
Kyverno and OPA are powerful policy engines that empower you to define and enforce policies. When used in conjunction with Palette’s Cluster Profiles, you can easily deploy policies across your entire Kubernetes fleet.
You can implement security best practices, standardize resource configurations, automate compliance checks, and streamline the management of your Kubernetes infrastructure. When you build your policies into Cluster Profiles, you can trust that your clusters will remain compliant with your organizational policies, allowing developers to focus on building applications.
If you want to level up your policy game in Kubernetes, consider exploring the capabilities of the Palette platform and our Profiles. Take a look at our Docs, or book a demo to get a quick tour and we’ll set you up to try Palette for yourself.
