Using Open Policy Agent With Kubermatic Kubernetes Platform

Kubernetes October 16, 2020

This article shows you how to use Open Policy Agent for policy making on a Kubernetes cluster managed by Kubermatic Kubernetes Platform (KKP).

To use Open Policy Agent with Kubernetes, you have two options.

You can use it as an admission controller with kube-mgmt: visit this extensive tutorial to see how to do that.

We recommend using the more modern option with the policy controller Gatekeeper. Among other reasons, Gatekeeper provides an audit functionality that periodically checks all resources for violation; this is a function you have to implement yourself for the kube-mgmt option.

What is Gatekeeper?

Every time you create, update or delete a Kubernetes resource, you have the option to execute an admission controller webhook. That webhook is a HTTP callback that allows you to modify objects (mutating admission webhooks) or reject admission requests (validating admission webhooks).

Gatekeeper is that type of webhook, but at the moment only a validating one, with a mutating webhook in the works.

Installing Gatekeeper with Kubermatic Kubernetes Platform

The easiest way to integrate Gatekeeper with KKP is to install a prebuilt image from the Gatekeeper server.

Create a new KKP cluster, download the kubeconfig and connect to the cluster. See our tutorials for details on that.

Execute:

$ kubectl apply -f https://raw.githubusercontent.com/open-policy-agent/gatekeeper/master/deploy/gatekeeper.yaml

You should see the following output:

namespace/gatekeeper-system created
customresourcedefinition.apiextensions.k8s.io/configs.config.gatekeeper.sh created
customresourcedefinition.apiextensions.k8s.io/constraintpodstatuses.status.gatekeeper.sh created
customresourcedefinition.apiextensions.k8s.io/constrainttemplatepodstatuses.status.gatekeeper.sh created
customresourcedefinition.apiextensions.k8s.io/constrainttemplates.templates.gatekeeper.sh created
serviceaccount/gatekeeper-admin created
role.rbac.authorization.k8s.io/gatekeeper-manager-role created
clusterrole.rbac.authorization.k8s.io/gatekeeper-manager-role created
rolebinding.rbac.authorization.k8s.io/gatekeeper-manager-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/gatekeeper-manager-rolebinding created
secret/gatekeeper-webhook-server-cert created
service/gatekeeper-webhook-service created
deployment.apps/gatekeeper-audit created
deployment.apps/gatekeeper-controller-manager created
validatingwebhookconfiguration.admissionregistration.k8s.io/gatekeeper-validating-webhook-configuration created

If you prefer to install from a Helm template, execute:

$ helm repo add gatekeeper https://raw.githubusercontent.com/open-policy-agent/gatekeeper/master/charts/gatekeeper
$ helm install gatekeeper/gatekeeper

Using Gatekeeper

You will need to define two resources:

Using the examples from the Gatekeeper docs, create a ConstraintTemplate first:

apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
  name: k8srequiredlabels
spec:
  crd:
    spec:
      names:
        kind: K8sRequiredLabels
        listKind: K8sRequiredLabelsList
        plural: k8srequiredlabels
        singular: k8srequiredlabels
      validation:
        # Schema for the `parameters` field
        openAPIV3Schema:
          properties:
            labels:
              type: array
              items: string
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8srequiredlabels

        violation[{"msg": msg, "details": {"missing_labels": missing}}] {
          provided := {label | input.review.object.metadata.labels[label]}
          required := {label | label := input.parameters.labels[_]}
          missing := required - provided
          count(missing) > 0
          msg := sprintf("you must provide labels: %v", [missing])
        }

And then a Constraint:

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredLabels
metadata:
  name: ns-must-have-gk
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Namespace"]
  parameters:
    labels: ["gatekeeper"]

This will install a ConstraintTemplate in your cluster which stipulates that the object must provide all labels specified by the template, as well as a Constraint which specifies that the policy must be enforced in all namespaces. The match field allows you to indicate to which resources a constraint will be applied. For example, you can select resources with certain apiGroups or kinds and specify a list of white-listed or black-listed namespaces.

More Examples

Visit these repos to see more examples for ConstraintTemplates and Constraints.

Audit

Now that you have defined a policy, you can reject resources which violate it.

Resources which violate this policy might have existed in the cluster before you defined the policy. In those cases, Gatekeeper brings an audit functionality which allows you to discover those resources and enforce the policy on them as well. Results are stored in the “violations” field of the Constraint object.

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredLabels
# ...
status:
  auditTimestamp: "2019-05-11T01:46:13Z"
  enforced: true
  violations:
  - enforcementAction: deny
    kind: Namespace
    message: 'you must provide labels: {"gatekeeper"}'
    name: default
  - enforcementAction: deny
    kind: Namespace
    message: 'you must provide labels: {"gatekeeper"}'
    name: gatekeeper-system
  - enforcementAction: deny
    kind: Namespace
    message: 'you must provide labels: {"gatekeeper"}'
    name: kube-public
  - enforcementAction: deny
    kind: Namespace
    message: 'you must provide labels: {"gatekeeper"}'
    name: kube-system

You can adjust how often Gatekeeper audits resources during installation. To do that, you have to edit the prebuilt yaml file and add a parameter --audit-interval to the gatekeeper-audit resource, set to any number of seconds, or to --audit-interval=0 to disable audit. By default, the audit runs once every 60 seconds.

Debugging

By default, Gatekeeper runs in its least verbose mode, INFO. You can set it to DEBUG by passing the argument --log-level=DEBUG during installation.

Uninstalling Gatekeeper

If you installed Gatekeeper from the prebuilt image, just run kubectl delete on that image:

$ kubectl delete -f https://raw.githubusercontent.com/open-policy-agent/gatekeeper/master/deploy/gatekeeper.yaml

If you installed Gatekeeper using Helm:

$ helm delete <release name> --purge

Additional Information

Irina Lindt

Irina Lindt

Software Engineer