This article shows you how to use Open Policy Agent for policy making on a Kubernetes cluster managed by Kubermatic Kubernetes Platform (KKP).
To use Open Policy Agent with Kubernetes, you have two options.
You can use it as an admission controller with kube-mgmt: visit this extensive tutorial to see how to do that.
We recommend using the more modern option with the policy controller Gatekeeper. Among other reasons, Gatekeeper provides an audit functionality that periodically checks all resources for violation; this is a function you have to implement yourself for the kube-mgmt option.
What is Gatekeeper?
Every time you create, update or delete a Kubernetes resource, you have the option to execute an admission controller webhook. That webhook is a HTTP callback that allows you to modify objects (mutating admission webhooks) or reject admission requests (validating admission webhooks).
Gatekeeper is that type of webhook, but at the moment only a validating one, with a mutating webhook in the works.
Installing Gatekeeper with Kubermatic Kubernetes Platform
The easiest way to integrate Gatekeeper with KKP is to install a prebuilt image from the Gatekeeper server.
Create a new KKP cluster, download the kubeconfig and connect to the cluster. See our tutorials for details on that.
Execute:
$ kubectl apply -f https://raw.githubusercontent.com/open-policy-agent/gatekeeper/master/deploy/gatekeeper.yaml
You should see the following output:
namespace/gatekeeper-system created
customresourcedefinition.apiextensions.k8s.io/configs.config.gatekeeper.sh created
customresourcedefinition.apiextensions.k8s.io/constraintpodstatuses.status.gatekeeper.sh created
customresourcedefinition.apiextensions.k8s.io/constrainttemplatepodstatuses.status.gatekeeper.sh created
customresourcedefinition.apiextensions.k8s.io/constrainttemplates.templates.gatekeeper.sh created
serviceaccount/gatekeeper-admin created
role.rbac.authorization.k8s.io/gatekeeper-manager-role created
clusterrole.rbac.authorization.k8s.io/gatekeeper-manager-role created
rolebinding.rbac.authorization.k8s.io/gatekeeper-manager-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/gatekeeper-manager-rolebinding created
secret/gatekeeper-webhook-server-cert created
service/gatekeeper-webhook-service created
deployment.apps/gatekeeper-audit created
deployment.apps/gatekeeper-controller-manager created
validatingwebhookconfiguration.admissionregistration.k8s.io/gatekeeper-validating-webhook-configuration created
If you prefer to install from a Helm template, execute:
$ helm repo add gatekeeper https://raw.githubusercontent.com/open-policy-agent/gatekeeper/master/charts/gatekeeper
$ helm install gatekeeper/gatekeeper
Using Gatekeeper
You will need to define two resources:
- A ConstraintTemplate that will use the Rego language to describe the policy you want to enforce
- A Constraint that will tell Gatekeeper where and how to use the ConstraintTemplate.
Using the examples from the Gatekeeper docs, create a ConstraintTemplate first:
apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
name: k8srequiredlabels
spec:
crd:
spec:
names:
kind: K8sRequiredLabels
listKind: K8sRequiredLabelsList
plural: k8srequiredlabels
singular: k8srequiredlabels
validation:
# Schema for the `parameters` field
openAPIV3Schema:
properties:
labels:
type: array
items: string
targets:
- target: admission.k8s.gatekeeper.sh
rego: |
package k8srequiredlabels
violation[{"msg": msg, "details": {"missing_labels": missing}}] {
provided := {label | input.review.object.metadata.labels[label]}
required := {label | label := input.parameters.labels[_]}
missing := required - provided
count(missing) > 0
msg := sprintf("you must provide labels: %v", [missing])
}
And then a Constraint:
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredLabels
metadata:
name: ns-must-have-gk
spec:
match:
kinds:
- apiGroups: [""]
kinds: ["Namespace"]
parameters:
labels: ["gatekeeper"]
This will install a ConstraintTemplate in your cluster which stipulates that the object must provide all labels specified by the template, as well as a Constraint which specifies that the policy must be enforced in all namespaces. The match field allows you to indicate to which resources a constraint will be applied. For example, you can select resources with certain apiGroups or kinds and specify a list of white-listed or black-listed namespaces.
More Examples
Visit these repos to see more examples for ConstraintTemplates and Constraints.
Audit
Now that you have defined a policy, you can reject resources which violate it.
Resources which violate this policy might have existed in the cluster before you defined the policy. In those cases, Gatekeeper brings an audit functionality which allows you to discover those resources and enforce the policy on them as well. Results are stored in the “violations” field of the Constraint object.
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredLabels
# ...
status:
auditTimestamp: "2019-05-11T01:46:13Z"
enforced: true
violations:
- enforcementAction: deny
kind: Namespace
message: 'you must provide labels: {"gatekeeper"}'
name: default
- enforcementAction: deny
kind: Namespace
message: 'you must provide labels: {"gatekeeper"}'
name: gatekeeper-system
- enforcementAction: deny
kind: Namespace
message: 'you must provide labels: {"gatekeeper"}'
name: kube-public
- enforcementAction: deny
kind: Namespace
message: 'you must provide labels: {"gatekeeper"}'
name: kube-system
You can adjust how often Gatekeeper audits resources during installation. To do that, you have to edit the prebuilt yaml file and add a parameter --audit-interval to the gatekeeper-audit resource, set to any number of seconds, or to --audit-interval=0 to disable audit. By default, the audit runs once every 60 seconds.
Debugging
By default, Gatekeeper runs in its least verbose mode, INFO. You can set it to DEBUG by passing the argument --log-level=DEBUG during installation.
Uninstalling Gatekeeper
If you installed Gatekeeper from the prebuilt image, just run kubectl delete on that image:
$ kubectl delete -f https://raw.githubusercontent.com/open-policy-agent/gatekeeper/master/deploy/gatekeeper.yaml
If you installed Gatekeeper using Helm:
$ helm delete <release name> --purge
Additional Information
- For more information on Gatekeeper, check out Github.
- To ask Gatekeeper-specific questions, join the #kubernetes-policy channel in the OPA Slack.
- To report vulnerabilities, email them to open-policy-agent-security.
