Using OPA Gatekeeper to enforce policies in Kubernetes

Here's a scenario. You have two teams working on the same Kubernetes cluster; each team has their own namespace, and private image repositories. Each team shouldn't have access to the other teams resources.

When team A deploys their application, the container image is pulled by Kubernetes node where the container was scheduled. Team B can now access the locally cached container image if they know the image name, and have their image pull policy set to ifNotPresent.

I needed a way prevent team B from accessing the locally cached image, so I used OPA Gatekeeper to do this. OPA Gatekeeper allows you to create custom defined policies that allow you to enforce what can and cannot be done within your cluster. In this instance, I want to create a policy that disallows imagePullPolicy to be set to Always. This will ensure that the namespace must contain the appropriate imagePullSecret to the private image registry.

Here's what the policy looks like - it's defined using the ConstraintTemplate resource and the policy is written in the rego language.

apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
  name: k8simagepullpolicy
  annotations:
    metadata.gatekeeper.sh/title: "Enforce imagePullPolicy Always"
    metadata.gatekeeper.sh/version: 1.1.0
    description: Enforce imagePullPolicy Always for namespaces matching specified prefixes
spec:
  crd:
    spec:
      names:
        kind: k8simagePullPolicy
      validation:
        openAPIV3Schema:
          type: object
          properties:
            namespacePrefix:
              description: A list of namespace prefixes where this policy should apply
              type: array
              items:
                type: string
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8simagepullpolicy

        violation[{"msg": msg}] {
          container := input.review.object.spec.containers[_]
          namespace_matches
          not container.imagePullPolicy == "Always"

          msg := sprintf("Container '%s' must have imagePullPolicy set to 'Always', found: '%s'", [container.name, get_pull_policy(container)])
        }

        violation[{"msg": msg}] {
          container := input.review.object.spec.initContainers[_]
          namespace_matches
          not container.imagePullPolicy == "Always"

          msg := sprintf("Init container '%s' must have imagePullPolicy set to 'Always', found: '%s'", [container.name, get_pull_policy(container)])
        }

        violation[{"msg": msg}] {
          container := input.review.object.spec.ephemeralContainers[_]
          namespace_matches
          not container.imagePullPolicy == "Always"

          msg := sprintf("Ephemeral container '%s' must have imagePullPolicy set to 'Always', found: '%s'", [container.name, get_pull_policy(container)])
        }

        namespace_matches {
          namespace := input.review.object.metadata.namespace
          prefix := input.parameters.namespacePrefix[_]
          startswith(namespace, prefix)
        }

        get_pull_policy(container) = policy {
          policy := container.imagePullPolicy
        } else = "not set" {
          true
        }

The policy has three violation rules with the same logic, but target different containers (containers, initContainers, ephemeralContainers). If all three of the conditions are met within the violation rule, the violation rule will be triggered, and the action defined in enforcementAction will be taken.

The policy is enforced using the k8simagePullPolicy resource.

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: k8simagePullPolicy
metadata:
  name: enforce-imagepullpolicy-for-prod
spec:
  enforcementAction: deny
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Pod"]
  parameters:
    namespacePrefix:
      - "t-"

enforcementAction

Set enforcementAction to one of the following values to decide which action you would like to take, when a policy matches.

• dryrun
• warn
• deny