Policy Management in Kubernetes

Introduction to Kyverno

Declarative Kubernetes policy in yaml
Kubernetes admission controller and CLI
Validate, mutate, generate or remove any Kubernetes resources
Verify container images and metadata

How It Works

Kyverno runs as a dynamic admission controller in a Kubernetes cluster.
Kyverno receives validating and mutating admission webhook HTTP callbacks from the Kubernetes API server.
Kyverno applies matching policies to return results that enforce admission policies or reject requests.
Policy enforcement is captured using Kubernetes events.
For existing resources, Kyverno creates Policy Reports in the cluster.
The Webhook is the server which handles incoming AdmissionReview requests from the Kubernetes API server and sends them to the Engine for processing.

Enable Kyverno in Kuberise.io

Set enabled: true in app-of-apps values file.

app-of-apps/values-platformName.yaml

ArgocdApplications:
  kyverno:
    enabled: true

Add your Kyverno policy yaml files
- Add your policies in values/defaults/platform/raw/kyverno directories, to apply to all platforms. (Adding policies in templates/kyverno/templates directory will not work as some of them contains similar syntax to Helm templating)
- Add your policies in values/platformName/raw/kyverno directory, to apply to a specific platform.

How to Validate Resources

Create a policy to require labels:

values/platformName/raw/kyverno/required-labels.yaml

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-labels
spec:
  rules:
  - name: check-team
    match:
      any:
      - resources:
          kinds:
          - Pod
          namespaces:
          - default
    skipBackgroundRequests: true
    validate:
      allowExistingViolations: true
      failureAction: Enforce
      message: "label 'team' is required"
      pattern:
        metadata:
          labels:
            team: "?*"

Test the policy:

Terminal

kubectl run nginx --image nginx                            # should fail
kubectl run nginx --image nginx --labels='team=alpha'      # should pass

Check the policy report:

Terminal

kubectl get policyreport -o wide

How to Mutate Resources

Create a policy to add labels:

values/platformName/raw/kyverno/add-labels.yaml

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: add-labels
spec:
  rules:
  - name: add-project
    skipBackgroundRequests: true
    match:
      any:
      - resources:
          kinds:
          - Pod
          namespaces:
          - default
    mutate:
      patchStrategicMerge:
        metadata:
          labels:
            +(project): bravo

Test the policy:

Terminal

kubectl run nginx --image nginx --labels='team=alpha'   # should add label 'project=bravo'

Check the policy report:

Terminal

kubectl get policyreport -o wide

How to Generate Resources

Add these ClusterRoles to enable Kyverno to have access to secrets:

values/platformName/raw/kyverno/clusterrol.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kyverno:secrets:view
  labels:
    rbac.kyverno.io/aggregate-to-admission-controller: "true"
    rbac.kyverno.io/aggregate-to-reports-controller: "true"
    rbac.kyverno.io/aggregate-to-background-controller: "true"
rules:
- apiGroups:
  - ''
  resources:
  - secrets
  verbs:
  - get
  - list
  - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kyverno:secrets:manage
  labels:
    rbac.kyverno.io/aggregate-to-background-controller: "true"
rules:
- apiGroups:
  - ''
  resources:
  - secrets
  verbs:
  - create
  - update
  - delete

Generate an Image Pull Secret

kubectl -n default create secret docker-registry regcred \
  --docker-server=myinternalreg.corp.com \
  --docker-username=john.doe \
  --docker-password=Passw0rd123! \
  [email protected]

Create a policy to generate that image pull secret to any new namespace:

values/platformName/raw/kyverno/generate-secret.yaml

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: sync-secrets
spec:
  rules:
  - name: sync-image-pull-secret
    match:
      any:
      - resources:
          kinds:
          - Namespace
    generate:
      apiVersion: v1
      kind: Secret
      name: regcred
      namespace: "{{request.object.metadata.name}}"
      synchronize: true
      clone:
        namespace: default
        name: regcred

Test the policy:

Terminal

kubectl create ns test            # should create secret 'regcred' in namespace 'test'