K
Q

Validate K8s YAML Files in a Git repo

5/11/2020

I have a set of K8s YAML descriptors as part of a project and I'm using kustomization to build them. I'm also using GitOps to do pull based deployments to my K8s cluster.

I now want to add some tests for my YAML files so that if I have any errors, I want to avoid or prevent Flux from pulling my changes into the cluster. So basically I want to do some unit test like thingy for my YAML files. I came across Kubeval and this could serve my purpose well. I'm just not sure how to use it.

Anyone already tried this? I want to basically do the following:

  1. As soon as I push some YAML files into my repo, Kubeval kicks in and validates all the YAML files in a set of folders that I specify

  2. If all the YAML files passes lint validations, then I want to proceed to the next stage where I call kustomize to build the deployment YAML.

  3. If the YAML files fail lint validation, then my CI fails and nothing should happen

Any ideas on how I could do this?

-- sparkr
kubernetes

Similar Questions

Kubeflow fails to deploy using both CLI and Console
Airflow Kubernetes executor , Tasks in queued state and pod says invalid image and uses local executor
Websocket returns 500 on client and 101 on server
Error in cassandra cronjob back-up: /var/lib/cassandra/data does not exist. error in Stream: container cassandra is not valid for pod <pod-name>

1 Answer

5/11/2020

Since my project is hosted on GitHub, I was able to get what I want using GitHub actions and kube-tools

So basically here is what I did!

  1. In my GitHub project, added a main.yaml under project-root/.github/workflows/main.yml

  2. The contents of my main.yaml is:

    name: ValidateKubernetesYAML

    branches: [ master ]   pull_request:
branches: [ master ]

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2
- name: Kubeval
  uses: stefanprodan/kube-tools@v1.2.0
  with:
    kubectl: 1.16.2
    kustomize: 3.4.0
    helm: 2.16.1
    helmv3: 3.0.0
    command: |
      echo "Run kubeval"
      kubeval -d base,dev,production --force-color --strict --ignore-missing-schemas

Now when someone issues a pull request into master, this validation kicks in and if it fails the changes does not get promoted into master branch which is what I want!

Here is the output of such a validation:

Run kubeval
WARN - Set to ignore missing schemas
PASS - base/application/plant-simulator-deployment.yaml contains a valid Deployment
PASS - base/application/plant-simulator-ingress-service.yaml contains a valid Ingress
PASS - base/application/plant-simulator-namespace.yaml contains a valid Namespace
PASS - base/application/plant-simulator-service.yaml contains a valid Service
WARN - base/kustomization.yaml containing a Kustomization was not validated against a schema
PASS - base/monitoring/grafana/grafana-deployment.yaml contains a valid Deployment
PASS - base/monitoring/grafana/grafana-service.yaml contains a valid Service
PASS - base/monitoring/plant-simulator-monitoring-namespace.yaml contains a valid Namespace
PASS - base/monitoring/prometheus/config-map.yaml contains a valid ConfigMap
PASS - base/monitoring/prometheus/prometheus-deployment.yaml contains a valid Deployment
PASS - base/monitoring/prometheus/prometheus-roles.yaml contains a valid ClusterRole
PASS - base/monitoring/prometheus/prometheus-roles.yaml contains a valid ServiceAccount
PASS - base/monitoring/prometheus/prometheus-roles.yaml contains a valid ClusterRoleBinding
PASS - base/monitoring/prometheus/prometheus-service.yaml contains a valid Service
PASS - dev/flux-patch.yaml contains a valid Deployment
WARN - dev/kustomization.yaml containing a Kustomization was not validated against a schema
PASS - production/flux-patch.yaml contains a valid Deployment
WARN - production/kustomization.yaml containing a Kustomization was not validated against a schema
-- sparkr
Source: StackOverflow