Permission denied for the Kaniko job in openshift cluster

3/29/2020

I trying to do a deployment through my k8s operator on openshift 3.11 cluster. When the kaniko job starts it gives me the following error.

Error: error resolving dockerfile path: copying dockerfile: open /kaniko/Dockerfile: 
permission denied
-- uvindu sri
cloud
docker
kaniko
kubernetes
openshift

2 Answers

3/30/2020

Kaniko is being introduced as a tool to Build container images in Kubernetes and Google Container Builder without privileges.

we’re excited to introduce kaniko, an open-source tool for building container images from a Dockerfile even without privileged root access. With kaniko, we both build an image from a Dockerfile and push it to a registry. Since it doesn’t require any special privileges or permissions, you can run kaniko in a standard Kubernetes cluster, Google Kubernetes Engine, or in any environment that can’t have access to privileges or a Docker daemon.

The issue you are experiencing was already mentioned at GoogleContainerTools/kaniko GitHub issue.

On January 11 this issue was tagged as Won't Fix so the only way is to run Kaniko as root using securityContext: runAsUser: 0

This isn't secure as once would think, which is mentioned by Kurt Madel in his blog Securely Building Container Images on Kubernetes:

running as root is an attack vector that many consider to be an unacceptable security hole - but the use of Pod Security Policies will reduce the attack surface of the Kaniko container running as part of a K8s Pod and provides greater security than the Docker based approaches we have already dismissed.

He also explains how one would use Kaniko the Easy Way

Jenkins X allows you to enable Kaniko as the default way to build and push container images for all of your Jenkins X CD jobs and will be automatically configured to push to the default container registry of the cloud where you install Jenkins X and Kaniko caching is automatically set up for you - resulting in fast, secure container image builds that are pushed to your default Jenkins X container registry.

Important: Jenkins X does not have OOTB support for Pod Security Policies as tracked by this GitHub issue. In my next post we will take a look at using Pod Security Policies with Jenkins X - but not just for Kaniko, because once you enable Pod Security Policy every K8s Role/ClusterRole has to have a Pod Security Policy associated to it.

Drawbacks for Kaniko

  • Requires running the Kaniko container as ROOT to execute most container builds
  • Doesn’t work with all Dockerfiles but keeps improving
  • Is slightly more complicated to setup than the good old docker build
-- Crou
Source: StackOverflow

3/29/2020

Add securityContext: runAsUser: 0 into pod spec to run it as root.

apiVersion: v1
kind: Pod
metadata:
  name: security-context-demo
spec:
  securityContext:
    runAsUser: 0
-- Arghya Sadhu
Source: StackOverflow