Trying to copy/mount file to AWX kubernetes pod but getting error

6/6/2019

I'm attempting to add a file to the /etc/ directory on an AWX task/web container in kubernetes. I'm fairly new to helm and I'm not sure what I'm doing wrong.

The only thing I've added to my helm chart is krb5 key in configmap and an additional volume and volume mount to both task and web container. The krb5.conf file is in charts/mychart/files/

ConfigMap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: {{ include "awx.fullname" . }}-application-config
  labels:
    app.kubernetes.io/name: {{ include "awx.name" . }}
    helm.sh/chart: {{ include "awx.chart" . }}
    app.kubernetes.io/instance: {{ .Release.Name }}
    app.kubernetes.io/managed-by: {{ .Release.Service }}
data:
  krb5: |-
  {{ .Files.Get "krb5.conf"}}
  secret_key: {{ .Values.awx_secret_key }}
  awx_settings: |
    *some stuff*

Deployment:

Volumes add to bottom of deployment.yaml

volumes:
  - name: {{ include "awx.fullname" . }}-application-config
    configMap:
      name: {{ include "awx.fullname" . }}-application-config
      items:
        - key: awx_settings
          path: settings.py
        - key: secret_key
          path: SECRET_KEY
  - name: {{ include "awx.fullname" . }}-application-config-krb5
    configMap:
      name: {{ include "awx.fullname" . }}-application-config
      items:
        - key: krb5
          path: krb5.conf

Volume Mounts add to both task/web container

 volumeMounts:
   - mountPath: /etc/tower
     name: {{ include "awx.fullname" . }}-application-config
   - mountPath: /etc
     name: {{ include "awx.fullname" . }}-application-config-krb5

I'm trying to mount a file to the containers in a kubernetes pod and am getting the following error:

  Warning  Failed     40s                kubelet, aks-prdnode-18232119-1  Error: failed to start container "web": Error response from daemon: OCI runtime create failed: container_linux.go:344: starting container process caused "process_linux.go:424: container init caused \"rootfs_linux.go:58: mounting \\\"/var/lib/docker/containers/d66044fe204abbf9a4d3772370d0f8d4184e339e59ad9a018f046eade03b8418/resolv.conf\\\" to rootfs \\\"/var/lib/docker/overlay2/d9fa9705d70bbb864ed526a96f6a2873b2720c41a9f9ef5b4a428902e4cf3c82/merged\\\" at \\\"/var/lib/docker/overlay2/d9fa9705d70bbb864ed526a96f6a2873b2720c41a9f9ef5b4a428902e4cf3c82/merged/etc/resolv.conf\\\" caused \\\"open /var/lib/docker/overlay2/d9fa9705d70bbb864ed526a96f6a2873b2720c41a9f9ef5b4a428902e4cf3c82/merged/etc/resolv.conf: read-only file system\\\"\"": unknown
-- Ian Clark
ansible-awx
containers
kubernetes
kubernetes-helm

1 Answer

6/7/2019

You'll want to use the subPath: option to "reach into" that -application-config-krb5 and mount only the one file:

- mountPath: /etc/krb5.conf
  name: {{ include "awx.fullname" . }}-application-config-krb5
  subPath: krb5.conf

since, as the error correctly points out, you for sure don't want to blow away the /etc directory of almost any container environment (it'll nuke /etc/passwd, /etc/hosts, resolv.conf, and a bazillion other important files)

-- mdaniel
Source: StackOverflow