kubectl exec permission denied

11/21/2019

I have a pod running mariadb container and I would like to backup my database but it fails with a Permission denied.

kubectl exec my-owncloud-mariadb-0 -it -- bash -c "mysqldump --single-transaction -h localhost -u myuser -ppassword mydatabase > owncloud-dbbackup_`date +"%Y%m%d"`.bak"

And the result is

bash: owncloud-dbbackup_20191121.bak: Permission denied
command terminated with exit code 1

I can't run sudo mysqldump because I get a sudo command not found.

I tried to export the backup file on different location: /home, the directory where mysqldump is located, /usr, ...

Here is the yaml of my pod:

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: "2019-11-20T14:16:58Z"
  generateName: my-owncloud-mariadb-
  labels:
    app: mariadb
    chart: mariadb-7.0.0
    component: master
    controller-revision-hash: my-owncloud-mariadb-77495ddc7c
    release: my-owncloud
    statefulset.kubernetes.io/pod-name: my-owncloud-mariadb-0
  name: my-owncloud-mariadb-0
  namespace: default
  ownerReferences:
  - apiVersion: apps/v1
    blockOwnerDeletion: true
    controller: true
    kind: StatefulSet
    name: my-owncloud-mariadb
    uid: 47f2a129-8d4e-4ae9-9411-473288623ed5
  resourceVersion: "2509395"
  selfLink: /api/v1/namespaces/default/pods/my-owncloud-mariadb-0
  uid: 6a98de05-c790-4f59-b182-5aaa45f3b580
spec:
  affinity:
    podAntiAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - podAffinityTerm:
          labelSelector:
            matchLabels:
              app: mariadb
              release: my-owncloud
          topologyKey: kubernetes.io/hostname
        weight: 1
  containers:
  - env:
    - name: MARIADB_ROOT_PASSWORD
      valueFrom:
        secretKeyRef:
          key: mariadb-root-password
          name: my-owncloud-mariadb
    - name: MARIADB_USER
      value: myuser
    - name: MARIADB_PASSWORD
      valueFrom:
        secretKeyRef:
          key: mariadb-password
          name: my-owncloud-mariadb
    - name: MARIADB_DATABASE
      value: mydatabase
    image: docker.io/bitnami/mariadb:10.3.18-debian-9-r36
    imagePullPolicy: IfNotPresent
    livenessProbe:
      exec:
        command:
        - sh
        - -c
        - exec mysqladmin status -uroot -p$MARIADB_ROOT_PASSWORD
      failureThreshold: 3
      initialDelaySeconds: 120
      periodSeconds: 10
      successThreshold: 1
      timeoutSeconds: 1
    name: mariadb
    ports:
    - containerPort: 3306
      name: mysql
      protocol: TCP
    readinessProbe:
      exec:
        command:
        - sh
        - -c
        - exec mysqladmin status -uroot -p$MARIADB_ROOT_PASSWORD
      failureThreshold: 3
      initialDelaySeconds: 30
      periodSeconds: 10
      successThreshold: 1
      timeoutSeconds: 1
    resources: {}
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /bitnami/mariadb
      name: data
    - mountPath: /opt/bitnami/mariadb/conf/my.cnf
      name: config
      subPath: my.cnf
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: default-token-pbgxr
      readOnly: true
  dnsPolicy: ClusterFirst
  enableServiceLinks: true
  hostname: my-owncloud-mariadb-0
  nodeName: 149.202.36.244
  priority: 0
  restartPolicy: Always
  schedulerName: default-scheduler
  securityContext:
    fsGroup: 1001
    runAsUser: 1001
  serviceAccount: default
  serviceAccountName: default
  subdomain: my-owncloud-mariadb
  terminationGracePeriodSeconds: 30
  tolerations:
  - effect: NoExecute
    key: node.kubernetes.io/not-ready
    operator: Exists
    tolerationSeconds: 300
  - effect: NoExecute
    key: node.kubernetes.io/unreachable
    operator: Exists
    tolerationSeconds: 300
  volumes:
  - name: data
    persistentVolumeClaim:
      claimName: data-my-owncloud-mariadb-0
  - configMap:
      defaultMode: 420
      name: my-owncloud-mariadb
    name: config
  - name: default-token-pbgxr
    secret:
      defaultMode: 420
      secretName: default-token-pbgxr
status:
  conditions:
  - lastProbeTime: null
    lastTransitionTime: "2019-11-20T14:33:22Z"
    status: "True"
    type: Initialized
  - lastProbeTime: null
    lastTransitionTime: "2019-11-20T14:34:03Z"
    status: "True"
    type: Ready
  - lastProbeTime: null
    lastTransitionTime: "2019-11-20T14:34:03Z"
    status: "True"
    type: ContainersReady
  - lastProbeTime: null
    lastTransitionTime: "2019-11-20T14:33:22Z"
    status: "True"
    type: PodScheduled
  containerStatuses:
  - containerID: docker://3898b6a20bd8c38699374b7db7f04ccef752ffd5a5f7b2bc9f7371e6a27c963a
    image: bitnami/mariadb:10.3.18-debian-9-r36
    imageID: docker-pullable://bitnami/mariadb@sha256:a89e2fab7951c622e165387ead0aa0bda2d57e027a70a301b8626bf7412b9366
    lastState: {}
    name: mariadb
    ready: true
    restartCount: 0
    state:
      running:
        startedAt: "2019-11-20T14:33:24Z"
  hostIP: 149.202.36.244
  phase: Running
  podIP: 10.42.2.56
  qosClass: BestEffort
  startTime: "2019-11-20T14:33:22Z"

Is their something I'm missing?

-- MHogge
kubectl
kubernetes
mariadb

2 Answers

11/21/2019

Given the pod YAML file you've shown, you can't usefully use kubectl exec to make a database backup.

You're getting a shell inside the pod and running mysqldump there to write out the dump file somewhere else inside the pod. You can't write it to the secret directory or the configmap directory, so your essential choices are either to write it to the pod filesystem (which will get deleted as soon as the pod exits, including if Kubernetes decides to relocate the pod within the cluster) or the mounted database directory (and your backup will survive exactly as long as the data it's backing up).

I'd run mysqldump from outside the pod. One good approach would be to create a separate Job that mounted some sort of long-term storage (or relied on external object storage; if you're running on AWS, for example, S3), connected to the database pod, and ran the backup that way. That has the advantage of being fairly self-contained (so you can debug it without interfering with your live database) and also totally automated (you could launch it from a Kubernetes CronJob).

kubectl exec doesn't seem to have the same flags docker exec does to control the user identity, so you're dependent on there being some path inside the container that its default user can write to. /tmp is typically world-writable so if you just want that specific command to work I'd try putting the dump file into /tmp/owncloud-dbbackup_....

-- David Maze
Source: StackOverflow

11/21/2019

You might not have permission to write to the location inside container. try the below command

use /tmp or some other location where you can dump the backup file

kubectl exec my-owncloud-mariadb-0 -it -- bash -c "mysqldump --single-transaction -h localhost -u myuser -ppassword mydatabase > /tmp/owncloud-dbbackup_`date +"%Y%m%d"`.bak"
-- P Ekambaram
Source: StackOverflow