chown: changing ownership of '/var/lib/mysql/': Operation not permitted in start pods in kubernetes

5/30/2020

When I start my mysql 5.7 pods in kubernetes v1.15.2, the logs shows like this:

2020-05-30 13:08:04+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.30-1debian10 started.
chown: changing ownership of '/var/lib/mysql/': Operation not permitted

this is my mysql yaml define:

apiVersion: v1
kind: Service
metadata:
  name: apollo-mysql
  namespace: sre
spec:
  ports:
  - port: 3306
  selector:
    app: mysql
  clusterIP: None
---
apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
kind: Deployment
metadata:
  name: apollo-mysql
  namespace: sre
spec:
  selector:
    matchLabels:
      app: apollo-mysql
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: apollo-mysql
    spec:
      containers:
      - image: mysql:5.7
        name: mysql
        env:
          # Use secret in real usage
        - name: MYSQL_ROOT_PASSWORD
          value: gfwge4LucnXwfefewegLwAd29QqJn4
        ports:
        - containerPort: 3306
          name: mysql
        volumeMounts:
        - name: apollo-mysql-persistent-storage
          mountPath: /var/lib/mysql
      volumes:
      - name: apollo-mysql-persistent-storage
        persistentVolumeClaim:
          claimName: apollo-mysql-pv-claim

and this is my pv define:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: nfs-apollo-mysql-pv1
  namespace: sre
  labels:
    alicloud-pvname: apollo-mysql-data-db
spec:
  capacity:
    storage: 5Gi
  storageClassName: apollo-mysql-data-db
  mountOptions:
    - vers=4.0
    - noresvport
  accessModes:
    - ReadWriteOnce
  nfs:
    server: "192.168.64.237"
    path: "/mnt/data/apollodb/apollopv"
  persistentVolumeReclaimPolicy: Retain

and I already change the mod to 777 like this:

chmod 777 /mnt/data/apollodb/apollopv

so where is the problem and what should I do to fix this? I check the selinux status and it is disabled.

[miao@meowk8sslave3 apollodb]$ /usr/sbin/sestatus -v
SELinux status:                 disabled

This is my nfs exports define:

/mnt/data/apollodb/apollopv *(rw,sync,no_subtree_check,no_root_squash)
-- Dolphin
kubernetes

1 Answer

5/30/2020

This yaml do not make much sense. You are still missing PVC yaml, which will need the storageClass to be mounted, which I doubt you have. Do you?

It looks like you are trying to mount an external NFS (with some cloud provider perhaps), in which case you need to have nfs-utils installed, and do mount -t nfs 192.168.64.237:/entrypoint /mnt/....

Your PV says /mnt/data/apollodb/apollopv, which should be the entrypoint to the nfs server, but it seems that you think that's where it is going to be mounted in your filesystem. So, this is a soup.

To make this work, get yourself an nfs server (get the IP address and the entrypoint; say 192.168.64.237:/nfs), then mount it in your pod as follows:

apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
kind: Deployment
metadata:
  name: apollo-mysql
  namespace: sre
spec:
  selector:
    matchLabels:
      app: apollo-mysql
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: apollo-mysql
    spec:
      containers:
      - image: mysql:5.7
        name: mysql
        env:
          # Use secret in real usage
        - name: MYSQL_ROOT_PASSWORD
          value: gfwge4LucnXwfefewegLwAd29QqJn4
        ports:
        - containerPort: 3306
          name: mysql
        volumeMounts:
        - name: apollo-mysql-persistent-storage
          mountPath: /var/lib/mysql
      volumes:
      - name: apollo-mysql-persistent-storage
        nfs:
          server: 192.168.64.237
          path: "/nfs"

You won't need the storage class, the PVC and the PV. Everything you need is in the pod, and externally you have the nfs server.

-- suren
Source: StackOverflow