I run several wordpress instances on my company's Kubernetes cluster. Each instance has its own persistency volume and a container. The only peculiarity of my setup, is that I mount several paths of the PV onto several paths of the container.
All my containers worked well since a couple of weeks ago, when we upgraded Kubernetes to the current version. Since then, the hell began.
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.11", GitCommit:"637c7e288581ee40ab4ca210618a89a555b6e7e9", GitTreeState:"clean", BuildDate:"2018-11-26T14:38:32Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.11", GitCommit:"637c7e288581ee40ab4ca210618a89a555b6e7e9", GitTreeState:"clean", BuildDate:"2018-11-26T14:25:46Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
When restarting a pod, if it gets scheduled to run on a different node, it get stuck on PodInitializing with the following event message
Multi-Attach error for volume "pvc-ac6b35f3-7716-11e8-adda-b60483de6a40" Volume is already exclusively attached to one node and can't be attached to another
Here are my resources.
It contains two directories and a file
html/
: directory with php fileslogs/
: directory with log filescontainer-data.txt
: a text file with some infoDefined as:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: rbd-wordpress-mysite
labels:
app: wordpress
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
kind: Deployment
apiVersion: apps/v1beta1
metadata:
name: wordpress-mysite
labels:
app: wordpress
namespace: unibz
spec:
template:
metadata:
name: wordpress-mysite
labels:
app: wordpress
namespace: unibz
spec:
containers:
- name: wordpress-mysite
image: myimage
volumeMounts:
- mountPath: "/root/container-data.txt"
name: wordpress-data
subPath: container-data.txt
- mountPath: "/var/www/html"
name: wordpress-data
subPath: html
- mountPath: "/var/log/apache2"
name: wordpress-data
subPath: logs
ports:
- containerPort: 80
name: wordpress-http
volumes:
- name: wordpress-data
persistentVolumeClaim:
claimName: rbd-wordpress-mysite
- name: wordpress-conf
configMap:
name: wordpress-conf
Is this way of using the persistency wrong? Could it be the cause of the Multi-Attach error?
Looks like you are trying to attache the same PVC
into a different node
.
Access Modes Claims use the same conventions as volumes when requesting storage with specific access modes
In your .yaml
I can see, you have setup accessModes: ReadWriteOnce
.
A
PersistentVolume
can be mounted on a host in any way supported by the resource provider. As shown in the table below, providers will have different capabilities and each PV’s access modes are set to the specific modes supported by that particular volume. For example, NFS can support multiple read/write clients, but a specific NFS PV might be exported on the server as read-only. Each PV gets its own set of access modes describing that specific PV’s capabilities.The access modes are:
ReadWriteOnce – the volume can be mounted as read-write by a single node
ReadOnlyMany – the volume can be mounted read-only by many nodes
ReadWriteMany – the volume can be mounted as read-write by many nodes
From Kubernetes docs regarding Persistent Volumes
you can read that CephFS does support all accessModes