I have a failing service with Kubernetes, it seems that service doesnt want to mount volume.
Unable to mount volumes for pod "metadata-api-local": timeout expired waiting for volumes to attach or mount for pod "metadata"/"metadata-api-local". list of unmounted volumes=[metadata-api-claim]. list of unattached volumes=[metadata-api-claim default-token-8lqmp]
Here is the log:
➜ metadata_api git:(develop) ✗ kubectl describe pod -n metadata metadata-api-local-f5bddb8f7-clmwq
Name: metadata-api-local-f5bddb8f7-clmwq
Namespace: metadata
Priority: 0
Node: minikube/192.168.0.85
Start Time: Wed, 18 Sep 2019 16:59:02 +0200
Labels: app=metadata-api-local
pod-template-hash=f5bddb8f7
Annotations: <none>
Status: Pending
IP:
Controlled By: ReplicaSet/metadata-api-local-f5bddb8f7
Containers:
metadata-api-local:
Container ID:
Image: metadata_api:local
Image ID:
Port: 18000/TCP
Host Port: 0/TCP
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Environment Variables from:
metadata-env Secret Optional: false
Environment: <none>
Mounts:
/var/lib/nodered-peer from metadata-api-claim (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-8lqmp (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
metadata-api-claim:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: metadata-api-claim
ReadOnly: false
default-token-8lqmp:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-8lqmp
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 14m default-scheduler Successfully assigned metadata/metadata-api-local-f5bddb8f7-clmwq to minikube
Warning FailedMount 47s (x6 over 12m) kubelet, minikube Unable to mount volumes for pod "metadata-api-local-f5bddb8f7-clmwq_metadata(94cbb26c-4907-4512-950a-29a25ad1ef20)": timeout expired waiting for volumes to attach or mount for pod "metadata"/"metadata-api-local-f5bddb8f7-clmwq". list of unmounted volumes=[metadata-api-claim]. list of unattached volumes=[metadata-api-claim default-token-8lqmp]
Here is my metadata_pvc.yml:
apiVersion: v1
kind: PersistentVolume
metadata:
name: metadata-api-pv
namespace: metadata
spec:
capacity:
storage: 1Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
claimRef:
namespace: metadata
name: metadata-api-claim
hostPath:
path: /data/metadata-api
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: metadata-api-claim
namespace: metadata
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: metadata-postgres-volume
namespace: metadata
spec:
capacity:
storage: 1Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
claimRef:
namespace: metadata
name: metadata-postgres-claim
hostPath:
path: /data/metadata-postgres
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: metadata-postgres-claim
namespace: metadata
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
When I list pv, I get:
➜ metadata_api git:(develop) ✗ kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
metadata-api-pv 1Gi RWO Retain Available metadata/metadata-api-claim 12m
metadata-postgres-volume 1Gi RWO Retain Available metadata/metadata-postgres-claim 12m
➜ metadata_api git:(develop) ✗ kubectl get pvc
No resources found.
What is failing ?
You shouldn't specify claimRef
, that field is automatically generated by Kubernetes controllers. Instead you should use storage classes for both your PersistentVolumes and PersistentVolumeClaims as that is the mechanism used to match them. Adding the storageClassName: name
field to both your PersistentVolumes and PersistentVolumeClaims should fix your issue.