flocker w/ k8s - volume in a detached state

12/10/2016

I'm running flocker 1.15.0 along side k8s 1.4.5, docker 1.10.3 on centos 7. I setup a flocker volume and when I try the basic sample here the k8s pod doesn't come up properly.

# flockerctl list
DATASET                                SIZE    METADATA              STATUS     SERVER
e06ac9f2-34da-4cc2-b5ed-ea6aef22d707   1.00G   name=my-flocker-vol   detached   faae089e (10.1.1.214)

k8s:

$ kubectl create -f https://raw.githubusercontent.com/kubernetes/kubernetes/master/examples/volumes/flocker/flocker-pod.yml
$ kubectl describe pod flocker-web
...
...
  23m           23m             1       {default-scheduler }                                            Normal          Scheduled       Successfully assigned flocker-web to kube-minion-1
  21m           3m              9       {kubelet kube-minion-1}                        Warning         FailedMount     MountVolume.SetUp failed for volume "kubernetes.io/flocker/89c4fc2a-bf2b-11e6-8353-0050568e8b79-www-root" (spec.Name: "www-root") pod "89c4fc2a-bf2b-11e6-8353-0050568e8b79" (UID: "89c4fc2a-bf2b-11e6-8353-0050568e8b79") with: Timed out waiting for the dataset_id: 'e06ac9f2-34da-4cc2-b5ed-ea6aef22d707' to be moved to the primary: 'faae089e-cf3a-4c22-9496-ce3ae2bfc6fc'
<nil>
  21m   1m      10      {kubelet kube-minion-1}                Warning FailedMount     Unable to mount volumes for pod "flocker-web_default(89c4fc2a-bf2b-11e6-8353-0050568e8b79)": timeout expired waiting for volumes to attach/mount for pod "flocker-web"/"default". list of unattached/unmounted volumes=[www-root]
  21m   1m      10      {kubelet kube-minion-1}                Warning FailedSync      Error syncing pod, skipping: timeout expired waiting for volumes to attach/mount for pod "flocker-web"/"default". list of unattached/unmounted volumes=[www-root]

from /var/log/messages:

Dec 10 17:55:59 kube-minion-1 kubelet: E1210 17:55:59.738342    2604 kubelet.go:1813] Unable to mount volumes for pod "flocker-web_default(89c4fc2a-bf2b-11e6-8353-0050568e8b79)": timeout expired waiting for volumes to attach/mount for pod "flocker-web"/"default". list of unattached/unmounted volumes=[www-root]; skipping pod
Dec 10 17:55:59 kube-minion-1 kubelet: E1210 17:55:59.738437    2604 pod_workers.go:184] Error syncing pod 89c4fc2a-bf2b-11e6-8353-0050568e8b79, skipping: timeout expired waiting for volumes to attach/mount for pod "flocker-web"/"default". list of unattached/unmounted volumes=[www-root]
Dec 10 17:56:00 kube-minion-1 kubelet: E1210 17:56:00.218853    2604 nestedpendingoperations.go:253] Operation for "\"kubernetes.io/flocker/89c4fc2a-bf2b-11e6-8353-0050568e8b79-www-root\" (\"89c4fc2a-bf2b-11e6-8353-0050568e8b79\")" failed. No retries permitted until 2016-12-10 17:56:00.718802626 -0500 EST (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kubernetes.io/flocker/89c4fc2a-bf2b-11e6-8353-0050568e8b79-www-root" (spec.Name: "www-root") pod "89c4fc2a-bf2b-11e6-8353-0050568e8b79" (UID: "89c4fc2a-bf2b-11e6-8353-0050568e8b79") with: Timed out waiting for the dataset_id: 'e06ac9f2-34da-4cc2-b5ed-ea6aef22d707' to be moved to the primary: 'faae089e-cf3a-4c22-9496-ce3ae2bfc6fc'

Any idea on how I can further debug this?

-- scottmf
kubernetes

1 Answer

12/11/2016

I see the problem, I used the loopback backend. Flocker move functionality isn't supported in this mode.

https://flocker-docs.clusterhq.com/en/latest/flocker-features/loopback-configuration.html#loopback-dataset-backend

-- scottmf
Source: StackOverflow