kubernetes create pod with iscsi volume

7/21/2018

I'm new to K8s and now i'm stuck with create a pod with iscsi volume I'm using openfiler as iscsi target, provide a LUN to K8s host nodes. I can see the disk through "multipath -l":

mpatha (14f504e46494c455265305742724d2d7949544e2d736c4744) dm-3 OPNFILER,VIRTUAL-DISK    
size=1.0G features='0' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=0 status=active
| `- 33:0:0:0 sdb 8:16 active undef unknown
`-+- policy='service-time 0' prio=0 status=enabled
  `- 34:0:0:0 sdc 8:32 active undef unknown

or "cat /proc/scsi/scsi"

Attached devices:
Host: scsi0 Channel: 00 Id: 00 Lun: 00
  Vendor: VMware   Model: Virtual disk     Rev: 2.0 
  Type:   Direct-Access                    ANSI  SCSI revision: 06
Host: scsi3 Channel: 00 Id: 00 Lun: 00
  Vendor: NECVMWar Model: VMware SATA CD00 Rev: 1.00
  Type:   CD-ROM                           ANSI  SCSI revision: 05
Host: scsi33 Channel: 00 Id: 00 Lun: 00
  Vendor: OPNFILER Model: VIRTUAL-DISK     Rev: 0   
  Type:   Direct-Access                    ANSI  SCSI revision: 04
Host: scsi34 Channel: 00 Id: 00 Lun: 00
  Vendor: OPNFILER Model: VIRTUAL-DISK     Rev: 0   
  Type:   Direct-Access                    ANSI  SCSI revision: 04

My deployment yaml file:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mysql-test
  labels:
    tier: db
    system: test
    name: mysql
spec:
  replicas: 1
  template:
    metadata:
      name: mysql
      labels:
        name: mysql
        tier: db
    spec:
      containers:
        - name: mysql
          image: linhnm/lab:linhnm-wp
          volumeMounts:
            - mountPath: /var/lib/mysql
              name: pv-sql
      volumes:
        - name: pv-sql
          iscsi:
            targetPortal: 172.20.30.76:3260
            iqn: iqn.2006-01.com.openfiler:tsn.11934dc96acf
            lun: 0
            fsType: ext4
            readOnly: false
  selector:
    matchLabels:
      tier: db
      name: mysql

My K8s cluster cannot create a pod, and here's event in describe pod command:

Events:
  Type     Reason                  Age               From                     Message
  ----     ------                  ----              ----                     -------
  Normal   Scheduled               3m                default-scheduler        Successfully assigned mysql-test-7f5d55b8d8-tdxkk to worker1
  Normal   SuccessfulAttachVolume  3m                attachdetach-controller  AttachVolume.Attach succeeded for volume "pv-sql"
  Normal   SuccessfulMountVolume   3m                kubelet, worker1         MountVolume.SetUp succeeded for volume "default-token-wqhhj"
  Warning  FailedMount             1m                kubelet, worker1         Unable to mount volumes for pod "mysql-test-7f5d55b8d8-tdxkk_default(bef99831-8cc9-11e8-a98d-00505695a133)": timeout expired waiting for volumes to attach or mount for pod "default"/"mysql-test-7f5d55b8d8-tdxkk". list of unmounted volumes=[pv-sql]. list of unattached volumes=[pv-sql default-token-wqhhj]
  Warning  FailedMount             43s (x8 over 2m)  kubelet, worker1         MountVolume.WaitForAttach failed for volume "pv-sql" : failed to get any path for iscsi disk, last err seen:
Could not attach disk: Timeout after 10s

Can anyone show me what should I adjust my lab? Or what material I should read to go over this problem? Thank you

-- Linh Nguyen
iscsi
kubernetes

1 Answer

8/7/2018

Finally, I think that i found what cause the problem. It might be my openfiler iscsi server. After create another linux iscsi server with targetd and change my volume manifest to new ones properties, K8s cluster can create pod, then attach volume smoothly.

-- Linh Nguyen
Source: StackOverflow