pod events shows persistentvolumeclaim "flink-pv-claim-11" is being deleted in kubernetes but binding success

8/19/2020

I am binding persistentvolumeclaim into my Job's pod, it shows:

persistentvolumeclaim "flink-pv-claim-11" is being deleted

but the persistent volume claim is exists and binded success.

s

and the pod has no log output. what should I do to fix this? this is the job yaml:

apiVersion: batch/v1
kind: Job
metadata:
  name: flink-jobmanager-1.11
spec:
  template:
    metadata:
      labels:
        app: flink
        component: jobmanager
    spec:
      restartPolicy: OnFailure
      containers:
        - name: jobmanager
          image: flink:1.11.0-scala_2.11
          env:
          args: ["standalone-job", "--job-classname", "com.job.ClassName", <optional arguments>, <job arguments>] # optional arguments: ["--job-id", "<job id>", "--fromSavepoint", "/path/to/savepoint", "--allowNonRestoredState"]
          ports:
            - containerPort: 6123
              name: rpc
            - containerPort: 6124
              name: blob-server
            - containerPort: 8081
              name: webui
          livenessProbe:
            tcpSocket:
              port: 6123
            initialDelaySeconds: 30
            periodSeconds: 60
          volumeMounts:
            - name: flink-config-volume
              mountPath: /opt/flink/conf
            - name: job-artifacts-volume
              mountPath: /opt/flink/usrlib
            - name: job-artifacts-volume
              mountPath: /opt/flink/data/job-artifacts
          securityContext:
            runAsUser: 9999  # refers to user _flink_ from official flink image, change if necessary
      volumes:
        - name: flink-config-volume
          configMap:
            name: flink-1.11-config
            items:
              - key: flink-conf.yaml
                path: flink-conf.yaml
              - key: log4j-console.properties
                path: log4j-console.properties
        - name: job-artifacts-volume
          persistentVolumeClaim:
            claimName: flink-pv-claim-11
-- Dolphin
kubernetes

2 Answers

8/19/2020

Make sure the job tied to this PVC is deleted. If any other job/pods are running and using this pvc then you can not delete the PVC until you delete job/pod.

To see which resources are in use at the moment from the PVC try to run:

kubectl get pods --all-namespaces -o=json | jq -c \ '.items[] | {name: .metadata.name, namespace: .metadata.namespace, claimName:.spec.volumes[] | select( has ("persistentVolumeClaim") ).persistentVolumeClaim.claimName }'

-- Dashrath Mundkar
Source: StackOverflow

8/19/2020

You have two volumeMounts named job-artifacts-volume, which may be causing some confusion.

-- David Anderson
Source: StackOverflow