When we delete a pvc, recycler suppose and run and delete data and put PV status back to "Available". But this is not happening even though the serviceaccount exist.

oc describe pv shows

Message:        Recycle failed: Pod was active on the node longer than the specified deadline

and

for pod "recycler-for-lv-1g-038_openshift-infra(91e648fd-e7b4-11e8-b089-40f2e91e23fb)": timeout expired waiting for volumes to attach/mount for pod "openshift-infra"/"recycler-for-lv-1g-101". list of unattached/unmounted volumes=[vol pv-recycler-controller-token-5rxmu]

and see serviceaccount below.

oc get serviceaccount pv-recycler-controller -n openshift-infra
NAME                     SECRETS   AGE
pv-recycler-controller   2         1y

version :

oc version
oc v3.6.173.0.129
kubernetes v1.6.1+5115d708d7
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://lbint-example.com:443
openshift v3.6.173.0.129
kubernetes v1.6.1+5115d708d7

PV details:

# oc describe pv/lv-1g-130
Name:           lv-1g-130
Labels:         <none>
Annotations:    pv.kubernetes.io/bound-by-controller=yes
StorageClass:
Status:         Failed
Claim:          db-buildingblocks/mongodb3-configsvr-data-mongodb3configsvr-0
Reclaim Policy: Recycle
Access Modes:   RWO,ROX,RWX
Capacity:       1Gi
Message:        Recycle failed: recycler pod was deleted
Source:
    Type:       NFS (an NFS mount that lasts the lifetime of a pod)
    Server:     <REMOVED_INFO>
    Path:       /f03796/t050402/data/vg_data/lv-1g-130
    ReadOnly:   false
Events:
  FirstSeen     LastSeen        Count   From                            SubObjectPath   Type            Reason                  Message
  ---------     --------        -----   ----                            -------------   --------        ------                  -------
  26m           26m             1       persistentvolume-controller                     Warning         VolumeFailedRecycle     Recycle failed: recycler pod was deleted
  23m           23m             1       persistentvolume-controller                     Warning         RecyclerPod             Recycler pod: Unable to mount volumes for pod "recycler-for-lv-1g-130_openshift-infra(832c2160-e7c8-11e8-b089-40f2e91e23fb)": timeout expired waiting for volumes to attach/mount for pod "openshift-infra"/"recycler-for-lv-1g-130". list of unattached/unmounted volumes=[vol pv-recycler-controller-token-4fxtu]
  19m           19m             1       persistentvolume-controller                     Normal          RecyclerPod             Recycler pod: Pod was active on the node longer than the specified deadline
  40d           18m             18      persistentvolume-controller                     Normal          RecyclerPod             Recycler pod: Successfully assigned recycler-for-lv-1g-130 to cluster1.example.com
  25m           18m             2       persistentvolume-controller                     Warning         RecyclerPod             Recycler pod: Unable to mount volumes for pod "recycler-for-lv-1g-130_openshift-infra(50f992fb-e7c8-11e8-b089-40f2e91e23fb)": timeout expired waiting for volumes to attach/mount for pod "openshift-infra"/"recycler-for-lv-1g-130". list of unattached/unmounted volumes=[vol pv-recycler-controller-token-4fxtu]
  24m           18m             2       persistentvolume-controller                     Warning         RecyclerPod             Recycler pod: Unable to mount volumes for pod "recycler-for-lv-1g-130_openshift-infra(72a32e18-e7c8-11e8-b089-40f2e91e23fb)": timeout expired waiting for volumes to attach/mount for pod "openshift-infra"/"recycler-for-lv-1g-130". list of unattached/unmounted volumes=[vol pv-recycler-controller-token-4fxtu]
  23m           18m             2       persistentvolume-controller                     Warning         RecyclerPod             Recycler pod: Unable to mount volumes for pod "recycler-for-lv-1g-130_openshift-infra(8c1d22f5-e7c8-11e8-b089-40f2e91e23fb)": timeout expired waiting for volumes to attach/mount for pod "openshift-infra"/"recycler-for-lv-1g-130". list of unattached/unmounted volumes=[vol]
  25m           16m             9       persistentvolume-controller                     Warning         RecyclerPod             Recycler pod: Error syncing pod