I'm currently trying to solve a problem in a Ethereum blockchain running in kubernetes (EKS). I do not posses knowledge on BC besides the basic but I do have access to all the configuration of this environment and I want to learn from this.

The problem I'm facing is that, when a new signer pod is created, the genesis initContainer fails with the following error:

INFO 10-01|13:48:58.132 Maximum peer count ETH=25 LES=0 total=25

INFO 10-01|13:48:58.132 Allocated cache and file handles database=/ethereum/geth/chaindata cache=16 handles=16

Fatal: Failed to open database: resource temporarily unavailable

Fatal: Failed to open database: resource temporarily unavailable

The signer deployment is configured so it has a shared volume between the pods:

spec:
  volumes:
    - name: config
      configMap:
        name: genesis-config-6hbt7fcf72
        items:
          - key: genesis.json
            path: genesis.json
        defaultMode: 420
    - name: chaindata
      persistentVolumeClaim:
        claimName: volume-signer1
  initContainers:
    - name: genesis
      image: 'ethereum/client-go:alltools-v1.8.27'
      command:
        - geth
      args:
        - '--datadir=/ethereum'
        - init
        - /tmp/json/genesis.json
      resources: {}
      volumeMounts:
        - name: config
          readOnly: true
          mountPath: /tmp/json/genesis.json
          subPath: genesis.json
        - name: chaindata
          mountPath: /ethereum
      terminationMessagePath: /dev/termination-log
      terminationMessagePolicy: File
      imagePullPolicy: IfNotPresent

I think this error may be caused exactly by this shared volume, as it's trying to init a new BC database that already exists but I'm not sure of this as this is my first time working with this (I didn't configure this, I'm just trying to fix it).

As this is a production environment I can't test this so easily but I do have non-production environments where I can test stuff.

EDIT 1

Adding the PVC definition:

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: volume-signer1
  namespace: production
  labels:
    owner: ChaiVault
  annotations:
    pv.kubernetes.io/bind-completed: 'yes'
    pv.kubernetes.io/bound-by-controller: 'yes'
    volume.beta.kubernetes.io/storage-provisioner: kubernetes.io/aws-ebs
  finalizers:
    - kubernetes.io/pvc-protection
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 30Gi
  volumeName: pvc-35404d2c-e3c9-11e9-b77c-02da19f13e4c
  storageClassName: gp2
  volumeMode: Filesystem
status:
  phase: Bound
  accessModes:
    - ReadWriteOnce
  capacity:
    storage: 30Gi