I'm testing out whether I can mount data from S3 using initContainer. What I intended and expected was same volume being mounted to both initContainer and Container. Data from S3 gets downloaded using InitContainer to mountPath called /s3-data, and as the Container is run after the initContainer, it can read from the path the volume was mounted to.

However, the Container doesn't show me any logs, and just says 'stream closed'. The initContainer shows logs that data were successfully downloaded from S3.

What am I doing wrong? Thanks in advance.

apiVersion: batch/v1
kind: Job
metadata:
  name: train-job
spec:
  template:
    spec:
      initContainers:
      - name: data-download
        image: <My AWS-CLI Image>
        command: ["/bin/sh", "-c"]
        args:
          - aws s3 cp s3://<Kubeflow Bucket>/kubeflowdata.tar.gz /s3-data
        volumeMounts:
        - mountPath: /s3-data
          name: s3-data
        env:
        - name: AWS_ACCESS_KEY_ID
          valueFrom:
            secretKeyRef: {key: AWS_ACCESS_KEY_ID, name: aws-secret}
        - name: AWS_SECRET_ACCESS_KEY
          valueFrom:
            secretKeyRef: {key: AWS_SECRET_ACCESS_KEY, name: aws-secret}
      containers:
      - name: check-proper-data-mount
        image: <My Image>
        command: ["/bin/sh", "-c"]
        args:
          - cd /s3-data
          - echo "Just s3-data dir"
          - ls
          - echo "After making a sample file"
          - touch sample.txt
          - ls
        volumeMounts:
        - mountPath: /s3-data
          name: s3-data
      volumes:
      - name: s3-data
        emptyDir: {}
      restartPolicy: OnFailure
  backoffLimit: 6