I'm trying to ssh from a pod into a remote server while specifying an identity file. This fails with the following error:

admin@123.123.123.123: Permission denied (publickey).

I've made sure I can connect from my local host with the same set of public and private keys. It only fails when I try to connect from a bash shell inside the pod's container. My Job definition is as follows :

---
apiVersion: batch/v1
kind: Job
metadata:
  name: {{ .Release.Name }}-volumes-population-{{ .Release.Revision }}
spec:
  template:
    spec:
      containers:
      - name: populate-volumes
        image: {{ .Values.gitlab.image_repository.repository }}/{{ .Values.phpfpm.image.name }}:{{ .Values.phpfpm.image.version }}
        imagePullPolicy: IfNotPresent
        ports:
        - name: ssh
          containerPort: 22
        args:
        - /bin/bash
        - -c
        - |
            echo "Testing ssh connection..."
            ssh -i/etc/ssh/hetzner_box admin@123.123.123.123
        volumeMounts:
        - name: hetzner-box-identity
          mountPath: /etc/ssh/hetzner_box.pub
          subPath: .pub
        - name: hetzner-box-identity
          mountPath: /etc/ssh/hetzner_box
          subPath: .key
      volumes:
      - name: hetzner-box-identity
        secret:
          secretName: {{ .Release.Name }}-hetzner-box-identity
          defaultMode: 256
          items:
          - key: .pub
            path: .pub
          - key: .key
            path: .key

Edit 1:
After further inquiries I've manage to notice that the key pair is passphrase less. I've managed to login using a different key pair, protected by a passphrase. My goal is automation and is therefore unacceptable to have passphrase protected keys. Is there a reason the ssh daemon is refusing to authenticate a passphrase less key?