K8s, volume from secret is loaded into directory instead of file

3/1/2022

I'm using helm and k8s to deploy an opensearch cluster. I'm loading the TLS certificates as plain text from aws secret manager using the aws secret manager by mumoshu.

---
apiVersion: mumoshu.github.io/v1alpha1
kind: AWSSecret
metadata:
  name: "{{ .Values.AWSSecrets.name }}-{{ .Values.AWSSecrets.admin }}"
spec:
  stringDataFrom:
    secretsManagerSecretRef:
      secretId: "{{ .Values.global.clusterName }}/admin"
      versionId: {{ .Values.awsSecretVersionID }}
---
apiVersion: mumoshu.github.io/v1alpha1
kind: AWSSecret
metadata:
  name: "{{ .Values.AWSSecrets.name }}-{{ .Values.AWSSecrets.adminKey }}"
spec:
  stringDataFrom:
    secretsManagerSecretRef:
      secretId: "{{ .Values.global.clusterName }}/admin-key"
      versionId: {{ .Values.awsSecretVersionID }}
---

and in my deployment:

spec:
  serviceName: opensearch-cluster-master-headless
  selector:
    matchLabels:
      app: {{ .Values.global.name }}
  replicas: {{  .Values.replicas.master }}
  template:
      containers:
          # spec of containers ...
          volumeMounts:
            - name: admin-key
              mountPath: {{ .Values.adminKeyCertPathOnMachine }}
              subPath: admin-key.pem
              readOnly: true
            - name: admin
              mountPath: {{ .Values.adminCertPathOnMachine }}
              subPath: admin.pem
              readOnly: true
      volumes:
        - name: admin
          secret:
            secretName: "{{ .Values.AWSSecrets.name }}-{{ .Values.AWSSecrets.admin }}"
        - name: admin-key
          secret:
            secretName: "{{ .Values.AWSSecrets.name }}-{{ .Values.AWSSecrets.adminKey }}"

My problem is that this configuration loads the secrets as directories instead of files.

It did work when the certificates where in a yaml file and specified the:

items:
  - key:
    path:

but once I removed it, it became a directory. I removed it since saving the certificates in aws secret had to be a plain text to maintain the multiline structure of the certificate so JSON format wasn't possible.

-- yovel cohen
aws-secrets-manager
kubernetes
kubernetes-helm
kubernetes-secrets

0 Answers