kubectl - cert manager - credentials not found

7/6/2018

I want to have TLS termination enabled on ingress (on top of kubernetes) on google cloud platform.

My ingress cluster is working, my cert manager is failing with the error message

textPayload:  "2018/07/05 22:04:00 Error while processing certificate during sync: Error while creating ACME client for 'domain': Error while initializing challenge provider googlecloud: Unable to get Google Cloud client: google: error getting credentials using GOOGLE_APPLICATION_CREDENTIALS environment variable: open /opt/google/kube-cert-manager.json: no such file or directory
"  

This is what I did in order to get into the current state:

  • created cluster, deployment, service, ingress
  • executed:

    gcloud --project 'project' iam service-accounts create kube-cert-manager-sv-security --display-name "kube-cert-manager-sv-security"

    gcloud --project 'project' iam service-accounts keys create ~/.config/gcloud/kube-cert-manager-sv-security.json --iam-account kube-cert-manager-sv-security@'project'.iam.gserviceaccount.com

    gcloud --project 'project' projects add-iam-policy-binding --member serviceAccount:kube-cert-manager-sv-security@'project'.iam.gserviceaccount.com --role roles/dns.admin

    kubectl create secret generic kube-cert-manager-sv-security-secret --from-file=/home/perre/.config/gcloud/kube-cert-manager-sv-security.json

and created the following resources:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: kube-cert-manager-sv-security-data
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi

apiVersion: v1
kind: ServiceAccount
metadata:
  name: kube-cert-manager-sv-security
  namespace: default
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
    name: kube-cert-manager-sv-security
rules:
  - apiGroups: ["*"]
    resources: ["certificates", "ingresses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["*"]
    resources: ["secrets"]
    verbs: ["get", "list", "create", "update", "delete"]
  - apiGroups: ["*"]
    resources: ["events"]
    verbs: ["create"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: kube-cert-manager-sv-security-service-account
subjects:
  - kind: ServiceAccount
    namespace: default
    name: kube-cert-manager-sv-security
roleRef:
  kind: ClusterRole
  name: kube-cert-manager-sv-security
  apiGroup: rbac.authorization.k8s.io
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: certificates.stable.k8s.psg.io
spec:
  scope: Namespaced
  group: stable.k8s.psg.io
  version: v1
  names:
    kind: Certificate
    plural: certificates
    singular: certificate
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    app: kube-cert-manager-sv-security
  name: kube-cert-manager-sv-security
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: kube-cert-manager-sv-security
      name: kube-cert-manager-sv-security
    spec:
      serviceAccount: kube-cert-manager-sv-security
      containers:
        - name: kube-cert-manager
          env:
          - name: GCE_PROJECT
            value: solidair-vlaanderen-207315
          - name: GOOGLE_APPLICATION_CREDENTIALS
            value: /opt/google/kube-cert-manager.json
          image: bcawthra/kube-cert-manager:2017-12-10
          args:
            - "-data-dir=/var/lib/cert-manager-sv-security"
            #- "-acme-url=https://acme-staging.api.letsencrypt.org/directory"
            # NOTE: the URL above points to the staging server, where you won't get real certs.
            # Uncomment the line below to use the production LetsEncrypt server:
            - "-acme-url=https://acme-v01.api.letsencrypt.org/directory"
            # You can run multiple instances of kube-cert-manager for the same namespace(s),
            # each watching for a different value for the 'class' label
            - "-class=kube-cert-manager"
            # You can choose to monitor only some namespaces, otherwise all namespaces will be monitored
            #- "-namespaces=default,test"
            # If you set a default email, you can omit the field/annotation from Certificates/Ingresses
            - "-default-email=viae.it@gmail.com"
            # If you set a default provider, you can omit the field/annotation from Certificates/Ingresses
            - "-default-provider=googlecloud"
          volumeMounts:
            - name: data-sv-security
              mountPath: /var/lib/cert-manager-sv-security
            - name: google-application-credentials
              mountPath: /opt/google
      volumes:
        - name: data-sv-security
          persistentVolumeClaim:
            claimName: kube-cert-manager-sv-security-data
        - name: google-application-credentials
          secret:
            secretName: kube-cert-manager-sv-security-secret

anyone knows what I'm missing?

-- Vandeperre Maarten
google-cloud-platform
google-kubernetes-engine
kubernetes-ingress

1 Answer

4/16/2019

Your secret resource kube-cert-manager-sv-security-secret may contains a JSON file named kube-cert-manager-sv-security.json and it is not matched with GOOGLE_APPLICATION_CREDENTIALS value. You can confirm file name in the secret resource with kubectl get secret -oyaml YOUR-SECRET-NAME.

So you change the file path to the actual file name, cert-manager works fine.

- name: GOOGLE_APPLICATION_CREDENTIALS
#  value: /opt/google/kube-cert-manager.json
  value: /opt/google/kube-cert-manager-sv-security.json
-- translucens
Source: StackOverflow