I'm trying to setup TLS for a service that's available outside a Kubernetes cluster (AWS EKS). With cert-manager, I've successfully issued a certificate and configured ingress, but I'm still getting error NET::ERR_CERT_AUTHORITY_INVALID
. Here's what I have:
namespace tests
and hello-kubernetes
in it (both deployment and service have name hello-kubernetes-first
, serivce is ClusterIP with port
80 and targetPort
8080, deployment is based on paulbouwer/hello-kubernetes:1.8
, see details in my previous question)
DNS and ingress configured to show the service:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: hello-kubernetes-ingress
namespace: tests
spec:
ingressClassName: nginx
rules:
- host: test3.projectname.org
http:
paths:
- path: "/"
pathType: Prefix
backend:
service:
name: hello-kubernetes-first
port:
number: 80
Without configuring TLS, I can access test3.projectname.org via http and see the service (well, it tries to redirect me to https, I see NET::ERR_CERT_AUTHORITY_INVALID
, I go to insecure anyway and see the hello-kubernetes page).
note: I have nginx-ingress ingress controller; it was installed before me via the following chart:
apiVersion: v2
name: nginx
description: A Helm chart for Kubernetes
type: application
version: 4.0.6
appVersion: "1.0.4"
dependencies:
- name: ingress-nginx
version: 4.0.6
repository: https://kubernetes.github.io/ingress-nginx
and the values overwrites applied with the chart differ from the original ones mostly in extraArgs
: default-ssl-certificate: "nginx-ingress/dragon-family-com"
is uncommneted
cert-manager installed via kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.5.4/cert-manager.yaml
ClusterIssuer created with the following config:
apiVersion: cert-manager.io/v1 kind: ClusterIssuer metadata: name: letsencrypt-backoffice spec: acme: server: https://acme-staging-v02.api.letsencrypt.org/directory
# use https://acme-v02.api.letsencrypt.org/directory after everything is fixed and works
privateKeySecretRef: # this secret is created in the namespace of cert-manager
name: letsencrypt-backoffice-private-key
# email: <will be used for urgent alerts about expiration etc>
solvers:
# TODO: add for each domain/second-level domain/*.projectname.org
- selector:
dnsZones:
- test.projectname.org
- test2.projectname.org
- test3.projectname.org
http01:
ingress:
class: nginx
certificate in the tests
namespace. It's config is
apiVersion: cert-manager.io/v1 kind: Certificate metadata: name: letsencrypt-certificate-31 namespace: tests spec: secretName: tls-secret-31 issuerRef: kind: ClusterIssuer name: letsencrypt-backoffice commonName: test3.projectname.org dnsNames: - test3.projectname.org
Now, certificate is ready (kubectl get certificates -n tests
tells that) and to apply it, I add this to ingress's spec:
tls:
- hosts:
- test3.projectname.org
secretName: tls-secret-31
However, when I try to open test3.projectname.org via https, it still shows me the NET::ERR_CERT_AUTHORITY_INVALID
error. What am I doing wrong? How to debug this? I've checked up openssl s_client -connect test3.projectname.org:443 -prexit
* and it shows the following chain:
0 s:CN = test3.projectname.org
i:C = US, O = (STAGING) Let's Encrypt, CN = (STAGING) Artificial Apricot R3
1 s:C = US, O = (STAGING) Let's Encrypt, CN = (STAGING) Artificial Apricot R3
i:C = US, O = (STAGING) Internet Security Research Group, CN = (STAGING) Pretend Pear X1
2 s:C = US, O = (STAGING) Internet Security Research Group, CN = (STAGING) Pretend Pear X1
i:C = US, O = (STAGING) Internet Security Research Group, CN = (STAGING) Doctored Durian Root CA X3
and tells, among other output
Verification error: unable to get local issuer certificate
Unfortunately, I haven't found anything useful to try further, so any help is appreciated.
The reason that your certificates didn't work, it not because you used staging server, but because you didn't specify the tls object within the Ingress rules. Certbot's staging exists only for the purpose of testing, and for not "ban" you while you testing things out if you request more than 5 certificates/hour. When you verify that everything works as expected, you can use the normal non-staging server.
This is how it should be done:
Cluster Issuer object:
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: user@example.com
privateKeySecretRef:
name: letsencrypt-prod
solvers:
- http01:
ingress:
class: nginx
Ingress Object:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: hello-kubernetes-ingress
namespace: tests
labels:
app: hello-kubernetes-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
tls:
- hosts:
- test3.projectname.org
secretName: hello-tls
rules:
- host: test3.projectname.org
http:
paths:
- pathType: ImplementationSpecific
path: "/"
backend:
service:
name: hello-kubernetes-ingress
port:
number: 80
The certificate and the key, are stored in a secret called "hello-tls" which you didn't also specify in your initial example, hence the failure you was receiving.
Your ClusterIssuer refers to LetsEncrypt staging issuer. Remove that setting / the default should use their production setup. As pointed out in comments: https://acme-v02.api.letsencrypt.org/directory
Deleting the previously generated secrets or switching to new secrets should ensure your certificates would be re-generated, using the right issuer.
The staging issuer could be useful testing LetsEncrypt integration, it shouldn't be used otherwise.
Following the suggestion from SYN, I've fixed this by
switching ACME server in ClusterIssuer config from https://acme-staging-v02.api.letsencrypt.org/directory
to https://acme-v02.api.letsencrypt.org/directory
. The idea of the staging server seems to be: allow to debug certificate issuing (so that kubectl get certificate [-n <namespace>]
shows that READY
= true
) without providing actual trusted certificates; after certificate issuing is ok, one has to switch to the main server to get production certificates.
Updating certificates, tls secrets and ingress configs. Well, I'm not sure if there's a way to actually update certificates; instead, I've created new ones, which created new secrets, and then updated ingress configs (just secrets' names)