We are running a AKS Kubernetes cluster on Azure. I'm using the "NGINX Ingress Controller" and "cert-manager" for routing and certificate generation (through Let's Encrypt). I followed the basic setup advice from the Microsoft documentation: https://docs.microsoft.com/en-us/azure/aks/ingress-tls
When visiting our page in a web-browser, we notice nothing out of the ordinary at first - HTTPS was working fine. The browser can validate the Let's Encrypt certificate. However, we noticed later on that the ingress controller actually serves two certificates (one of which has a common name: "Kubernetes Ingress Controller Fake Certificate" and an Alternative name: "ingress.local"): https://www.ssllabs.com/ssltest/analyze.html?d=test-aks-ingress.switzerlandnorth.cloudapp.azure.com&hideResults=on
Long story short - yesterday, I tried everything from re-installing the Nginx-ingress and cert-manager to starting a new Azure Kubernetes Service from scratch, but every time I end up in the same situation.
I have read many of the discussions from people experiencing similar problems. Typically, they are a bit different though, as they don't actually see a valid certificate at all. I confirmed that we are using the production Let's Encrypt ClusterIssuer
:
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
name: letsencrypt
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: ***@***
privateKeySecretRef:
name: letsencrypt
solvers:
- http01:
ingress:
class: nginx
podTemplate:
spec:
nodeSelector:
"kubernetes.io/os": linux
I also created a new test-app with test-ingress:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: hello-world-ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$1
nginx.ingress.kubernetes.io/use-regex: "true"
cert-manager.io/cluster-issuer: letsencrypt
spec:
tls:
- hosts:
- test-aks-ingress.switzerlandnorth.cloudapp.azure.com
secretName: tls-secret
rules:
- host: test-aks-ingress.switzerlandnorth.cloudapp.azure.com
http:
paths:
- backend:
serviceName: aks-helloworld-one
servicePort: 80
path: /hello-world-one(/|$)(.*)
From my understanding there is usually some issue with the secret
for the people that have reported this previously. Here I assume that the ClusterIssuer
will generate the relevant certificates and store them in tls-secret
, which has been generated automatically:
Name: tls-secret
Namespace: test
Labels: <none>
Annotations: cert-manager.io/alt-names: test-aks-ingress.switzerlandnorth.cloudapp.azure.com
cert-manager.io/certificate-name: tls-secret
cert-manager.io/common-name: test-aks-ingress.switzerlandnorth.cloudapp.azure.com
cert-manager.io/ip-sans:
cert-manager.io/issuer-group: cert-manager.io
cert-manager.io/issuer-kind: ClusterIssuer
cert-manager.io/issuer-name: letsencrypt
cert-manager.io/uri-sans:
Type: kubernetes.io/tls
Data
====
tls.crt: 3530 bytes
tls.key: 1675 bytes
Maybe what I am still confused about is the different secrets / certificates at play here. The cert-manager
operates in the cert-manager
namespace and creates a letsencrypt
secret there, while my test-setup is running everything else in a test
namespace (including the ingress controller).
UPDATE But what is the actual problem here? Everything "just works" in a normal browser, right? Unfortunately, the real problem is that connections do not work for a specific client application, which may not have SNI support.
Is there a way to not have a default certificate? How would I change the configuration here to provide the "Let's Encrypt" signed certificate by default - is that possible?
It is expected behavior. By default Ingress controller creates self-signed certificate with CN indicating it's fake one. This is used when a request doesn't match to rules defined in Ingress. So when we access this URL from browser, it returns correct certificate but with openssl s_client without servername field, it doesn't match the rule defined in Ingress and goes to default backend and returns self-signed certificate.
You can also specify default certificate for Ingress. Refer https://github.com/kubernetes/ingress-nginx/issues/4674 for more details.