How to choose ingress controller in values.yaml
Context
I have a k3s (v1.19.3+k3s3) cluster and I am using the 'out of the box' traefik ingress controler
kubectl get svc -n kube-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kube-dns ClusterIP 10.43.0.10 <none> 53/UDP,53/TCP,9153/TCP 5d16h
metrics-server ClusterIP 10.43.127.152 <none> 443/TCP 5d16h
traefik-prometheus ClusterIP 10.43.73.131 <none> 9100/TCP 5d16h
traefik LoadBalancer 10.43.121.125 111.11.11.111 80:32492/TCP,443:31586/TCP 5d16h
kubectl -n kube-system describe svc traefik
Name: traefik
Namespace: kube-system
Labels: app=traefik
app.kubernetes.io/managed-by=Helm
chart=traefik-1.81.0
heritage=Helm
release=traefik
Annotations: meta.helm.sh/release-name: traefik
meta.helm.sh/release-namespace: kube-system
Selector: app=traefik,release=traefik
Type: LoadBalancer
IP: 10.43.121.125
LoadBalancer Ingress: 111.11.11.111
Port: http 80/TCP
TargetPort: http/TCP
NodePort: http 32492/TCP
Endpoints: 10.42.0.6:80
Port: https 443/TCP
TargetPort: https/TCP
NodePort: https 31586/TCP
Endpoints: 10.42.0.6:443
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
The issue
I do not find how to use this ingress controller using a helm chart (say, this ghost chart). In my understanding of how Helm charts work and the purpose they fulfill, Ingress templating allow to use different ingress controllers. Here is the chart's ingress.yaml
...
kind: Ingress
metadata:
name: {{ $fullName }}
labels:
{{- include "ghost.labels" . | nindent 4 }}
annotations:
nginx.ingress.kubernetes.io/proxy-body-size: {{ $maxSize }}
{{- with .Values.ingress.annotations }}
{{- toYaml . | nindent 4 }}
{{- end }}
...Based on this, I would say that I need to edit the annotations section of values.yaml to use traefik instead of default kubernetes.io/ingress.class: nginx
ingress:
enabled: true
annotations:
kubernetes.io/ingress.class: traefik # what should I use?
hosts:
- host: some.domain.io
paths:
- /
tls:
- secretName: chart-example-tls
hosts:
- some.domain.ioHowever, no matter what I try in annotations, helm install keep getting
Error: UPGRADE FAILED: failed to create resource: Internal error occurred: failed calling webhook "validate.nginx.ingress.kubernetes.io": Post "https://ingress-nginx-controller-admission.ingress-nginx.svc:443/networking/v1beta1/ingresses?timeout=10s": service "ingress-nginx-controller-admission" not foundas if helm were looking for an Nginx ingress controler, no matter what. Is this a possible limitation of charts (like this one) or am I misconceiving how Helm charts work? How can I use the default traefik ingress on this chart's deployment?
Similar Questions
2 Answers
No, as per I know - you are not able to chose exact ingress controller for these chart. Both Nginx and Traefik use same ports, so you have to define proper ingress controller in advance.
From chart documentation i see that :
This chart has been tested to work with NGINX Ingress, cert-manager, bla bla
And the helm options you have are:
From all the above written I have a question|proposition: Dont you want to switch to NGINX ingress controller in K3s?
Getting your cluster ready First order of business is that you need to get your cluster have no other ingress controllers as this would conflict with your current ingress controllers. This is because **NGINX and Traefik both listens to 80 and 443, which causes conflicts with each other, so make sure your K3s cluster only has one kind of the other.**
For the best results, your K3s cluster must be installed with the --no-deploy-traefik argument, which will cause the K3s cluster to deploy only with the bare container orchestrator without the default HTTP backend, which is Traefik.
To make sure you don't have a Traefik controller installed, run kubectl get deployments -n kube-system to see if Traefik is gone.
If you have verified Traefik is not there, then you may proceed with the next part.
Installing NGINX There's two ways to install NGINX, either from a Helm chart or a direct kubectl apply -f <file>
Using a Helm chart The NGINX ingress is available in the Helm Stable Charts, and if you have the stable registry already installed in your Helm CLI, simply run helm install stable/nginx-ingress.
You may also modify the installation using the following configurations stated here.
You were correct in your thinking, e.g. in a Helm chart you should be able to define by annotations the ingress correctly.
For example https://github.com/stefanprodan/podinfo/blob/master/charts/podinfo/templates/ingress.yaml
which defines annotations like:
{{- with .Values.ingress.annotations }}
annotations:
{{ toYaml . | indent 4 }}you could apply traefik specific annotations like
annotations:
traefik.ingress.kubernetes.io/router.entrypoints: websecure
traefik.ingress.kubernetes.io/router.tls.certresolver: myresolver
traefik.ingress.kubernetes.io/router.middlewares: kube-system-traefik-forward-auth@kubernetescrdIn the chart from your question: https://github.com/groundhog2k/helm-charts/blob/master/charts/ghost/templates/ingress.yaml#L16
the annotation
nginx.ingress.kubernetes.io/proxy-body-sizeis hardcoded. That shouldn't stop the deployment to work properly. You are stating that it looks as Helm is "looking" for Nginx ingress controller which is not true, afaik. Helm is not looking for anything. Also apart from that specific reference in annotations, i can't find anything nginx hardcoded, values are commented out. https://github.com/groundhog2k/helm-charts/blob/master/charts/ghost/values.yaml#L60 So the correct answer is that you still had a faulty nginx ingress deployment.