I am using the default installation of k3s (release v1.17.0+k3s.1) and verified it is working correctly on my Raspberry Pi cluster.
After installing kubernetes-dashboard using the recommended instructions (https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/) I confirmed this works, after I exposed it outside the cluster using the k3s traefik-based load balancer.
However, when I try exposing it using the traefik ingress controller I was unable to connect to kubernetes dashboard, with the browser showing a HTTP 500 error i.e. Internal Server Error. Below is my ingress yaml and kubernetes-dashboard pod logs showing the TLS certificate error. I believe the certificate error is due to traefik not knowing about (and trusting) the TLS certificate used by kubernetes dashboard.
a. k8s-dashboard-Ingress
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kubernetes-dashboard
annotations:
kubernetes.io/ingress.class: traefik
traefik.ingress.kubernetes.io/ssl-redirect: "true"
spec:
rules:
- host: k8s-services
http:
paths:
- path: /k8s-dashboard
backend:
serviceName: kubernetes-dashboard
servicePort: 443
Note. the domain "k8s-services" domain is defined in my pi cluster machines and browsing computers host file.
b. Kubernetes dashboard pod logs
...
2020/01/18 06:04:19 Auto-generating certificates
2020/01/18 06:04:19 Metric client health check failed: the server is currently unable to handle the request (get services dashboard-metrics-scraper). Retrying in 30 seconds.
2020/01/18 06:04:19 Successfully created certificates
2020/01/18 06:04:19 Serving securely on HTTPS port: 8443
2020/01/18 06:04:49 Successful request to sidecar
2020/01/18 06:23:28 http: TLS handshake error from 10.42.0.113:52698: remote error: tls: bad certificate
2020/01/18 06:24:33 http: TLS handshake error from 10.42.0.113:52886: remote error: tls: bad certificate
2020/01/18 06:24:41 http: TLS handshake error from 10.42.0.113:52908: remote error: tls: bad certificate
2020/01/18 06:24:47 http: TLS handshake error from 10.42.0.113:52926: remote error: tls: bad certificate
2020/01/18 06:24:53 http: TLS handshake error from 10.42.0.113:52948: remote error: tls: bad certificate
2020/01/18 06:25:37 http: TLS handshake error from 10.42.0.113:53076: remote error: tls: bad certificate
2020/01/18 06:25:41 http: TLS handshake error from 10.42.0.113:53090: remote error: tls: bad certificate
2020/01/18 06:25:44 http: TLS handshake error from 10.42.0.113:53102: remote error: tls: bad certificate
2020/01/18 06:31:22 http: TLS handshake error from 10.42.0.113:54144: remote error: tls: bad certificate
2020/01/18 06:31:29 http: TLS handshake error from 10.42.0.113:54158: remote error: tls: bad certificate
...
My issue seems very similiar to the following Stack Overflow issue: Kubernetes dashboard through Ingress
My first attempt to address this issue is to add the "ssl.insecureSkipVerify: 'true'" to the traefik.toml file in k3s. Unfortunately I could not find a global traefik.toml file (and traefik seems to be installed using a Helm Chart).
Please let me know ways of updating the "ssl.insecureSkipVerify" setting in traefik, or other alternate solutions.