K8S & Traefik on DO with a valid SSL/HTTPS but got 502 gateway and strange logs :(

6/18/2019

I guess it's my first question on SO.

This issue drive me crazy and I don't know what to do or search right now.

I'm trying to setup a k8s cluster on DigitalOcean with HELM and Traefik.

I followed the tutorial of this blog post

I've opened a ticket on DO to have some help for a valid SSL/HTTTS certificate and it's seems to work.

BUT when I have a valid certificate I got a "502 Bad Gateway The server returned an invalid or incomplete response." for every single domain name configured.

When I disable the SSL and remove the certificate, everything works, I can access to every host but you know... no certificate is bad! :)

Moreover, when i activate the SSL/HTTPS on the helm-values.yaml, I got strange logs !

Here an extract, I don't even know what's happening here !

{"level":"debug","msg":"Received Kubernetes event kind *v1.Endpoints","time":"2019-06-17T23:44:55Z"}
{"level":"debug","msg":"Skipping Kubernetes event kind *v1.Endpoints","time":"2019-06-17T23:44:55Z"}
{"level":"debug","msg":"http: TLS handshake error from 10.244.0.1:38666: tls: first record does not look like a TLS handshake","time":"2019-06-17T23:44:55Z"}
10.244.2.1 - - [17/Jun/2019:23:44:55 +0000] "GET / HTTP/1.0" 302 5 "-" "-" 3209 "entrypoint redirect for http" "/" 0ms
10.244.0.1 - - [17/Jun/2019:23:44:55 +0000] "GET / HTTP/1.0" 302 5 "-" "-" 3210 "entrypoint redirect for http" "/" 0ms
10.244.0.1 - - [17/Jun/2019:23:44:55 +0000] "GET / HTTP/1.0" 302 5 "-" "-" 3211 "entrypoint redirect for http" "/" 0ms
10.244.2.1 - - [17/Jun/2019:23:44:55 +0000] "GET / HTTP/1.0" 302 5 "-" "-" 3212 "entrypoint redirect for http" "/" 0ms
{"level":"debug","msg":"Received Kubernetes event kind *v1.Endpoints","time":"2019-06-17T23:44:55Z"}
{"level":"debug","msg":"Skipping Kubernetes event kind *v1.Endpoints","time":"2019-06-17T23:44:55Z"}
10.135.133.233 - - [17/Jun/2019:23:44:55 +0000] "GET / HTTP/1.0" 302 5 "-" "-" 3213 "entrypoint redirect for http" "/" 0ms
10.244.2.1 - - [17/Jun/2019:23:44:56 +0000] "GET / HTTP/1.0" 302 5 "-" "-" 3214 "entrypoint redirect for http" "/" 0ms
10.244.0.1 - - [17/Jun/2019:23:44:56 +0000] "GET / HTTP/1.0" 302 5 "-" "-" 3215 "entrypoint redirect for http" "/" 0ms
10.244.0.1 - - [17/Jun/2019:23:44:56 +0000] "GET / HTTP/1.0" 302 5 "-" "-" 3216 "entrypoint redirect for http" "/" 0ms
10.244.2.1 - - [17/Jun/2019:23:44:57 +0000] "GET / HTTP/1.0" 302 5 "-" "-" 3217 "entrypoint redirect for http" "/" 0ms
{"level":"debug","msg":"Received Kubernetes event kind *v1.Endpoints","time":"2019-06-17T23:44:57Z"}
{"level":"debug","msg":"Skipping Kubernetes event kind *v1.Endpoints","time":"2019-06-17T23:44:57Z"}
10.135.162.241 - - [17/Jun/2019:23:44:57 +0000] "GET / HTTP/1.0" 302 5 "-" "-" 3218 "entrypoint redirect for http" "/" 0ms
{"level":"debug","msg":"Received Kubernetes event kind *v1.Endpoints","time":"2019-06-17T23:44:57Z"}
{"level":"debug","msg":"Skipping Kubernetes event kind *v1.Endpoints","time":"2019-06-17T23:44:57Z"}
10.135.162.241 - - [17/Jun/2019:23:44:57 +0000] "GET / HTTP/1.0" 302 5 "-" "-" 3219 "entrypoint redirect for http" "/" 0ms
10.135.133.233 - - [17/Jun/2019:23:44:58 +0000] "GET / HTTP/1.0" 302 5 "-" "-" 3220 "entrypoint redirect for http" "/" 0ms
10.244.0.1 - - [17/Jun/2019:23:44:58 +0000] "GET / HTTP/1.0" 302 5 "-" "-" 3221 "entrypoint redirect for http" "/" 0ms
10.244.2.1 - - [17/Jun/2019:23:44:58 +0000] "GET / HTTP/1.0" 302 5 "-" "-" 3222 "entrypoint redirect for http" "/" 0ms
10.244.0.1 - - [17/Jun/2019:23:44:58 +0000] "GET / HTTP/1.0" 302 5 "-" "-" 3223 "entrypoint redirect for http" "/" 0ms
10.244.2.1 - - [17/Jun/2019:23:44:58 +0000] "GET / HTTP/1.0" 302 5 "-" "-" 3224 "entrypoint redirect for http" "/" 0ms
10.135.133.233 - - [17/Jun/2019:23:44:58 +0000] "GET / HTTP/1.0" 302 5 "-" "-" 3225 "entrypoint redirect for http" "/" 0ms
10.244.2.1 - - [17/Jun/2019:23:44:59 +0000] "GET / HTTP/1.0" 302 5 "-" "-" 3226 "entrypoint redirect for http" "/" 0ms
10.244.0.1 - - [17/Jun/2019:23:44:59 +0000] "GET / HTTP/1.0" 302 5 "-" "-" 3227 "entrypoint redirect for http" "/" 0ms
{"level":"debug","msg":"Received Kubernetes event kind *v1.Endpoints","time":"2019-06-17T23:44:59Z"}
{"level":"debug","msg":"Skipping Kubernetes event kind *v1.Endpoints","time":"2019-06-17T23:44:59Z"}
{"level":"debug","msg":"Received Kubernetes event kind *v1.Endpoints","time":"2019-06-17T23:44:59Z"}
{"level":"debug","msg":"Skipping Kubernetes event kind *v1.Endpoints","time":"2019-06-17T23:44:59Z"}
10.244.0.1 - - [17/Jun/2019:23:44:59 +0000] "GET / HTTP/1.0" 302 5 "-" "-" 3228 "entrypoint redirect for http" "/" 0ms
10.244.2.1 - - [17/Jun/2019:23:45:00 +0000] "GET / HTTP/1.0" 302 5 "-" "-" 3229 "entrypoint redirect for http" "/" 0ms
10.135.162.241 - - [17/Jun/2019:23:45:00 +0000] "GET / HTTP/1.0" 302 5 "-" "-" 3230 "entrypoint redirect for http" "/" 0ms
10.135.162.241 - - [17/Jun/2019:23:45:00 +0000] "GET / HTTP/1.0" 302 5 "-" "-" 3231 "entrypoint redirect for http" "/" 0ms
Logs from 6/17/19 11:44 PM to 6/17/19 11:45 PM UTC

I tried a lot of things, I'm clearly not an expert in k8s right now !

Here a modified version of my helm-values.yaml :

ssl:
  enabled: true        
  enforced: true       
  insecureSkipVerify: false 
  permanentRedirect: false 

debug:
  enabled: true

accessLogs:
  enabled: true

acme:
  enabled: true
  staging: true
  logging: true
  email: email@dontworry.co
  challengeType: "dns-01"   
  dnsProvider:
    name:  digitalocean
    digitalocean:
      DO_AUTH_TOKEN: "MY_DO_AUTH_TOKEN"

  persistence:
     enabled: true

  domains:
    enabled: true
    domainsList:
      - main: "*.mydomain.cloud"

service:
  annotations:
    service.beta.kubernetes.io/do-loadbalancer-certificate-id: "A_CERTIFICATE_ID_VALID"
    service.beta.kubernetes.io/do-loadbalancer-protocol: "https"
    service.beta.kubernetes.io/do-loadbalancer-algorithm: "round_robin"
    service.beta.kubernetes.io/do-loadbalancer-tls-ports: "443"

metrics:
  prometheus:
    enabled: false

rbac:
  enabled: true

replicas: 1

dashboard:
  enabled: true
  domain: traefik.mydomain.cloud
  auth:
    basic:
      traefik: # generated with : htpasswd -c traefik traefik
  ingress:
    annotations:
      kubernetes.io/ingress.class: traefik

And here one Ingress used to access to my API :

---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: traefik
    traefik.frontend.rule.type: PathPrefixStrip
  name: domain-network
spec:
  rules:
  - host: 
    http:
      paths:
      - path: /
        backend:
          serviceName: domain-service-gateway
          servicePort: http

Of course, domain-service-gateway is a service, the exposed port is 80.

The whole cluster is available without SSL/HTTPS settings, but when enable... 502 bad gateway!

It should works but I guess I'm missing something important here.

I hope the gods of StackOverflow will hear my prayers :)

Thx a lot for reading everything and trying to be help me!

-- David
kubernetes
kubernetes-helm
ssl
traefik

0 Answers