k8s nginx ingress kibana

3/2/2019

I have a k8s cluster on which i have deployed ELK my kibana deployment and service looks like

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    service: kibana
  name: kibana
spec:
  replicas: 1
  selector:
    matchLabels:
      service: kibana
  strategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        service: kibana
    spec:
      containers:
        - image: docker.elastic.co/kibana/kibana:6.6.0
          name: kibana
          ports:
            - containerPort: 5601
          resources:
            requests:
              memory: 1Gi
            limits:
              memory: 1Gi
      restartPolicy: Always
      imagePullSecrets:
        - name: regcred
---
apiVersion: v1
kind: Service
metadata:
  labels:
    service: kibana
  name: kibana
spec:
  ports:
    - name: "5601"
      port: 5601
      targetPort: 5601
  selector:
    service: kibana
  type: NodePort

and the nginx ingress looks like

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: loadbalancer-https
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: true
    nginx.ingress.kubernetes.io/force-ssl-redirect: true
    nginx.ingress.kubernetes.io/secure-backends: "true"
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/app-root: /
    nginx.org/ssl-services: "kibana"
    kubernetes.io/ingress.class: nginx
spec:
  tls:
    - hosts:
        - kibana.some.com
      secretName: secret
  rules:
    - host: kibana.some.com
      http:
        paths:
          - path: /
            backend:
              serviceName: kibana
              servicePort: 5601

But i get 502 Bad Gateway and if i look on the nginx ingress logs i see

2019/03/02 01:25:09 [error] 875#875: *470787 upstream prematurely closed connection while reading response header from upstream, client: 10.138.82.98, server: kibana.some.com, request: "GET /favicon.ico HTTP/2.0", upstream: "http://10.244.2.86:5601/favicon.ico", host: "kibana.some.com", referrer: "https://kibana.some.com/"

It seems pretty straightforward but i don't know what im missing here. I would appreciate some help.

Thanks

-- bitgandtter
docker
kibana
kubernetes
nginx-ingress

1 Answer

3/2/2019

It should be related to your nginx.ingress.kubernetes.io/secure-backends: "true" configuration in your ingress. Did you configure HTTPS/TLS support for your Kibana in Pod? Otherwise I would suggest removing this annotation.

Also, please note that nginx.ingress.kubernetes.io/secure-backends: "true" annotation is deprecated.

Ref:

-- Fei
Source: StackOverflow