Gcp health check not working on a securised kibana with xPack

6/14/2019

I'm configuring a securised ELK cluster on GKE using the free xpack Basic authentication.

I've build a k8s StatefullSet elastcisearch manifest with xpack-security-enabled to true and so on. My kibana deployment has a readinessProbes pointing to '/api/status' with a Authorization header containing the correct base64 user:password encoding.

Here is my kibana deployment and the associated Ingress :

apiVersion: apps/v1
kind: Deployment
metadata:
  name: kibana
  namespace: kube-logging
  labels:
    app: kibana
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kibana
  template:
    metadata:
      labels:
        app: kibana
    spec:
      containers:
      - name: kibana
        image: docker.elastic.co/kibana/kibana:7.1.1
        livenessProbe:
          httpGet:
            path: /api/status
            port: 5601
            httpHeaders:
            - name: Authorization
              value: Basic blabla==
          initialDelaySeconds: 40
          timeoutSeconds: 5
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /api/status
            port: 5601
            httpHeaders:
              - name: Authorization
                value: Basic blabla==
          initialDelaySeconds: 40
          failureThreshold: 3
          timeoutSeconds: 5
          periodSeconds: 10
        resources:
          limits:
            cpu: 1000m
          requests:
            cpu: 100m
        env:
          - name: ELASTICSEARCH_URL
            value: http://elasticsearch:9200
          - name: ELASTICSEARCH_USERNAME
            value: kibana
          - name: ELASTICSEARCH_PASSWORD
            value: blabla
        ports:
        - name: kibana
          containerPort: 5601
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: kibana
  namespace: kube-logging
spec:
  backend:
    serviceName: kibana
    servicePort: 5601

When I apply the ingress, GCP create automatically an HTTP health check for the load balancer on the path '/' and expect a 200 code status. But kibana expect an Authorization header to respond a 200.

If I manually update the HTTP load balancer health check to a TCP one, everything is fine but GCP automatically revert my change and my kibana deployment become inaccessible again

-- Manu
elastic-stack
google-cloud-platform
google-kubernetes-engine

0 Answers