K
Q

gke backend services in UNHEALTHY state when liveness and readiness probe is of type exec and healthcheck url required authentication

7/2/2021

I am trying to configure the application in the GKE cluster.

current situation

  • my application is using authenticated health checks.
  • to handle this I am using exec instead of httpGet. it enables me to use curl with authentication details (with HTTP get it is providing 401 (unauthorized error))
  • when I execute the helm chart I can see
    • pod is in a ready state with container 1/1 (earlier it was showing me (with HTTP get) as 0/1 )

code exec code for readiness and liveness probe.

exec [bash -c curl -G --fail --silent --show-error -u $AUTH_ADMIN_USERNAME:$AUTH_ADMIN_PASSWORD localhost:$PORT/actuator/health] delay=60s timeout=10s period=30s #success=1 #failure=8

expected this backend service (with readiness and liveness probe) in a healthy state

-- Ganesh Pol
google-cloud-platform
google-kubernetes-engine
kubernetes

Similar Questions

Connect to different VPC inside of Kubernetes pod
Kubernetes HPA node delete grace period
azure single vpn gateway for connecting multiple resources in different azure aks cluster
custom outgoing network path for kubernetes pod
digitalocean kubernetes loadbalancer

1 Answer

7/5/2021

It looks like you are having problem during authorizing your request. For example, on a server with token authentication configured, and anonymous access enabled, a request providing an invalid bearer token would receive a 401 Unauthorized error. You need to eliminate first the 401 error.

Here you can find a guide, how to set up Kubelet authentication:

By default, requests to the kubelet's HTTPS endpoint that are not rejected by other configured authentication methods are treated as anonymous requests, and given a username of system:anonymous and a group of system:unauthenticated. To disable anonymous access and send 401 Unauthorized responses to unauthenticated requests:

  • start the kubelet with the --anonymous-auth=false flag To enable X509 client certificate authentication to the kubelet's HTTPS endpoint:
  • start the kubelet with the --client-ca-file flag, providing a CA bundle to verify client certificates with
  • start the apiserver with --kubelet-client-certificate and --kubelet-client-key flags
  • see the apiserver authentication documentation for more details To enable API bearer tokens (including service account tokens) to be used to authenticate to the kubelet's HTTPS endpoint:
  • ensure the authentication.k8s.io/v1beta1 API group is enabled in the API server
  • start the kubelet with the --authentication-token-webhook and --kubeconfig flags
  • the kubelet calls the TokenReview API on the configured API server to determine user information from bearer tokens

You can find also a guide about Kubelet authorization:

There are many possible reasons to subdivide access to the kubelet API:

  • anonymous auth is enabled, but anonymous users' ability to call the kubelet API should be limited
  • bearer token auth is enabled, but arbitrary API users' (like service accounts) ability to call the kubelet API should be limited
  • client certificate auth is enabled, but only some of the client certificates signed by the configured CA should be allowed to use the kubelet API

See also:

-- Mikołaj Głodziak
Source: StackOverflow