I am installing nginx ingress controller (k8s.gcr.io/ingress-nginx/controller:v0.40.2) on a GKE cluster v1.17.12-gke.1504 with 3 nodes.
The ingress-nginx-controller is deployed as a Deployment; I don't want to deploy it as a DaemonSet because I'm going to have a cluster with many nodes in the future and it's going to be a waste of resources.
The creation of the kubernetes resources completes fine and the load balancer is created in GCE but the associated health check appears as failed.
I have accessed each of the cluster nodes to check the endpoint of the healt check and I see that it only works on the node where the ingress-nginx-controller pod has been created.
In the node where the pod has been raised it returns a 200:
$ curl localhost:32203/healthz -v
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 32203 (#0)
> GET /healthz HTTP/1.1
> Host: localhost:32203
> User-Agent: curl/7.64.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: application/json
< Date: Fri, 30 Oct 2020 09:03:42 GMT
< Content-Length: 111
<
{
"service": {
"namespace": "ingress-nginx",
"name": "ingress-nginx-controller"
},
"localEndpoints": 1
* Connection #0 to host localhost left intact
}* Closing connection 0
But on nodes where the pod is not up it returns a 503:
~ $ curl localhost:32203/healthz -v
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 32203 (#0)
> GET /healthz HTTP/1.1
> Host: localhost:32203
> User-Agent: curl/7.64.1
> Accept: */*
>
< HTTP/1.1 503 Service Unavailable
< Content-Type: application/json
< Date: Fri, 30 Oct 2020 09:03:20 GMT
< Content-Length: 111
<
{
"service": {
"namespace": "ingress-nginx",
"name": "ingress-nginx-controller"
},
"localEndpoints": 0
* Connection #0 to host localhost left intact
}* Closing connection 0
How can I get the health check to work without having to lift a pod from the ingress-nginx-controller on each node?
the problem was that I had configured externalTrafficPolicy: "Local"
in the service.