I have a nodejs microservice running on GKE that serves html/js assets on our clients' sites. The configuration for this is as follows: Ingress > NodePort Service > Pods (4 replicas). If for some season the pods are failing or the Node goes down, the microservice will then take the full 30s to timeout. That causes delayed page load times for our clients. What I need to happen is in the event of a failure that the NodePort service cuts off the connection or responds with a 502 error after 2 seconds.

I've tried two ways of manipulating the same setting. The first is creating a BackendConfig following the docs: https://cloud.google.com/kubernetes-engine/docs/how-to/configure-backend-service

My config looks like this:

apiVersion: cloud.google.com/v1beta1
kind: BackendConfig
metadata:
  name: timeout-config
spec:
  timeoutSec: 2

Then I connected it to my service like this:

apiVersion: v1
kind: Service
metadata:
  annotations:
    beta.cloud.google.com/backend-config: '{"ports": {"80":"timeout-config"}}'
  labels:
    run: <MICROSERVICE>
  name: <MICROSERVICE>
spec:
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 3000
  selector:
    run: <MICROSERVICE>
  type: NodePort

I tested this configuration by having my microservice alternate between returning 200 and 502 for the health check endpoint every 30 seconds. That caused the pod to be restarted about every 30s which would cut off communication with the pod. I expected that once it was being restarted that the request would timeout and default to the 2-second setting I had configured. However, it still took 30 seconds to receive the 502 error.

The second method I tried was to set the timeout to 2 seconds using gcloud. I did so by following the docs here: https://github.com/kubernetes/ingress-gce/blob/e72479ba461fedae5fc5bf64999f28ba3125004d/examples/websocket/README.md#change-backend-timeout

That method did not work either. What other methods can I use to get my service to timeout after 2 seconds on GKE?