I have a pod running inside k8s that takes takes requests to register VMs into the k8s service registry for service discovery and istio traffic control. This pod is running in GKE and using glcoud credentials to connect to the pod it is currently running in.

This endpoint works fine for a while, but if it's receiving a large amount of subsequent requests, eventually it starts throwing the error:

Unable to connect to the server: read tcp 10.92.2.69:47436->xx.xx.xx.xx:443: read: connection reset by peer

The only thing I can think of is that there is some sort of rate limiting on the kube-apiserver or some proxy sitting between the kube-apiserver and my pod. Does anyone have any experience with this issue?