I have a single node minikube cluster with 5 pods of same docker image. I want to make 5 or more requests at same time to this node and want requests to be distributed across pods. But when I make requests equal to or greater than the pods in node, some of the requests fail (500 internal server error). I confirmed the request did not reach the pod through their logs so I think that the node could not handle that many requests at same time.

Is there a way I can make parallel requests to a node and have them properly distributed across the pods and buffered in case there are no available pods.

Used this tutorial to create my cluster

Used this for scaling using following cmd

kubectl scale deployment --replicas=5

I am using Python requests lib to request the node ip. For parallel requests I am using multiprocessing pool with 5 processes

def fn_to_req_node(string):
     requests.post(http://nodeip:nodeport, data=json.dumps(string), headers=headers)

pool= Pool(processes=5)
pool.apply_async(fn_to_req_node, (string1))
pool.apply_async(fn_to_req_node, (string2))
pool.apply_async(fn_to_req_node, (string3))
pool.apply_async(fn_to_req_node, (string4))
pool.apply_async(fn_to_req_node, (string5))

UPDATE : Tried on GCE with external Load balancer but same issue - 500 internal server error for some requests.