One of our microservice(worker component - nature is short lived) is actually getting deployed on K8s pods in an autoscale fashion, sometimes this number goes to few thousands as well based upon load and this worker is bound to make connections with various persistent services, since these services come with some resource limit, so we're getting bottlenecked at access level, so my ask is, do we have some way in Kubernetes(similar to some sort of gateway/proxy) which narrow down multiplex requests to limit under resource limits. Let's say every pod makes a connection to MySQL server which has an active connection limit of 50, so if we keep spinning new pods(requirement of 1 MySQL connection), then we can not spin more than 50 pods concurrently.
You can setup a Pod Quota for a Namespace.
If you can spin those Pods on a separate Namespace, you can limit the number of running pods with creating a ResourceQuota object, lets call is quota-pod.yaml:
apiVersion: v1
kind: ResourceQuota
metadata:
name: pod-demo
spec:
hard:
pods: "2"kubectl create -f quota-pod.yaml --namespace=quota-pod-example
If you check kubectl get resourcequota pod-demo --namespace=quota-pod-example --output=yaml, you would get something like:
spec:
hard:
pods: "2"
status:
hard:
pods: "2"
used:
pods: "0"In the description of the for example 3 replica nginx deployment you would see:
Replicas: 1 desired | 1 updated | 1 total | 1 available | 0 unavailable
...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ScalingReplicaSet 2m deployment-controller Scaled up replica set nginx-1-7cb5b65464 to 3
Normal ScalingReplicaSet 16s deployment-controller Scaled down replica set nginx-1-7cb5b65464 to 1And kubectl get deployment nginx -o yaml would show:
...
status:
availableReplicas: 1
conditions:
- lastTransitionTime: 2018-12-05T10:42:45Z
lastUpdateTime: 2018-12-05T10:42:45Z
message: Deployment does not have minimum availability.
reason: MinimumReplicasUnavailable
status: "False"
type: Available
- lastTransitionTime: 2018-12-05T10:42:45Z
lastUpdateTime: 2018-12-05T10:42:45Z
message: 'pods "nginx-6bd764c757-4gkfq" is forbidden: exceeded quota: pod-demo,
requested: pods=1, used: pods=2, limited: pods=2'I recommend checking K8s docs Create a ResourceQuota for more information.