I am aware of the pros of having multiple containers in one pod but What are the cons of having multiple containers. We have a 20k pod prod requirement and our current infra supports max 900 pods for one namespace.which is best suitable approach to accommodate this request.
I'm not a devops guy, but from developer's perspective:
If you have many containers in POD:
you should be aware of: lifecycle issues - which container starts first, which ends first, and so forth
how to evaluate the "operability" of a pod as a logical unit. If there is one container in the pod, then its clear - if the container runs - that's ok. But if you have many, what kind of probes (readiness, liveness) do you define? What happens if one container is dead, is the whole pod still operational?
potentially its easier to find the place in kubernetes cluster for the replica of small pods than big ones (having many constainers and and therefor probably consuming much more resources)
you sacrifice the potential scalability flexibility. Say container A and B are in the same pod. What if you want to scale out only container A.
the same item as above applies to release management flexibility (rollout, canary releases, fallbacks to previous version - all that stuff)
you'll have to think about the solution of gathering logs/metrics from the containers inside the pod. This can be easy or tedious depending on the actual technology stack, but this is a point that you'll have to solve anyway. Arguably when you have multiple containers in the pod the solution might get more complicated.
somewhat less convenient operability with kubectl
. Ok this is minor, but still. You'll always have to add that -c
flag. Want to see the logs of the pod? Add -c <container-name>
. Want to do kubectl exec -it
- again, -c <container-name>
.
Of course sometimes running sidecar container is a valid case (for service mashes for example). But all-in-all its a tool for a job.
Interestingly, I've found an article that talks about giving side car containers a "special attitude". This can be somewhat relevant although from the question I understand that you don't consider "side cars" if I've got you right