What are the dis advantages of putting multiple containers in a pod?

5/7/2020

I am aware of the pros of having multiple containers in one pod but What are the cons of having multiple containers. We have a 20k pod prod requirement and our current infra supports max 900 pods for one namespace.which is best suitable approach to accommodate this request.

-- Karthik Reddy
containers
kubernetes

1 Answer

5/7/2020

I'm not a devops guy, but from developer's perspective:

If you have many containers in POD:

  • you should be aware of: lifecycle issues - which container starts first, which ends first, and so forth

  • how to evaluate the "operability" of a pod as a logical unit. If there is one container in the pod, then its clear - if the container runs - that's ok. But if you have many, what kind of probes (readiness, liveness) do you define? What happens if one container is dead, is the whole pod still operational?

  • potentially its easier to find the place in kubernetes cluster for the replica of small pods than big ones (having many constainers and and therefor probably consuming much more resources)

  • you sacrifice the potential scalability flexibility. Say container A and B are in the same pod. What if you want to scale out only container A.

  • the same item as above applies to release management flexibility (rollout, canary releases, fallbacks to previous version - all that stuff)

  • you'll have to think about the solution of gathering logs/metrics from the containers inside the pod. This can be easy or tedious depending on the actual technology stack, but this is a point that you'll have to solve anyway. Arguably when you have multiple containers in the pod the solution might get more complicated.

  • somewhat less convenient operability with kubectl. Ok this is minor, but still. You'll always have to add that -c flag. Want to see the logs of the pod? Add -c <container-name>. Want to do kubectl exec -it - again, -c <container-name>.

Of course sometimes running sidecar container is a valid case (for service mashes for example). But all-in-all its a tool for a job.

Interestingly, I've found an article that talks about giving side car containers a "special attitude". This can be somewhat relevant although from the question I understand that you don't consider "side cars" if I've got you right

-- Mark Bramnik
Source: StackOverflow