I'm trying to figure out how to organize K8s namespaces for the development cluster.
Now we have multiple development namespaces (per team).
There are tons of pods (about 100-200) in a single namespace.
1-5 pods per feature-branch deployment.
We use Helm to make deployments. But some of the teammates say that it's hard to manage it.
The new idea is making a namespace per feature-branch deployment.
Now, I see the main issue is in TLS (and others) secrets sync sharing across namespaces. But it can be resolved by making a CronJob.
Are there any advantages or disadvantages to this approach?
Namespace per (review) feature-branch is the way to go.
Isolating each deployment group makes it manageable...
Also if you use the kubernetes dashboard the namespace overview will make more sense.
The idea of syncing secrets and configMaps by default is great if you are really reusing each and all of those, and they are never really namespace specific.
Generating secrets and configMaps dynamically at the moment of namespace creation and adding them then and there for that namespace and not sync is another way to go.
There is a reason why secrets and configMaps are isolated, namespace-specific and reside in a specific namespace. Secrets and configMaps can only be referenced by pods residing in the same namespace.
Just because you can sync doesn't mean you should...
If you still insist on syncing then have 1 group of 'syncable-shared-secrets", and another group that is namespace-specific.
https://kubernetes.io/docs/concepts/configuration/secret/#restrictions
https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/#restrictions
Its definitely a good approach to use namespaces for restricting the deployments to feature teams.
But deploying 50+ pods becomes difficult to manage per namespace, especially if the pods contains 10+ conatiners. So you will tend to manage 50X10=500 containers per deployment team.
1-5 pods per feature-branch deployment.
This is really a great way to go about using a namespace, but still yet you will have lots and lots of namespace to remember when you initally said you have arounf 100-200 pods.
Hope you are using rbac in k8s