I have an application with 5 microservices (iam, courses...). I want to know which is the best approach to migrate them to kubernetes. I was thinking to create namespaces by enviroment as google recomendes: 1. prod 2. dev 3. staging
then I thought that may be better create namespace by environment and microservices. 1. iam-prod 2. iam-dev 3. iam-staging 1. courses-prod 2. courses-dev 3. courses-staging ... but this approach can be a little bit difficult to handle. Because I need to communicate between each other.
Which approach do you think that is better?
You can run multiple microservices on the same namespace. So, I would go with prod, dev and staging namespaces where you can have one or multiple instances of each micro-service.
yet, If you want to use separate namespaces for separate microservices environments, they still can communicate using service. The DNS URL will be, SERVICE_NAME.NAMESPACE.SVC. ref: https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/
If you go with second approach you will create unnecessary complexity without achieving any benefit. Also think of situation if your micro-services grows ,are you going to create new cluster for each one.This is not at all recommended.
Concept of Namespace should not be linked to applications but it is related to users.Refer k8 doc as below
"Namespaces are intended for use in environments with many users spread across multiple teams, or projects. For clusters with a few to tens of users, you should not need to create or think about namespaces at all. Start using namespaces when you need the features they provide."
Also even if first approach is recommended, please have separate cluster for prod as this should be more secure and highly available with proper disaster recovery plan ready and tested.
None of the above are ideal solutions. I’ll go over why.
Namespaces are the easiest boundary to use for managing RBAC permissions. In general, you will want to use the pre-provisioned admin and editor cluster roles to constrain access for users to use namespaces. This means people and services that share namespaces also share visibility of secrets. So the namespace becomes the blast radius for compromising secrets.
In order to reduce the blast radius of secrets exposure you can either micromanage resource level role binding (which is unreasonable overhead without additional automation and tooling) or segregate services across namespaces so that only tightly couple services share a namespace.
Kubernetes resource isolation is relatively poor between namespaces. There’s no way to force a namespace to deploy into a different node pool than another namespace without custom admission controllers. So resource isolation is effectively opt-in, which is both insecure and unenforceable.
Because of this, is actually more secure and better resource isolated to have different environments (dev, staging, prod) in seperate K8s clusters all together. But this is obviously more expensive and more management overhead. So it’s only cost effective when you have many services and enough resource usage to justify the added overhead.
The consequence of poor resource isolation is that your dev and staging workloads can effectively DOS your prod workloads simply by using shared resources. CPU/memory/disk are the obvious culprits. These can be enforced by custom admission controllers. But the more insidious problem is sharing ingress proxies, load balancer, and networking, which is harder to isolate between namespaces.
Another consequence of poor isolation is that dev services with poor security can be compromised, allowing horizontal access to prod services. Realistically, no one deploys dev apps as production ready and secure. So without hard isolation, your security is at risk too.
Quotas are managed at the namespace level. So if you want to isolate quota by environment AND team, you can’t use namespaces for both. And if you want to have quota by project, you’d need a project per namespace. The only way to handle all three is with multiple clusters, multiple namespaces, and multiple node pools with custom deployment/admission enforcement of that creates a makeshift hierarchy or matrix.
Namespaces are flat. If you use them for env you can’t use them for org or team level access control. If you use them for team level access control your engineers can use them for component/project/system level abstraction boundaries. You can only choose one or the chaos will be unmanageable.
Unfortunately, the namespace abstraction is being used for 3 or 4 use cases in the Kubernetes community, and it’s the not really ideal for any of them. So either you pick an non-ideal use case to optimize for or you manage multiple clusters and write a bunch of custom automation to handle all the use cases.
Go with one name space for each environment. You can also define resource quota per names paces. That way each application environment can be independently managed
Just like the other answer, you should create namespace isolation for prod, dev and staging. This will ensure a couple of nuances are taken care of...