I have >20 teams of 4-5 developers & QA’s each working on a separate release running in parallel. Each release is a feature that involves a change in 5-6 microservices. I currently have 3 environments (DEV, SIT, and UAT separated as namespaces) on one on-prem cluster & one dedicated on-prem cluster for Prod. With just one Dev env, the releases are throttled as only one release is deployed in Dev and only when it's promoted to SIT does the next feature release get deployed to dev. This sequential promotion queues up the releases and delays them. With over 100 microservices, am sure this issue isn’t unique and there is a well-optimized solution already in use by large development teams. Is one env per feature a good model to handle the above scenario?
Two things that would help are extensive use of test automation and feature toggles.
Create a continuous integration release pipeline that deploys to DEV, runs a suite of regression tests and then if successful automatically deploys to SIT. Even with just one DEV environment the limitation on speed will be purely how fast the test automation suite can run.
The feature toggles help because they allow you to release more frequently, even during the development of new functionality.