I have a kubernetes cluster with a nodejs API and two mongodb replica sets. And it's working great, but we are still in development and having to wait for the deployment pipeline to finish every time we make a change is just taking to long.

So we expose our mongodb primary server via loadbalancers so we can access it from our local development environment without the need to push every change to the cluster just to test something.

The problem now is that we want the environments to be identical to make sure everything works (and because we want to use mongodb transactions). So we want to connect to the replicaset instead of just the primary mongodb pod.

Creating a loadbalancer for every single pod in the replicaset and listing them all as host seems to work, but as we have to pay per public IP address it quickly adds up.

So my question is if there is a way that would allow us to only use one IP address per replicaset, and if so, how to do that.

Edit:

What I tried with Ingress so far:

• Connecting Domains to the MongoDB Pods (e.g. mongo1.my-site.com, mongo2.my-site.com)
• This does not work because MongoDB need TCP connections
• In the ingress config you can only bind one port to one service
• Hooking up port 27017 to MongoDB1 Primary
• Hooking up port 27018 to MongoDB2 Primary
• MongoDB1 connects with mongo --host "mongo1.my-site.com:27017" --authenticationDatabase my_database --username my_user --password my_password
• MongoDB1 connects with mongo --host "my_replicaSet/mongo1.my-site.com:27017" --authenticationDatabase my_database --username my_user --password my_password
• MongoDB2 connects with mongo --host "mongo2.my-site.com:27018" --authenticationDatabase my_database --username my_user --password my_password
• MongoDB2 does not connect with mongo --host "my_replicaSet/mongo2.my-site.com:27018" --authenticationDatabase my_database --username my_user --password my_password
• MongoDB2 replica mode says it cant find a way to connect to "internal cluster Pod address"
• Changin the Port of MongoDB2 deployment to 27018
• No change, can still not connect to replica set, only to individual Pod

What else I tried:

• Attaching a LoadBalancer to each MongoDB Pod individually
• This works as it should
• Because of the number of pods running and cost per public IP this is way to expensive
• Attaching only one LoadBalancer to the Primary Pod
• Now both MongoDBs are acting like MongoDB2 on Ingress

• Can connect directly to Primary, but not when trying to the replicaSet

Edit:

Please note that mongo1.my-site.com and mongo2.my-site.com are handeled by the same ingress, so I can not use the same port twice as I can only define one port per ingress, regardless of the domain, I could also change both URLs to be the same. I just wrote it like that to better defferentiate my processes