I have a kubernetes cluster and 2 replicated databases.I want to be able to connect that databases inside cluster. Databases will be statefulset and have a headless server. Lets say that pod names are D-1 and D-2. D-1 will be master and D-2 will be replica. I want to redirect all my traffic to master (D-1) but if master pod fails it should redirect traffic to replica(D-2). I can not do it on client side so i must find a way in kubernetes. How can i build this structure ? So if i curl to
database-service.default.svc.cluster.local
it must always go to D-1 pod and if D-1 pod is not available it must redirect all traffic to D-2.
I thought i can handle with this way(not tested): think about 2 database pods on 2 nodes.If D-2 has a readiness probe depend on D-1 is up(if D1-1 is up the D-2 has readiness probe situation and services mark this pod as unhealty) all the traffic will be redirected to the D-1(master).And if i replicate that databases from hostport i think it could be done.Is it a true approach ?
Also there is cloud native storage solutions.
The answer depends on DB type you are using.
Generally Service is just an abstract way to expose an application running on a set of Pods as a network service.
$ kubectl get svc -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
kubernetes ClusterIP 10.0.0.1 <none> 443/TCP 6d3h <none>
db-access ClusterIP 10.0.5.233 <none> 3306/TCP 3m app=my_db
Service in that example routes traffic to the Enpoint Slice according to the app=my_db
selector.
As you know, K8s evicts failed pods automatically that is why traffic isn't routed to the failed pods. However,
That is why in order to achieve the fail over you have described:
I want to redirect all my traffic to master (D-1) but if master pod fails it should redirect traffic to replica(D-2)
it looks like the "operator" is exactly what is needed here. Operator is not something that is included to K8s, but often provided by the DBVendor.
A good example here is the Oracle MySQL Operator for Kubernetes.
It is a K8s controller that can be installed into any existing K8s cluster. Once installed, it will enable users to create and manage production-ready MySQL clusters using a simple declarative configuration format.
It's feature rich, allows you to manage cluster, perform backups/restores, provides metrics, etc.
Additionally, you can consider using Cloud native distributed databases as @Jonas suggested earlier.
Hope that helps.