I'm setting up active/passive routing for an application in kube, that's outside of the typical K8S use case. I'm trying to find configuration related to routing or load balancing in headless services with multiple backends. So far I have managed to route traffic to my backends however I need to ensure that the traffic is routed correctly. The application requires TCP connections and the primary/secondary instances have differing configuration(requiring different deployment objects). If fail-over occurs, routing is expected to return to the primary once it is restored.

The routing consistently behaves as-desired, but no documentation or configuration would indicate as much. I've found documentation stating that it should be round-robin or random because of the order of the dns entries. The crux of the question is: can I depend on this behavior? Since this is undocumented and not explicitly configured I'm concerned that it will change in future versions, or deployments.

I'm using Rancher with the canal networking layer. I've read through both the calico and flannel docs. Neither Endpoints/endpoint slices, nor do dns entries indicate any order for routing.

Currently the setup has two deployments that are selected by a headless service. the deployed pods have a hostname of input-primary in deployment 1 and input-secondary in deployment 2. I can access either of them by dns as input-primary.myservice, or input-secondary.myservice.

the ingress controller tcp-services config map has an entry for my service:

25252: default/myservice:9999

and an abridged version of the k8s config:

ApiVersion: v1
kind: Service
metadata:
  name:myservice
spec:
  clusterIP: None
  ports:
  - name: input
    port: 9999
selector:
  app: myapp
  type: ClusterIP
----
apiVersion: apps/v1beta2
kind: Deployment
metadata:
  labels:
    app: myapp
  name: input-primary
spec:
  hostname: input-primary
  containers:
  - ports:
    - containerport: 9999
      name: input
      protocol: TCP
----
apiVersion: apps/v1beta2
kind: Deployment
metadata:
  labels:
    app: myapp
  name: input-secondary
spec:
  hostname: input-secondary
  containers:
  - ports:
    - containerport: 9999
      name: input
      protocol: TCP```