Kubernetes accessing a replica behind a load balancer

8/14/2018

I have the following deployment and a load balancer service to connect to it:

apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: slave
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: slave
    spec:
      containers:
      - name: slave
        envFrom:
        - configMapRef:
            name: my-env-var
        image: registry.azurecr.io/slave:latest
        ports:
        - containerPort: 8080
      imagePullSecrets:
      - name: acr-auth
---
apiVersion: v1
kind: Service
metadata:
  name: slave
spec:
  type: LoadBalancer
  ports:
  - port: 8080
  selector:
    app: slave

I connect to this deployment from another deployment, let's called it master, through "slave:8080" which using DNS it gets resolved to an IP. So the master sends a request to the slave for a job. The slave does a job and reply with a response containing a link to the result file. The link is something like slave:8080/results.txt. So the master can now download the results file. This works fine when I have one slave. Now, I want to have more than one by using replicas. The problem is that how could the master get the results file from a certain slave since they are all sitting behind a load balancer.

This is the description of my slave service:

Name:                     slave
Namespace:                default
Labels:                   <none>
Annotations:              kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"name":"slave","namespace":"default"},"spec":{"ports":[{"port":8080}],"selector...
Selector:                 app=slave
Type:                     LoadBalancer
IP:                       10.0.233.16
LoadBalancer Ingress:     41.137.169.141
Port:                     <unset>  8080/TCP
TargetPort:               8080/TCP
NodePort:                 <unset>  32546/TCP
Endpoints:                10.244.0.213:8080,10.244.0.214:8080
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

I see two endpoints for the replicas but these are internal hence unreachable.

So how can the master get the results file from the replica slave that performed the job?

I want to know if an easy solution can be done before considering having a mounted volume or a different approach.

Thanks!

-- Ziad Halabi
containers
kubernetes
load-balancing

1 Answer

8/17/2018

To me, this smells like a faulty design and you should fix it. In your slaves, after the pod is done processing the file - it is holding it locally - which is state being held locally and not a good idea.

After the processing of the job is done, you should store the file in something external - say S3 or an NFS drive which is mounted on all pods.

The second request - when it comes to the service asking for the file with the filename, irrespective of which pod the request goes to, the pod will go and fetch from the S3 bucket/NFS drive.

-- Vishal Biyani
Source: StackOverflow