kubernetes connection refused during deployment

5/12/2018

I'm trying to achieve a zero downtime deployment using kubernetes and during my test the service doesn't load balance well.

My kubernetes manifest is:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: myapp-deployment
spec:
  replicas: 3
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 0
      maxSurge: 1
  template:
    metadata:
      labels:
        app: myapp
        version: "0.2"
    spec:
      containers:
      - name: myapp-container
        image: gcr.io/google-samples/hello-app:1.0
        imagePullPolicy: Always
        ports:
          - containerPort: 8080
            protocol: TCP
        readinessProbe:
          httpGet:
            path: /
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 5
          successThreshold: 1

---

apiVersion: v1
kind: Service
metadata:
  name: myapp-lb
  labels:
    app: myapp
spec:
  type: LoadBalancer
  externalTrafficPolicy: Local
  ports:
    - port: 80
      targetPort: 8080
  selector:
    app: myapp

If I loop over the service with the external IP, let's say:

$ kubectl get services
NAME         TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)        AGE
kubernetes   ClusterIP      10.35.240.1    <none>           443/TCP        1h
myapp-lb     LoadBalancer   10.35.252.91   35.205.100.174   80:30549/TCP   22m

using the bash script:

while True
    do
        curl 35.205.100.174 
        sleep 0.2s
    done

I receive some connection refused during the deployment:

curl: (7) Failed to connect to 35.205.100.174 port 80: Connection refused

The application is the default helloapp provided by Google Cloud Platform and running on 8080.

Cluster information:

  • Kubernetes version: 1.8.8
  • Google cloud platform
  • Machine type: g1-small
-- thoas
google-cloud-platform
kubernetes

1 Answer

5/14/2018

It looks like your request goes to a not started pod. I have avoided this by adding a few parameters:

  • Liveness probe to be sure app has already started

  • maxUnavalible: 1 to deploy pods one by one

I still have some errors, but they are acceptable because they rarely happen . During the deployment, an error may occur once or twice, so with increasing load you will have a negligible amount of errors. I mean one or two errors per 2000 requests during the deployment.

-- Nick Rak
Source: StackOverflow