Go Microservices with Ambassador API Gateway

3/17/2019

I'm having a few issues getting Ambassador to work correctly. I'm new to Kubernetes and just teaching myself.

I have successfully managed to work through the demo material Ambassador provide - e.g /httpbin/ endpoint is working correctly, but when I try to deploy a Go service it is falling over.

When hitting the 'qotm' endpoint, the page this is the response:

upstream request timeout

Pod status:

CrashLoopBackOff

From my research, it seems to be related to the yaml file not being configured correctly but I'm struggling to find any documentation relating to this use case.

My cluster is running on AWS EKS and the images are being pushed to AWS ECR.

main.go:

package main

import (
    "fmt"
    "net/http"
    "os"
)

func main() {
    var PORT string
    if PORT = os.Getenv("PORT"); PORT == "" {
        PORT = "3001"
    }
    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        fmt.Fprintf(w, "Hello World from path: %s\n", r.URL.Path)
    })
    http.ListenAndServe(":" + PORT, nil)
}

Dockerfile:

FROM golang:alpine
ADD ./src /go/src/app
WORKDIR /go/src/app
EXPOSE 3001
ENV PORT=3001
CMD ["go", "run", "main.go"]

test.yaml:

apiVersion: v1
kind: Service
metadata:
  name: qotm
  annotations:
    getambassador.io/config: |
      ---
      apiVersion: ambassador/v1
      kind:  Mapping
      name:  qotm_mapping
      prefix: /qotm/
      service: qotm
spec:
  selector:
    app: qotm
  ports:
    - port: 80
      name: http-qotm
      targetPort: http-api
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: qotm
spec:
  replicas: 1
  strategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: qotm
    spec:
      containers:
        - name: qotm
          image: ||REMOVED||
          ports:
            - name: http-api
              containerPort: 3001
          readinessProbe:
            httpGet:
              path: /health
              port: 5000
            initialDelaySeconds: 30
            periodSeconds: 3
          resources:
            limits:
              cpu: "0.1"
              memory: 100Mi

Pod description:

Name:               qotm-7b9bf4d499-v9nxq
Namespace:          default
Priority:           0
PriorityClassName:  <none>
Node:               ip-192-168-89-69.eu-west-1.compute.internal/192.168.89.69
Start Time:         Sun, 17 Mar 2019 17:19:50 +0000
Labels:             app=qotm
                    pod-template-hash=3656908055
Annotations:        <none>
Status:             Running
IP:                 192.168.113.23
Controlled By:      ReplicaSet/qotm-7b9bf4d499
Containers:
  qotm:
    Container ID:   docker://5839996e48b252ac61f604d348a98c47c53225712efd503b7c3d7e4c736920c4
    Image:          IMGURL
    Image ID:       docker-pullable://IMGURL
    Port:           3001/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    1
      Started:      Sun, 17 Mar 2019 17:30:49 +0000
      Finished:     Sun, 17 Mar 2019 17:30:49 +0000
    Ready:          False
    Restart Count:  7
    Limits:
      cpu:     100m
      memory:  200Mi
    Requests:
      cpu:        100m
      memory:     200Mi
    Readiness:    http-get http://:3001/health delay=30s timeout=1s period=3s #success=1 #failure=3
    Environment:  <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-5bbxw (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  default-token-5bbxw:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-5bbxw
    Optional:    false
QoS Class:       Guaranteed
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                  From                                                  Message
  ----     ------     ----                 ----                                                  -------
  Normal   Scheduled  12m                  default-scheduler                                     Successfully assigned default/qotm-7b9bf4d499-v9nxq to ip-192-168-89-69.eu-west-1.compute.internal
  Normal   Pulled     10m (x5 over 12m)    kubelet, ip-192-168-89-69.eu-west-1.compute.internal  Container image "IMGURL" already present on machine
  Normal   Created    10m (x5 over 12m)    kubelet, ip-192-168-89-69.eu-west-1.compute.internal  Created container
  Normal   Started    10m (x5 over 11m)    kubelet, ip-192-168-89-69.eu-west-1.compute.internal  Started container
  Warning  BackOff    115s (x47 over 11m)  kubelet, ip-192-168-89-69.eu-west-1.compute.internal  Back-off restarting failed container
-- Lewis
amazon-eks
docker
go
kubernetes
yaml

1 Answer

3/17/2019

In your kubernetes deployment file you have exposed a readiness probe on port 5000 while your application is exposed on port 3001, also while running the container a few times I got OOMKilled so increased the memory limit. Anyways below deployment file should work fine.

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: qotm
spec:
  replicas: 1
  strategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: qotm
    spec:
      containers:
        - name: qotm
          image: <YOUR_IMAGE>
          imagePullPolicy: Always
          ports:
            - name: http-api
              containerPort: 3001
          readinessProbe:
            httpGet:
              path: /health
              port: 3001
            initialDelaySeconds: 30
            periodSeconds: 3
          resources:
            limits:
              cpu: "0.1"
              memory: 200Mi
-- Akash Srivastava
Source: StackOverflow