K
Q

Why is my nginx ingress controller on gke sending 503 response only for the docker image that I have built?

5/3/2020

I deployed an nginx ingress controller on my google cloud platform cluster based on this tutorial: https://cloud.google.com/community/tutorials/nginx-ingress-gke All test images works fine but whenever I deploy my own Dockerfile image I get a 503 response.

  • /hello,/coffee & /tea have test images and are working fine
  • /bob have a costumed Dockerfile images

Dockerfile

RUN apt-get update
RUN apt-get upgrade -y

COPY debconf.selections /tmp/
RUN debconf-set-selections /tmp/debconf.selections

RUN apt-get install -y zip unzip
RUN apt-get install -y \
    php7.0 \
    php7.0-bz2 \
    php7.0-cgi \
    php7.0-cli \
    php7.0-common \
    php7.0-curl \
    php7.0-dev \
    php7.0-enchant \
    php7.0-fpm \
    php7.0-gd \
    php7.0-gmp \
    php7.0-imap \
    php7.0-interbase \
    php7.0-intl \
    php7.0-json \
    php7.0-ldap \
    php7.0-mbstring \
    php7.0-mcrypt \
    php7.0-mysql \
    php7.0-odbc \
    php7.0-opcache \
    php7.0-pgsql \
    php7.0-phpdbg \
    php7.0-pspell \
    php7.0-readline \
    php7.0-recode \
    php7.0-snmp \
    php7.0-sqlite3 \
    php7.0-sybase \
    php7.0-tidy \
    php7.0-xmlrpc \
    php7.0-xsl \
    php7.0-zip
RUN apt-get install apache2 libapache2-mod-php7.0 -y
RUN apt-get install mariadb-common mariadb-server mariadb-client -y
RUN apt-get install postfix -y
RUN apt-get install git nodejs npm composer nano tree vim curl ftp -y
RUN npm install -g bower grunt-cli gulp

ENV LOG_STDOUT **Boolean**
ENV LOG_STDERR **Boolean**
ENV LOG_LEVEL warn
ENV ALLOW_OVERRIDE All
ENV DATE_TIMEZONE UTC
ENV TERM dumb

COPY app/ /var/www/html/
COPY run-lamp.sh /usr/sbin/

RUN a2enmod rewrite
RUN ln -s /usr/bin/nodejs /usr/bin/node
RUN chmod +x /usr/sbin/run-lamp.sh
RUN chown -R www-data:www-data /var/www/html

VOLUME /var/www/html
VOLUME /var/log/httpd
VOLUME /var/lib/mysql
VOLUME /var/log/mysql
VOLUME /etc/apache2

EXPOSE 80
EXPOSE 3306

CMD ["/usr/sbin/run-lamp.sh"]

ingress-resources.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-resource
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
  rules:
  - http:
      paths:
      - path: /hello
        backend:
          serviceName: hello-app
          servicePort: 8080
      - path: /coffee
        backend:
          serviceName: coffee-svc
          servicePort: 8080
      - path: /tea
        backend:
          serviceName: tea-svc
          servicePort: 8080
      - path: /bob
        backend:
          serviceName: bobtest-1
          servicePort: 8080

kubectl describe svc bobtest-1

Name:              bobtest-1
Namespace:         default
Labels:            app=bobtest-1
Annotations:       <none>
Selector:          app=bobtest-1
Type:              ClusterIP
IP:                10.12.5.134
Port:              <unset>  80/TCP
TargetPort:        80/TCP
Endpoints:         10.8.0.15:80
Session Affinity:  None
Events:            <none>

pod log

https://drive.google.com/file/d/10uVhzw6YnfX3tvgmnlhYKW47qy4r0zj5/view?usp=sharing

pod yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "1"
  creationTimestamp: "2020-05-03T06:46:35Z"
  generation: 1
  labels:
    app: nginx-ingress
    app.kubernetes.io/component: controller
    chart: nginx-ingress-1.36.3
    heritage: Tiller
    release: nginx-ingress
  name: nginx-ingress-controller
  namespace: default
  resourceVersion: "7023"
  selfLink: /apis/apps/v1/namespaces/default/deployments/nginx-ingress-controller
  uid: d4d4777b-8d09-11ea-b476-42010a80005d
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: nginx-ingress
      release: nginx-ingress
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: nginx-ingress
        app.kubernetes.io/component: controller
        component: controller
        release: nginx-ingress
    spec:
      containers:
      - args:
        - /nginx-ingress-controller
        - --default-backend-service=default/nginx-ingress-default-backend
        - --publish-service=default/nginx-ingress-controller
        - --election-id=ingress-controller-leader
        - --ingress-class=nginx
        - --configmap=default/nginx-ingress-controller
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
        image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.30.0
        imagePullPolicy: IfNotPresent
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        name: nginx-ingress-controller
        ports:
        - containerPort: 80
          name: http
          protocol: TCP
        - containerPort: 443
          name: https
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        resources: {}
        securityContext:
          allowPrivilegeEscalation: true
          capabilities:
            add:
            - NET_BIND_SERVICE
            drop:
            - ALL
          runAsUser: 101
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: nginx-ingress
      serviceAccountName: nginx-ingress
      terminationGracePeriodSeconds: 60
status:
  availableReplicas: 1
  conditions:
  - lastTransitionTime: "2020-05-03T06:47:07Z"
    lastUpdateTime: "2020-05-03T06:47:07Z"
    message: Deployment has minimum availability.
    reason: MinimumReplicasAvailable
    status: "True"
    type: Available
  - lastTransitionTime: "2020-05-03T06:46:35Z"
    lastUpdateTime: "2020-05-03T06:47:07Z"
    message: ReplicaSet "nginx-ingress-controller-5c798c9ffc" has successfully progressed.
    reason: NewReplicaSetAvailable
    status: "True"
    type: Progressing
  observedGeneration: 1
  readyReplicas: 1
  replicas: 1
  updatedReplicas: 1
-- Ibrahim
docker
google-kubernetes-engine
kubernetes
nginx
ubuntu

Similar Questions

Not able to deploy a simple public docker image and I can't build simple java app
canot get a working certificate on kubernetes
Why google-managed SSL certificate requires domain DNS to point to a static IP address instead of the load balancer itself?
Network gateway application in Kubernetes cluster that processes and forwards network packets

1 Answer

5/3/2020

Make sure servicePort defined in the ingress matches with the port defined in bobtest-1 service and the service's TargetPort matches with the containerPort defined in the pod spec.

Another thing to note here is that there is probably nothing being served at path /bob. But nginx by default will send traffic to the /bob as defined in the path. To avoid that you can add below annotation in the ingress resource.

nginx.ingress.kubernetes.io/rewrite-target: /

With this nginx will send traffic to the root of the app / meaning when you hit /bob the request will go to bobtest-1 instead of bobtest-1/bob

Use curl from another pod for testing the service directly.

kubectl run curl --image=radial/busyboxplus:curl -i --tty

If you don't see a command prompt, try pressing enter.

[ root@curl:/ ]$ curl bobtest-1
-- Arghya Sadhu
Source: StackOverflow