Not able to replace NGINX with NGINX Plus as reverse proxy for microservices on Google Cloud using Kubernetes

7/5/2016

I am following this nice tutorial of How to create a scalable API with micro-services on Google Cloud using Kubernetes.

I have created 4 micro-services and to expose the services I am using NGINX Plus.

Note : The purpose of NGINX Plus / NGINX here is to work as reverse proxy.

Below is the directory Structure :

-nginx

--Dockerfile

--deployment.yaml

--index.html

--nginx-repo.crt

--nginx-repo.key

--nginx.conf

--svc.yaml

Details of files can be seen here. I am pasting the Docker file & nginx.conf here :

Dockerfile (Original with NGINX Plus):

FROM debian:8.3

RUN apt-get update && apt-get -y install wget

RUN mkdir -p /etc/ssl/nginx && wget -P /etc/ssl/nginx https://cs.nginx.com/static/files/CA.crt

COPY nginx-repo.key /etc/ssl/nginx/nginx-repo.key
COPY nginx-repo.crt /etc/ssl/nginx/nginx-repo.crt

RUN wget http://nginx.org/keys/nginx_signing.key && apt-key add nginx_signing.key

RUN apt-get -y install apt-transport-https libcurl3-gnutls lsb-release
RUN printf "deb https://plus-pkgs.nginx.com/debian `lsb_release -cs` nginx-plus\n" | tee /etc/apt/sources.list.d/nginx-plus.list
RUN wget -P /etc/apt/apt.conf.d https://cs.nginx.com/static/files/90nginx

RUN apt-get update && apt-get -y install nginx-plus

RUN mkdir /data
COPY index.html /data/index.html

COPY nginx.conf /etc/nginx/conf.d/backend.conf
RUN rm /etc/nginx/conf.d/default.conf

CMD ["nginx", "-g", "daemon off;"]

nginx.conf (Original with NGINX Plus):

resolver 10.11.240.10 valid=5s;

upstream reverse-backend {
    zone reverse-backend 64k;
    server reverse.default.svc.cluster.local resolve;
}

upstream arrayify-backend {
    zone arrayify-backend 64k;
    server arrayify.default.svc.cluster.local resolve;
}

upstream lower-backend {
    zone lower-backend 64k;
    server lower.default.svc.cluster.local resolve;
}

upstream upper-backend {
    zone upper-backend 64k;
    server upper.default.svc.cluster.local resolve;
}

server {
    listen 80;

    root /data;

    location / {
        index  index.html index.htm;
    }

    status_zone backend-servers;

    location /reverse/ {
        proxy_pass http://reverse-backend/;
    }

    location /arrayify/ {
        proxy_pass http://arrayify-backend/;
    }

    location /lower/ {
        proxy_pass http://lower-backend/;
    }

    location /upper/ {
        proxy_pass http://upper-backend/;
    }

}

server {
    listen 8080;

    root /usr/share/nginx/html;

    location = /status.html { }

    location /status {
        status;
    }
}

Everything seem to be working fine with NGINX Plus and I am able to hit all 4 micro-services with with url eg. http://x.y.z.w/service[1|2|3|4]/?str=testnginx , where http://x.y.z.w is my external ip and NGINX is taking care of routing requests internally. Now I am willing to do the same work without NGINX Plus by using NGINX only.

Below are the updated files for NGINX :

Dockerfile (Updated for NGINX) :

    FROM debian:8.3
    RUN apt-get update && apt-get -y install wget
    RUN mkdir -p /etc/ssl/nginx && wget -P /etc/ssl/nginx https://cs.nginx.com/static/files/CA.crt
    #COPY nginx-repo.key /etc/ssl/nginx/nginx-repo.key
    #COPY nginx-repo.crt /etc/ssl/nginx/nginx-repo.crt
    #RUN wget http://nginx.org/keys/nginx_signing.key && apt-key add nginx_signing.key
    RUN apt-get -y install apt-transport-https libcurl3-gnutls lsb-release
    #RUN printf "deb https://plus-pkgs.nginx.com/debian `lsb_release -cs` nginx-plus\n" | tee /etc/apt/sources.list.d/nginx-plus.list
    RUN wget -P /etc/apt/apt.conf.d https://cs.nginx.com/static/files/90nginx
    RUN apt-get update && apt-get -y install nginx
    RUN mkdir /data
    COPY index.html /data/index.html
    COPY nginx.conf /etc/nginx/conf.d/backend.conf
    #RUN rm /etc/nginx/conf.d/default.conf
    CMD ["nginx", "-g", "daemon off;"]

nginx.conf (Updated for NGINX) :

resolver 10.3.240.10 valid=5s;

upstream reverse-backend {
    zone reverse-backend 64k;
    server reverse.default.svc.cluster.local;
}

upstream arrayify-backend {
    zone arrayify-backend 64k;
    server arrayify.default.svc.cluster.local;
}
upstream lower-backend {
    zone lower-backend 64k;
    server lower.default.svc.cluster.local;
}
upstream upper-backend {
    zone upper-backend 64k;
    server upper.default.svc.cluster.local;
}
server {
    listen 80;
    root /data;
    location / {
        index  index.html index.htm;
    }
#    status_zone backend-servers;
    location /reverse/ {
        proxy_pass http://reverse-backend/;
    }
    location /arrayify/ {
        proxy_pass http://arrayify-backend/;
    }
    location /lower/ {
        proxy_pass http://lower-backend/;
    }
    location /upper/ {
        proxy_pass http://upper-backend/;
    }
}
#server {
#    listen 8080;
#
#    root /usr/share/nginx/html;
#
#    location = /status.html { }
#
#    location /status {
#        status;
#    }
#}

Basically, I have removed resolve and server statements, which are the features of NGINX Plus and able to create docker image, uploading it on google container and creating my deployments and service but getting 404 not found.

Am I missing something here or this is the limitation of NGINX ?

Please suggest if anyone has any suggestion or prior experience working with NGINX, Docker and Kubernets on Google Cloud.

-- Nikhil Maheshwari
docker
kubectl
kubernetes
nginx

0 Answers