Kubernetes pod Troubleshoot

1/18/2020

I deployed my container in kubernetes pod, andd pod and related services are up and running.

please find the below pod and deployment status of the pods

root@jenkins-linux-vm:/home/admin/kubernetes# kubectl get pods
NAME                                  READY   STATUS    RESTARTS   AGE
angular-deployment-5d5fbf967c-zvzvl   1/1     Running   0          70m

root@jenkins-linux-vm:/home/admin/kubernetes# kubectl get svc
NAME              TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
angular-service   NodePort   10.96.16.68    <none>        80:31000/TCP     79m

root@jenkins-linux-vm:/home/admin/kubernetes# kubectl get deployment
NAME                 READY   UP-TO-DATE   AVAILABLE   AGE
angular-deployment   1/1     1            1           70m

please find the below curl access

root@jenkins-linux-vm:/home/admin/kubernetes# kubectl exec -it angular-deployment-5d5fbf967c-zvzvl curl 10.0.0.1:31000
curl: (7) Failed to connect to 10.0.0.1 port 31000: Connection refused
command terminated with exit code 7

Even though i am not able to access my application services in the browser as like below.

https://10.0.0.1:31000
apiVersion: apps/v1
kind: Deployment 
metadata: 
  name: angular-deployment 
spec: 
  selector: 
    matchLabels: 
      app: frontend-app 
  replicas: 1
  template: 
    metadata: 
      labels: 
        app: frontend-app 
    spec: 
      containers: 
      - name: frontend-app 
        image: ${IMAGE_NAME}:${IMAGE_TAG} 
        ports: 
        - containerPort: 80 
---
kind: Service 
apiVersion: v1 
metadata: 
  name: angular-service 
spec: 
  selector: 
    app: frontend-app 
  ports: 
  - protocol: TCP 
    port: 80 
    targetPort: 80 
    nodePort: 31000 
  type: NodePort
root@jenkins-linux-vm:/home/admin# kubectl describe pod angular-deployment-556c47f666-9d2x4
Name:         angular-deployment-556c47f666-9d2x4
Namespace:    pre-release
Priority:     0
Node:         poc-worker2/10.0.0.2
Start Time:   Sat, 18 Jan 2020 08:47:35 +0000
Labels:       app=frontend-app
              pod-template-hash=556c47f666
Annotations:  <none>
Status:       Running
IP:           10.32.0.8
IPs:
  IP:           10.32.0.8
Controlled By:  ReplicaSet/angular-deployment-556c47f666
Containers:
  frontend-app:
    Container ID:   docker://43fea22e4c1d49e0c94fc8aca3a4b41df44b5f91f45ea29ede263c5a6bcf6503
    Image:          frontend-app:future-master-fix-f2d2a8bd
    Image ID:       docker://sha256:0099587db89de9ef999a7d1f087d4781e73c491b17e89392e92b08d2f935ad27
    Port:           80/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Sat, 18 Jan 2020 08:47:40 +0000
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-r67p7 (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  default-token-r67p7:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-r67p7
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason     Age   From                       Message
  ----    ------     ----  ----                       -------
  Normal  Scheduled  21s   default-scheduler          Successfully assigned pre-release/angular-deployment-556c47f666-9d2x4 to poc-worker2
  Normal  Pulled     17s   kubelet, poc-worker2  Container image "frontend-app:future-master-fix-f2d2a8bd" already present on machine
  Normal  Created    16s   kubelet, poc-worker2  Created container frontend-app
  Normal  Started    16s   kubelet, poc-worker2  Started container frontend-app

please find the below are the svc for deployment

root@jenkins-linux-vm:/home/admin# kubectl describe svc angular-service
Name:                     angular-service
Namespace:                pre-release
Labels:                   <none>
Annotations:              kubectl.kubernetes.io/last-applied-configuration:
                            {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"name":"angular-service","namespace":"pre-release"},"spec":{"ports":[{"no...
Selector:                 app=frontend-app
Type:                     NodePort
IP:                       10.96.227.143
Port:                     <unset>  80/TCP
TargetPort:               80/TCP
NodePort:                 <unset>  31000/TCP
Endpoints:                10.32.0.4:80,10.32.0.8:80
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

Please find the docker file here

FROM node:12.2.0

# set working directory
WORKDIR /app

# add `/app/node_modules/.bin` to $PATH
ENV PATH /app/node_modules/.bin:$PATH

# install and cache app dependencies
COPY package.json /app/package.json

# add app
COPY . /app

# start app
CMD ng serve --host 0.0.0.0

Can you please some one help me how to fix this issue

-- Gowthami Viswa
kubernetes

2 Answers

1/18/2020

If you are accessing the pod via nodeip:nodeport which will only work if you are accessing the pod from outside the cluster using a browser.

Here is a guide on how to expose an application via NodePort. In this case the nodes of your kubernetes cluster need to be accessible i.e should have public ip.

You should be using the cluster ip to access the pod from within the cluster i.e via exec from another pod as shown in below command.

kubectl exec -it angular-deployment-5d5fbf967c-zvzvl curl 10.96.16.68:80

I have a feeling that your docker container is not listening on port 80.

-- Arghya Sadhu
Source: StackOverflow

1/21/2020

Here I could see the issue is with your Angular Dockerfile, you are using ng serve if you see your dependencies in package.json you will see "@angular/cli": "*" in order to have that inside your docker you need to add RUN npm install this will install all your dependencies inside your docker container, and you can do ng serve but ng serve is for development locally, it's not a good approach I would say.

To identify these kind of issues, it's advisable to run something like this on local machine in order to find if your docker container is working fine or not, before you could deploy it onto to k8s cluster, as you know kubernetes is a very big universe it will take time to identify the actual problem.

Ok coming to the issue (I can simply add a single command to your dockerfile and add my answer, but I wouldn't suggest that approach. So adding the complete answer which would look good), when you are deploying some frontend related application your docker image need to have the capabilities of serving the index.html page it's the end product after you build your Angular or React applications.

There are several ways this could be done. And there are several tutorials explaining the same, here is something I would suggest, your dockerfile should look like this. Adding comments what they do.

#Stage0: builder, based on Node alpine imagine to build and compile your angular code
FROM node:10-alpine as builder 

WORKDIR /app 

COPY package*.json /app/

# This is one thing you forgot in your dockerfile, if you add this it might work
RUN npm install 

COPY . . 

# This is normal build, it does ng build
RUN npm run build

#Stage 1, based on Nginx imagine, to have only the compile app inside nginx folders to serve
FROM nginx:1.15

COPY --from=builder /app/dist/ /usr/share/nginx/html

# This one copies the local nginx.conf file as default.conf for nginx to let it serve
COPY ./nginx.conf /etc/nginx/conf.d/default.conf

Please make sure you have nginx.conf file inside your code, file level is same as package.json

server {
  listen 80;

  sendfile on;

  default_type application/octet-stream;


  gzip on;
  gzip_http_version 1.1;
  gzip_disable      "MSIE [1-6]\.";
  gzip_min_length   1100;
  gzip_vary         on;
  gzip_proxied      expired no-cache no-store private auth;
  gzip_types        text/plain text/css application/json application/javascript application/x-javascript text/xml application/xml application/xml+rss text/javascript;
  gzip_comp_level   9;

  root /usr/share/nginx/html;

  location / {
     #try_files $uri $uri/ /index.html =404;
    expires -1d;
    alias /usr/share/nginx/html/;
    try_files $uri$args $uri$args/ /index.html =404;
    location ~* \.(?:ico|css|js|gif|jpe?g|png|svg|woff|woff2|ttf|eot)$ {
            add_header Access-Control-Allow-Origin *;
    }
  }
}

Make sure you run docker run on local machine before you deploy it on to cluster.

Hope this helps.

-- BinaryMonster
Source: StackOverflow