K
Q

nginx-ingress k8s on Google no IP address

3/4/2019

I am practicing the k8s by following the ingress chapter. I am using Google Cluster. Specification are as follows

master: 1.11.7-gke.4
node: 1.11.7-gke.4

$ kubectl get nodes -o wide
NAME                                   STATUS   ROLES    AGE   VERSION         INTERNAL-IP   EXTERNAL-IP      OS-IMAGE                             KERNEL-VERSION   CONTAINER-RUNTIME
gke-singh-default-pool-a69fa545-1sm3   Ready    <none>   6h    v1.11.7-gke.4   10.148.0.46   35.197.128.107   Container-Optimized OS from Google   4.14.89+         docker://17.3.2
gke-singh-default-pool-a69fa545-819z   Ready    <none>   6h    v1.11.7-gke.4   10.148.0.47   35.198.217.71    Container-Optimized OS from Google   4.14.89+         docker://17.3.2
gke-singh-default-pool-a69fa545-djhz   Ready    <none>   6h    v1.11.7-gke.4   10.148.0.45   35.197.159.75    Container-Optimized OS from Google   4.14.89+         docker://17.3.2

master endpoint: 35.186.148.93

DNS: singh.hbot.io (master IP)

To keep my question short. I post my source code in the snippet and links back to here.

Files:

deployment.yaml ingress.yaml ingress-rules.yaml

Problem:
curl http://singh.hbot.io/webapp1 got timed out

Description 

$ kubectl get deployment -n nginx-ingress
NAME            DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
nginx-ingress   1         1         1            0           2h

nginx-ingress deployment is not available.

$ kubectl describe deployment -n nginx-ingress
Name:                   nginx-ingress
Namespace:              nginx-ingress
CreationTimestamp:      Mon, 04 Mar 2019 15:09:42 +0700
Labels:                 app=nginx-ingress
Annotations:            deployment.kubernetes.io/revision: 1
                        kubectl.kubernetes.io/last-applied-configuration:
                          {"apiVersion":"extensions/v1beta1","kind":"Deployment","metadata":{"annotations":{},"name":"nginx-ingress","namespace":"nginx-ingress"},"s...
Selector:               app=nginx-ingress
Replicas:               1 desired | 1 updated | 1 total | 0 available | 1 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  1 max unavailable, 1 max surge
Pod Template:
  Labels:           app=nginx-ingress
  Service Account:  nginx-ingress
  Containers:
   nginx-ingress:
    Image:       nginx/nginx-ingress:edge
    Ports:       80/TCP, 443/TCP
    Host Ports:  0/TCP, 0/TCP
    Args:
      -nginx-configmaps=$(POD_NAMESPACE)/nginx-config
      -default-server-tls-secret=$(POD_NAMESPACE)/default-server-secret
    Environment:
      POD_NAMESPACE:   (v1:metadata.namespace)
      POD_NAME:        (v1:metadata.name)
    Mounts:           <none>
  Volumes:            <none>
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      True    MinimumReplicasAvailable
OldReplicaSets:  <none>
NewReplicaSet:   nginx-ingress-77fcd48f4d (1/1 replicas created)
Events:          <none>

pods: 

$ kubectl get pods --all-namespaces=true
NAMESPACE       NAME                                              READY   STATUS             RESTARTS   AGE
default         webapp1-7d67d68676-k9hhl                          1/1     Running            0          6h
default         webapp2-64d4844b78-9kln5                          1/1     Running            0          6h
default         webapp3-5b8ff7484d-zvcsf                          1/1     Running            0          6h
kube-system     event-exporter-v0.2.3-85644fcdf-xxflh             2/2     Running            0          6h
kube-system     fluentd-gcp-scaler-8b674f786-gvv98                1/1     Running            0          6h
kube-system     fluentd-gcp-v3.2.0-srzc2                          2/2     Running            0          6h
kube-system     fluentd-gcp-v3.2.0-w2z2q                          2/2     Running            0          6h
kube-system     fluentd-gcp-v3.2.0-z7p9l                          2/2     Running            0          6h
kube-system     heapster-v1.6.0-beta.1-5685746c7b-kd4mn           3/3     Running            0          6h
kube-system     kube-dns-6b98c9c9bf-6p8qr                         4/4     Running            0          6h
kube-system     kube-dns-6b98c9c9bf-pffpt                         4/4     Running            0          6h
kube-system     kube-dns-autoscaler-67c97c87fb-gbgrs              1/1     Running            0          6h
kube-system     kube-proxy-gke-singh-default-pool-a69fa545-1sm3   1/1     Running            0          6h
kube-system     kube-proxy-gke-singh-default-pool-a69fa545-819z   1/1     Running            0          6h
kube-system     kube-proxy-gke-singh-default-pool-a69fa545-djhz   1/1     Running            0          6h
kube-system     l7-default-backend-7ff48cffd7-trqvx               1/1     Running            0          6h
kube-system     metrics-server-v0.2.1-fd596d746-bvdfk             2/2     Running            0          6h
kube-system     tiller-deploy-57c574bfb8-xnmtj                    1/1     Running            0          1h
nginx-ingress   nginx-ingress-77fcd48f4d-rfwbk                    0/1     CrashLoopBackOff   35         2h

describe pod 

$ kubectl describe pods -n nginx-ingress
Name:               nginx-ingress-77fcd48f4d-5rhtv
Namespace:          nginx-ingress
Priority:           0
PriorityClassName:  <none>
Node:               gke-singh-default-pool-a69fa545-djhz/10.148.0.45
Start Time:         Mon, 04 Mar 2019 17:55:00 +0700
Labels:             app=nginx-ingress
                    pod-template-hash=3397804908
Annotations:        <none>
Status:             Running
IP:                 10.48.2.10
Controlled By:      ReplicaSet/nginx-ingress-77fcd48f4d
Containers:
  nginx-ingress:
    Container ID:  docker://5d3ee9e2bf7a2060ff0a96fdd884a937b77978c137df232dbfd0d3e5de89fe0e
    Image:         nginx/nginx-ingress:edge
    Image ID:      docker-pullable://nginx/nginx-ingress@sha256:16c1c6dde0b904f031d3c173e0b04eb82fe9c4c85cb1e1f83a14d5b56a568250
    Ports:         80/TCP, 443/TCP
    Host Ports:    0/TCP, 0/TCP
    Args:
      -nginx-configmaps=$(POD_NAMESPACE)/nginx-config
      -default-server-tls-secret=$(POD_NAMESPACE)/default-server-secret
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    255
      Started:      Mon, 04 Mar 2019 18:16:33 +0700
      Finished:     Mon, 04 Mar 2019 18:16:33 +0700
    Ready:          False
    Restart Count:  9
    Environment:
      POD_NAMESPACE:  nginx-ingress (v1:metadata.namespace)
      POD_NAME:       nginx-ingress-77fcd48f4d-5rhtv (v1:metadata.name)
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from nginx-ingress-token-zvcwt (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  nginx-ingress-token-zvcwt:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  nginx-ingress-token-zvcwt
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                  From                                           Message
  ----     ------     ----                 ----                                           -------
  Normal   Scheduled  26m                  default-scheduler                              Successfully assigned nginx-ingress/nginx-ingress-77fcd48f4d-5rhtv to gke-singh-default-pool-a69fa545-djhz
  Normal   Created    25m (x4 over 26m)    kubelet, gke-singh-default-pool-a69fa545-djhz  Created container
  Normal   Started    25m (x4 over 26m)    kubelet, gke-singh-default-pool-a69fa545-djhz  Started container
  Normal   Pulling    24m (x5 over 26m)    kubelet, gke-singh-default-pool-a69fa545-djhz  pulling image "nginx/nginx-ingress:edge"
  Normal   Pulled     24m (x5 over 26m)    kubelet, gke-singh-default-pool-a69fa545-djhz  Successfully pulled image "nginx/nginx-ingress:edge"
  Warning  BackOff    62s (x112 over 26m)  kubelet, gke-singh-default-pool-a69fa545-djhz  Back-off restarting failed container

Fix container terminated
Add to the command to ingress.yaml to prevent container finish running and get terminated by k8s. command: [ "/bin/bash", "-ce", "tail -f /dev/null" ]

Ingress has no IP address from GKE. Let me have a look in details

describe ingress: 

$ kubectl describe ing
Name:             webapp-ingress
Namespace:        default
Address:
Default backend:  default-http-backend:80 (10.48.0.8:8080)
Rules:
  Host  Path  Backends
  ----  ----  --------
  *
        /webapp1   webapp1-svc:80 (<none>)
        /webapp2   webapp2-svc:80 (<none>)
                   webapp3-svc:80 (<none>)
Annotations:
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{},"name":"webapp-ingress","namespace":"default"},"spec":{"rules":[{"http":{"paths":[{"backend":{"serviceName":"webapp1-svc","servicePort":80},"path":"/webapp1"},{"backend":{"serviceName":"webapp2-svc","servicePort":80},"path":"/webapp2"},{"backend":{"serviceName":"webapp3-svc","servicePort":80}}]}}]}}

Events:
  Type     Reason     Age                     From                     Message
  ----     ------     ----                    ----                     -------
  Warning  Translate  7m45s (x59 over 4h20m)  loadbalancer-controller  error while evaluating the ingress spec: service "default/webapp1-svc" is type "ClusterIP", expected "NodePort" or "LoadBalancer"; service "default/webapp2-svc" is type "ClusterIP", expected "NodePort" or "LoadBalancer"; service "default/webapp3-svc" is type "ClusterIP", expected "NodePort" or "LoadBalancer"

From this line I got all the ultimate solution from Christian Roy Thank you very much.

Fix the ClusterIP It is default value then I have to edit my manifest file using NodePort as follow

apiVersion: v1
kind: Service
metadata:
  name: webapp1-svc
  labels:
    app: webapp1
spec:
  type: NodePort
  ports:
  - port: 80
  selector:
    app: webapp1

And that is.

-- Sarit
docker
kubernetes
nginx

Similar Questions

Cannot set kubelet healthz bind address with kubeadm
How to forward Traffic to specific pod?
Reconfigure postgres with kubectl apply command
Kubernetes - Accessing a ClusterIP service using Minikube
Enabling Istio in a namespacae
Azure pods app connect to MSSQL server installed in Azure VM
On AKS are System node pool the same as master node?

2 Answers

3/4/2019

I have to add command in order to let the container not finish working.

command: [ "/bin/bash", "-ce", "tail -f /dev/null" ]

-- Sarit
Source: StackOverflow
3/4/2019

The answer is in your question. The describe of your ingress shows the problem.

You did kubectl describe ing and the last part of that output was:

Events:
  Type     Reason     Age                     From                     Message
  ----     ------     ----                    ----                     -------
  Warning  Translate  7m45s (x59 over 4h20m)  loadbalancer-controller  error while evaluating the ingress spec: service "default/webapp1-svc" is type "ClusterIP", expected "NodePort" or "LoadBalancer"; service "default/webapp2-svc" is type "ClusterIP", expected "NodePort" or "LoadBalancer"; service "default/webapp3-svc" is type "ClusterIP", expected "NodePort" or "LoadBalancer"

The important part is:

error while evaluating the ingress spec: service "default/webapp1-svc" is type "ClusterIP", expected "NodePort" or "LoadBalancer"

Solution

Just change all your services to be of type NodePort and it will work.

-- roychri
Source: StackOverflow