K
Q

nginx-ingress stopped working, despite Ready service + pod ("does not have any active Endpoint.")

3/2/2022

I have transferred my microk8s setup to a new server and found that the once-working ingress setup my trial setup stopped working.

I am running this minimal whoami-app:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: whoami
  namespace: default
  labels:
    app: whoami
spec:
  selector:
    matchLabels:
      app: whoami
  template:
    metadata:
      labels:
        app: whoami
    spec:
      containers:
      - name: whoami
        image: containous/whoami
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: whoami
  namespace: default
spec:
  selector:
    app: whoami
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: whoami
  namespace: default
  annotations:
    kubernetes.io/ingress.class: public
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: whoami
            port:
              number: 80

Pod is up and running, service exposed it properly, but the ingress is not working:

kubectl get services whoami
NAME     TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
whoami   ClusterIP   10.152.183.184   <none>        80/TCP    26m

curl 10.152.183.184
Hostname: whoami-567b85d54d-qbbd5
IP: 127.0.0.1
IP: ::1
IP: 10.1.76.7
IP: fe80::e850:aaff:fe72:91c4
RemoteAddr: 192.168.0.102:21910
GET / HTTP/1.1
Host: 10.152.183.184
User-Agent: curl/7.68.0
Accept: */*
kubectl get ingress whoami
NAME     CLASS    HOSTS   ADDRESS     PORTS   AGE
whoami   <none>   *       127.0.0.1   80      28m

The nginx-ingress-controller log shows these entries:

controller.go:1076] Service "default/whoami" does not have any active Endpoint.

But again, accessing through the clusterIP works, so both the Pod and the Service are doing their job.

-- petwri
ingress-controller
kubernetes
kubernetes-ingress
microk8s
nginx-ingress

Similar Questions

How to check kubelet configurations currently in use?
Limit access to Kubernetes secret by RBAC
how does kubernetes service decide which backend pod to route to
How to check a particular script execution is completed by a Kubernetes pod
Logstash with helm in Kubernetes : grok filter not working

2 Answers

3/3/2022

I don't really know how it happened, but the endpoint was not matching the pod IP. I deleted the endpoint manually usind kubectl delete endpoints whoami, it got recreated with the now correct IP, now ingress seems to work.

-- petwri
Source: StackOverflow
3/16/2022

I am glad to hear you managed it work.

I tried to find the reason of that behavior but unfortunately I haven't found it yet. At this link there are some solutions to similar warning. Maybe it will help someone.

First one is situation when ingress class of the ingress controller mismatch ingress class in the ingress resource manifest used for services.

Nginx installation command, short example:

helm install stable/nginx-ingress \   
--name ${INGRESS_RELEASE_NAME} \   
--namespace ${K8S_NAMESPACE} \   
--set controller.scope.enabled=true \   
--set controller.scope.namespace=${K8S_NAMESPACE} \   
--set controller.ingressClass=${NGINX_INGRESS_CLASS}

ingress resource spec. , excerpt:

apiVersion: extensions/v1beta1 
kind: Ingress metadata:  
labels:
   annotations:
    # folowing line is not valid for K8s or Helm, 
    # but reflects the values must be the same
    kubernetes.io/ingress.class: ${NGINX_INGRESS_CLASS}

In second solution is including id directive in service selector:

  selector:
    app: whoami
    id: "0"

See also these questions on StackOverflow: 1. K8 Ingress does not have an endpoint 2. Kubernetes Service does not have active Endpoint 3. Kubernetes - ingress-nginx "no active endpoint" error

-- kkopczak
Source: StackOverflow