No nodes available in AWS EKS

3/22/2019

I am trying to setup an AWS EKS cluster and followed the instructions in https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html. But while trying to setup the dashboard I realized I do not have any nodes and don't know why.

Yes, I made sure that the CloudFormation stack for my nodes has the exact name of my EKS cluster and I applied the aws-auth-cm.yaml with the stacks NodeInstanceRole.

A similar question came up here Amazon Kubernetes AWS-EKS is not getting created properly or not synched with kubectl but there is no usable information there.

Here is some info:

$ kubectl get svc
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   <IP>         <none>        443/TCP   11d

$ kubectl get all --namespace=default
NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
service/kubernetes   ClusterIP   <IP>         <none>        443/TCP   11d

$ kubectl get nodes --namespace=kube-system
No resources found.

get nodes gives No resources found for any of the namespaces I have: kube-system, kube-public, or default.

$ kubectl get all --namespace=kube-system
NAME                                        READY     STATUS    RESTARTS   AGE
pod/heapster-7ff8d6bf9f-qv56t               0/1       Pending   0          44m
pod/kube-dns-69ff9fcd66-ls9w7               0/3       Pending   0          11d
pod/kubernetes-dashboard-669f9bbd46-xthpq   0/1       Pending   0          45m
pod/monitoring-influxdb-cc95575b9-qjk4r     0/1       Pending   0          44m

NAME                           TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)         AGE
service/heapster               ClusterIP   <IP>             <none>        80/TCP          44m
service/kube-dns               ClusterIP   <IP>             <none>        53/UDP,53/TCP   11d
service/kubernetes-dashboard   ClusterIP   <IP>             <none>        443/TCP         45m
service/monitoring-influxdb    ClusterIP   <IP>             <none>        8086/TCP        44m

NAME                                            DESIRED   CURRENT   READY     UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
daemonset.apps/aws-node                         0         0         0         0            0           <none>          11d
daemonset.apps/kube-proxy                       0         0         0         0            0           <none>          11d
daemonset.apps/nvidia-device-plugin-daemonset   0         0         0         0            0           <none>          34m

NAME                                   DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/heapster               1         1         1            0           44m
deployment.apps/kube-dns               1         1         1            0           11d
deployment.apps/kubernetes-dashboard   1         1         1            0           45m
deployment.apps/monitoring-influxdb    1         1         1            0           44m

NAME                                              DESIRED   CURRENT   READY     AGE
replicaset.apps/heapster-7ff8d6bf9f               1         1         0         44m
replicaset.apps/kube-dns-69ff9fcd66               1         1         0         11d
replicaset.apps/kubernetes-dashboard-669f9bbd46   1         1         0         45m
replicaset.apps/monitoring-influxdb-cc95575b9     1         1         0         44m

UPDATE:

After I send a Kubernetes API call from my backend to launch some pods I get the following output (kubectl get jobs is similar) but still no actual instances spin up and the jobs never execute.

$ kubectl get svc
kubernetes                                      ClusterIP   <IP>     <none>        443/TCP                                                       14d
svc-3d1de3dc-7ca1-1f83-7a5e-548e5c5b4d12        NodePort    <IP>     <none>        8082:32433/TCP,8081:30374/TCP,8096:31178/TCP,8083:31067/TCP   1m
svc-3d1de3dc-7ca1-1f83-7a5e-548e5c5b4d12-sut0   ClusterIP   <IP>     <none>        50001/TCP                                                     1m
svc-7429ae6b-6f4b-012b-2d93-7c244109090b        NodePort    <IP>     <none>        8082:30736/TCP,8081:31272/TCP,8096:30857/TCP,8083:30949/TCP   1m
svc-7429ae6b-6f4b-012b-2d93-7c244109090b-sut0   ClusterIP   <IP>     <none>        50001/TCP                                                     1m
svc-89e81309-f6e4-21b1-ab8d-ecfb74d08010        NodePort    <IP>     <none>        8082:32072/TCP,8081:31236/TCP,8096:30000/TCP,8083:31257/TCP   1m
svc-89e81309-f6e4-21b1-ab8d-ecfb74d08010-sut0   ClusterIP   <IP>     <none>        50001/TCP                                                     1m
svc-c196e61f-098e-7b10-9d05-6d06ccbdbb9f        NodePort    <IP>     <none>        8082:32541/TCP,8081:30305/TCP,8096:31738/TCP,8083:31846/TCP   1m
svc-c196e61f-098e-7b10-9d05-6d06ccbdbb9f-sut0   ClusterIP   <IP>     <none>        50001/TCP                                                     1m
-- kilgoretrout
amazon-eks
amazon-web-services
aws-eks
kubernetes

0 Answers