I've AWS EKS cluster in my organization for internal practice purpose.
Currently installed applications on this cluster are - Jenkins, Nexus, Spinnaker, Grafana, Prometheus
Now in order to save the cost and optimal usage, I used to scale up to 8 nodes every morning and down to 0 nodes every evening.
I've observed that sometimes when I wanted to see the kubectl logs
or run kubectl exec
command I used to get this error
Error from server: no preferred addresses found; known addresses: []
To quickly resolve it, I delete the pod and let it created automatically again, and then this error would gone.
However, today I'm running jenkins build, where Jenkins master spin up a jenkins-slave pod dynamically to run the build process, and everytime I'm getting the same error again in the jenkins logs
java.net.ProtocolException: Expected HTTP 101 response but was '500 Internal Server Error'
hudson.remoting.ProxyException: io.fabric8.kubernetes.client.KubernetesClientException: no preferred addresses found; known addresses: []
Please advise how to resolve this issue as its becoming frequent now.
I've also found that in worker nodes Internal IPs are not assigned, may be due to that, this issue is occurring.
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx Ready <none> 3h7m v1.12.10-eks-aae39f <none> <none> Amazon Linux 2 4.14.171-136.231.amzn2.x86_64 docker://18.9.9
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx Ready <none> 3h7m v1.12.10-eks-aae39f <none> <none> Amazon Linux 2 4.14.171-136.231.amzn2.x86_64 docker://18.9.9
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx Ready <none> 3h7m v1.12.10-eks-aae39f <none> <none> Amazon Linux 2 4.14.171-136.231.amzn2.x86_64 docker://18.9.9
Thanks