Node in EKS doesn't resolve DNS names of RDS (IP working)

4/29/2019

Once again I shall require help from Stack Overflow :).

We have a fresh public access endpoint EKS Cluster, an app inside the nodes that return something from the RDS. The VPC of the cluster is VPC peering with the private VPC that holds the RDS. We also have Accepter DNS resolution enabled. The Accepter is the RDS VPC.

When SSH-ing into my worker nodes, and we telnet the RDS, it resolves it. Initially, the Connection String was establish with the Endpoint. It didn't reach the database. I changed it to the IP of the RDS and it worked.

When doing with the DNS names, it takes:

1) lots of time to load,

2)

"Unable to retrieve Error: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached."

Therefore I was wondering if any of you faced this issue and how you solved it? There seems to be a lot of fun regarding DNS resolution with EKS and I'm not exactly sure why the instance can resolve but not the pod.

Thank you for your help!

-- shrimpy
amazon-eks
amazon-rds
amazon-web-services
dns
kubernetes

1 Answer

4/30/2019

Okay so we found the answer! It was SO LONG to find it, so i'm gonna save you that trouble if you happen to have the same problem/configuration than us.

  1. You need port 53 outbound in NaCL and SG. That's the way kubernetes checks DNS. (DNS problem on AWS EKS when running in private subnets)
  2. In the connection string, Data source, we previously had "Data Source=DNSName;etc". We changed it to "Data source=tcp:DNSName".

That was it

2 days for that. :D

EDIT: I might add I faced the same problem in another environment/aws account (53 was the answer but slightly differently): Pods in EKS: can't resolve DNS (but can ping IP)

-- shrimpy
Source: StackOverflow