I provisioned kubernetes cluster using kops:
kops create cluster \
--zones ap-southeast-1a \
--dns-zone=${DNS_ZONE} \
--node-count 3 \
--vpc=${VPC_ID} \
--subnets=${SUBNET_ID} \
${NAME}
Ec2 instances are provisioned and I can see them in console.
DNS_ZONE is abc.in, a public hosted zone in route53
api.sp-southeast-1.es.k8s.abc.in. A 54.123.123.123
api.internal.sp-southeast-1.es.k8s.abc.in. A 172.123.123.123
etcd-a.internal.sp-southeast-1.es.k8s.abc.in. A 172.123.123.123
etcd-events-a.internal.sp-southeast-1.es.k8s.abc.in. A 172.123.123.123
I've gone through the stack overflow discussions on the same problem.
I'm using 3 t2.micro instances for master and 3 t2.medium instances for Nodes
Inbound rules for master node are :
Custom TCP Rule TCP 4003 - 65535 sg-0559ecf7ab (nodes.sp-southeast-1.es.k8s.abc.in)
Custom TCP Rule TCP 2382 - 4000 sg-0559674a0d3ab (nodes.sp-southeast-1.es.k8s.abc.in)
All traffic All All sg-0ecc38031f44 (masters.sp-southeast-1.es.k8s.abc.in)
SSH TCP 22 0.0.0.0/0
Custom UDP Rule UDP 1 - 65535 sg-0557674a0d3ab (nodes.sp-southeast-1.es.k8s.abc.in)
Custom TCP Rule TCP 1 - 2379 sg-0557674a0d3ab (nodes.sp-southeast-1.es.k8s.abc.in)
Custom TCP Rule TCP 443 0.0.0.0/0
when I enter
kops validate cluster
> unexpected error during validation: error listing nodes:
Get https://api.sp-southeast-1.es.k8s.abc.in/api/v1/nodes:
dial tcp 54.123.123.123:443: i/o timeout
I have upgraded my production kube cluster and I encounter the same problem. We use AWS cloud.
The problem was due to the incorrect dns configuration (AWS Route 53) of the cluster api, maybe is a bug into kops?
To solve I manually set the correct ip address for the kube api and then the api is now reachable.
UPDATE: I found that the problem was due to an incorrect kops upgrade, the kubelet responsible for the network service was not run correctly. The cluster use a 1.10 kubernetes but aws ec2 images are at 1.9. Force cluster to 1.9.8 version, the cluster upgrade run correctly and the kubelet network service updates dns correctly.