I am having an issue with Kubernetes on GKE. I am unable to resolve services by their name using internal DNS.
this is my configuration
Google GKE v1.15
kubectl get namespaces
NAME STATUS AGE
custom-metrics Active 183d
default Active 245d
dev Active 245d
kube-node-lease Active 65d
kube-public Active 245d
kube-system Active 245d
stackdriver Active 198d
I've deployed a couple of simple services based on openjdk 11 docker image and made with spring boot + actuator in order to have a /actuator/health endpoint to test in dev
kubectl get pods --namespace=dev
NAME READY STATUS RESTARTS AGE
test1-5d86946c49-h9t9l 1/1 Running 0 3h1m
test2-5bb5f4ff8d-7mzc8 1/1 Running 0 3h10mIf i try to execute under
kubectl --namespace=dev exec -it test1-5d86946c49-h9t9 -- /bin/bash
root@test1-5d86946c49-h9t9:/app# cat /etc/resolv.conf
nameserver 10.40.0.10
search dev.svc.cluster.local svc.cluster.local cluster.local europe-west1-b.c.back-office-236415.internal c.back-office-236415.internal google.internal
options ndots:5
root@test1-5d86946c49-h9t9:/app# nslookup test2
Server: 10.40.0.10
Address: 10.40.0.10#53
** server can't find test2: NXDOMAINThe same issue occurs if I try using test2 service and try to resolve test1. There is a special configuration for namespace to enable DNS resolve? Shouldn't this be automatic?
I have reproduced this using master version 1.15 and and type of service as ‘ClusterIP’. I am able to do look up from the Pod of one service to another. For creating Kubernetes Services in a Google Kubernetes Engine cluster [1] might be helpful.
To see the services: $ kubectl get svc --namespace=default
To access the deployment: $ kubectl exec -it [Pod Name] sh
To lookup: $ nslookup [Service Name]
Every Service defined in the cluster (including the DNS server itself) is assigned a DNS name. By default, a client Pod’s DNS search list will include the Pod’s own namespace and the cluster’s default domain.
“Normal” (not headless) Services are assigned a DNS A record for a name of the form my-svc.my-namespace.svc.cluster-domain.example. This resolves to the cluster IP of the Service.
For “Headless” (without a cluster IP) Services are also assigned a DNS A record for a name.Though this resolves to the set of IPs of the pods selected by the Service.
However, DNS policies can be set on a per-pod basis. Currently Kubernetes supports the following pod-specific DNS policies. These policies are specified in the dnsPolicy field of a Pod Spec [2]:
“Default“: The Pod inherits the name resolution configuration from the node that the pods run on.
“ClusterFirst“: Any DNS query that does not match the configured cluster domain suffix, such as “www.kubernetes.io”, is forwarded to the upstream nameserver inherited from the node. Cluster administrators may have extra stub-domain and upstream DNS servers configured.
“ClusterFirstWithHostNet“: For Pods running with hostNetwork, need to set its DNS policy “ClusterFirstWithHostNet”.
“None“: It allows a Pod to ignore DNS settings from the Kubernetes environment. All DNS settings are supposed to be provided using the dnsConfig field in the Pod Spec.
[1]-https://cloud.google.com/kubernetes-engine/docs/how-to/exposing-apps [2]-https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#pod-s-dns-config