Nginx resolver for Kubernetes with skydns

11/5/2018

I can't find a way to make an nginx pod resolve another kubernetes services URLs.

I am NOT using kube-dns , we are using kube2sky solely and we are not going to implement kube-dns yet, so I need to fix in this scenario.

For example, I want nginx to resolve a service URL app.mynamespace.svc.skydns.local but if I run a ping to that URL it resolves successfully.

My nginx config part is:

location /api/v1/namespaces/mynamespace/services/app/proxy/ {
          resolver 127.0.0.1
          set \$endpoint \"http://app.mynamespace.svc.skydns.local/\";
          proxy_pass \$endpoint;
          proxy_http_version 1.1;
          proxy_set_header Connection \"upgrade\";
        }

I need to specify the target upstream in a variable because I want nginx to starts even if the target is not available, if I don't specify in variable nginx crashes when starting up because the upstream needs to be available and resolvable.

The problem I think is the resolver value, I've tried with 127.0.0.1, with 127.0.0.11, and with the skydns IP specified in configuration 172.40.0.2:53:

etcdctl get /skydns/config
{"dns_addr":"0.0.0.0:53","ttl":4294967290,"nameservers":["172.40.0.2:53"]}

But nginx cannot resolve the URL yet.

What IP should I specify in the resolver field in nginx config for kubernetes and skydns config?

Remember that we don't have kube-dns.

Thank you.

-- Claudio Saavedra
dns
kubernetes
nginx
reverse-proxy
skydns

1 Answer

11/5/2018

I don't think resolving app.mynamespace.svc.skydns.local has anything to do with configuring the upstream DNS servers. Generally, for that, you configure a well-known DNS server like 8.8.8.8 or your cloud infrastructure DNS server which would be perhaps 172.40.0.2. For example as described in the docs:

$ curl -XPUT http://127.0.0.1:4001/v2/keys/skydns/config \
-d value='{"dns_addr":"127.0.0.1:5354","ttl":3600, "nameservers": ["8.8.8.8:53","8.8.4.4:53"]}'

You might want to check the logs of your kube2sky2 pod, for any guidance and that all the config options are specified like --kube-master-url, --etcd-server. Maybe it can't talk to the Kubernetes master and receive updates of running pods so that the SRV entries get updates.

-- Rico
Source: StackOverflow