Are Services in Kubernetes simply reverse proxies that allow a constant point of contact?

10/4/2018

I'm researching Kubernetes Services (that is one of the kinds of k8s components, much like Pods and ReplicaSets). They seem to function like reverse proxies, but I know k8s internally uses DNS, so perhaps they are some kind of load balancing DNS? Also it would somehow mean, that since a Pod can be relocated or exist on many nodes it couldn't simply be a Reverse Proxy since it too would need to be addressable, but share a single IP on many machines... (obviously struggling to imagine how they were built without looking directly at the source code -- yet).

What makes up a K8s Service? DNS + Reverse Proxy, or something more/less? Some kind of networking trick?

-- lucidquiet
dns
kubernetes
reverse-proxy

2 Answers

10/4/2018

Regular ClusterIP services

Ensuring network connectivity for type: ClusterIP Services is the responsibility of the kube-proxy -- a component that typically runs on each and every node of your cluster. The kube-proxy does this by intercepting outgoing traffic from Pods on each node and filtering traffic targeted at service IPs. Since it is connected to the Kubernetes API, the kube-proxy can determine which Pod IPs are associated with each service IP and can forward the traffic accordingly.

Conceptually, the kube-proxy might be considered similar to a reverse proxy (hence the name), but typically uses IPtables rules (or, starting at Kubernetes 1.9 optionally IPVS). Each created service will result in a set of IPtables rules on every node that intercepts and forwards traffic targeted at the service IP to the respective Pod IPs (service IPs are purely virtual and exist only in these IPtables rules; nowhere in the entire cluster you will find an actual network interface holding that IP).

Load balancing is also implemented via IPtables rules (or IPVS). Load balancing always occurs on the source node that the traffic originates from.

Here's an example from the Debug Services section of the documentation:

u@node$ iptables-save | grep hostnames
-A KUBE-SEP-57KPRZ3JQVENLNBR -s 10.244.3.6/32 -m comment --comment "default/hostnames:" -j MARK --set-xmark 0x00004000/0x00004000
-A KUBE-SEP-57KPRZ3JQVENLNBR -p tcp -m comment --comment "default/hostnames:" -m tcp -j DNAT --to-destination 10.244.3.6:9376
-A KUBE-SEP-WNBA2IHDGP2BOBGZ -s 10.244.1.7/32 -m comment --comment "default/hostnames:" -j MARK --set-xmark 0x00004000/0x00004000
-A KUBE-SEP-WNBA2IHDGP2BOBGZ -p tcp -m comment --comment "default/hostnames:" -m tcp -j DNAT --to-destination 10.244.1.7:9376
-A KUBE-SEP-X3P2623AGDH6CDF3 -s 10.244.2.3/32 -m comment --comment "default/hostnames:" -j MARK --set-xmark 0x00004000/0x00004000
-A KUBE-SEP-X3P2623AGDH6CDF3 -p tcp -m comment --comment "default/hostnames:" -m tcp -j DNAT --to-destination 10.244.2.3:9376
-A KUBE-SERVICES -d 10.0.1.175/32 -p tcp -m comment --comment "default/hostnames: cluster IP" -m tcp --dport 80 -j KUBE-SVC-NWV5X2332I4OT4T3
-A KUBE-SVC-NWV5X2332I4OT4T3 -m comment --comment "default/hostnames:" -m statistic --mode random --probability 0.33332999982 -j KUBE-SEP-WNBA2IHDGP2BOBGZ
-A KUBE-SVC-NWV5X2332I4OT4T3 -m comment --comment "default/hostnames:" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-X3P2623AGDH6CDF3
-A KUBE-SVC-NWV5X2332I4OT4T3 -m comment --comment "default/hostnames:" -j KUBE-SEP-57KPRZ3JQVENLNBR

For more information, have a look at the Virtual IPs and Service Proxies section in the manual.

Headless services

Besides regular ClusterIP services, there are also Headless Services (which are declares by specifying the property clusterIP: None when creating the service). These will not use the kube-proxy; instead, their DNS hostname will directly resolve to all Pod IPs that are associated with the service. Load balancing is achieved via regular DNS round-robin.

-- helmbert
Source: StackOverflow

10/4/2018

For services and routing within the cluster mostly iptables except when you are explicitly using ipvs (which would partially use iptables) or something like Cillium (which allows you to plugin BFP). The kube-proxy in each node manages the iptable rules. More information on how the proxy modes here.

On the DNS side, Kubernetes runs either kube-dns (1.10 or older) or coredns (1.11 or newer) and essentially all services and pods get registered in the that DNS service for other pods/containers to find them. For example, a service will have an FQDN like this: my-service.my-namespace.svc.cluster.local More information on that here.

-- Rico
Source: StackOverflow