K
Q

How to expose redirection between pods outside k8s cluster

7/21/2017

I'm to setup druid cluster with k8s and I'm seeking for help about how to expose redirection between pods outside k8s cluster. Say I have two ClusterIP services to expose pods outside k8s.

  • service(10.0.0.1:8080) -> pod(hostname: coordinator)
  • service1(10.0.0.2:8080) -> pod1(hostname: coordinator1)
  • pod and pod1 are druid coordinator groups communicating via Zookepper. Since pod1 is the leader, every request to pod1 will be redirect to pod.
  • In this setup, I'm good with service but facing redirection issues while visiting service.
    • When I visit service1(10.0.0.2:8080) via browser, I'll be redirect to pod via its hostname, i.e. coordinator:8081.
    • However, coordinator is unkown outside k8s cluster and thus unreachable.

Could you please give me some suggestion on how to deal with this situation? Any tips is appreciate.

Here is the return after running wget -S -O - 10.0.0.1:8081

--2017-07-21 16:36:18-- http://10.0.0.1:8081/ Connecting to 10.0.0.1:8081... connected. HTTP request sent, awaiting response... HTTP/1.1 307 Temporary Redirect Date: Fri, 21 Jul 2017 08:36:25 GMT Location: http://coordinator:8081/ Content-Length: 0 Server: Jetty(9.3.16.v20170120) Location: http://coordinator:8081/ [following] --2017-07-21 16:36:18-- http://coordinator:8081/ Resolving coordinator (coordinator)... failed: Temporary failure in name resolution. wget: unable to resolve host address 'coordinator'

-- E. Tang
kubernetes

Similar Questions

Facing issue to setup kubernetes on CentOS 7.3(Master) and Ubuntu 16.04
Can't get wordpress+mysql running on minikube
How can I see a list of currently dormant cronjobs that have not yet ran?
What kubectl command can I use to get events sorted by specific fields and print only specific details of events?

3 Answers

7/21/2017

The problem is not with kubernetes. The application server (Jetty) issues the redirect (to 'coordinator'; not 'coordinator1/2' which is another problem), not kube-proxy.

Probably the simplest solution is to setup an nginx inside your cluster to do reverse proxy to handle that redirect.

(You also don't need 2 services like you have now either)

-- Janos Lenart
Source: StackOverflow
7/21/2017

One solution that comes to my mind (although a bit overenginered) is to make sure you always hit only the leader.

If you can create a readiness check against your pods that returns ok if it is the leader and link them to a common service, that service will always direct to the active leader.

-- Radek 'Goblin' Pieczonka
Source: StackOverflow
2/27/2018

nginx + confd

I had exactly the same problem and found the best solution in installing an nginx reverse proxy, forwarding all traffic to the current leader. The nginx config gets updated by confd, which is watching zookeeper for changes at the overlord/coordinator discovery keys.

To set things up, I found this article quite helpful.

-- Christoph Hösler
Source: StackOverflow