I create a one-replica zookeeper + kafka cluster with the official kafka chart from the official incubator repo:

helm install --name mykafka -f kafka.yaml incubator/kafka

This gives me two pods:

kubectl get pods
NAME                  READY     STATUS
mykafka-kafka-0       1/1       Running
mykafka-zookeeper-0   1/1       Running

And four services (in addition to the default kubernetes service)

kubectl get service
NAME                         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)
kubernetes                   ClusterIP   10.96.0.1       <none>        443/TCP
mykafka-kafka                ClusterIP   10.108.143.59   <none>        9092/TCP
mykafka-kafka-headless       ClusterIP   None            <none>        9092/TCP
mykafka-zookeeper            ClusterIP   10.109.43.48    <none>        2181/TCP
mykafka-zookeeper-headless   ClusterIP   None            <none>        2888/TCP,3888/TCP

If I shell into the zookeeper pod:

> kubectl exec -it mykafka-zookeeper-0 -- /bin/bash

I use the curl tool to test TCP connectivity. I expect a communications error as the server isn't using HTTP, but if curl can't even connect and I have to ctrl-C out, then the TCP connection isn't working.

I can access the local pod through curl localhost:2181:

root@mykafka-zookeeper-0:/# curl localhost:2181
curl: (52) Empty reply from server

I can access other pod through curl mykafka-kafka:9092:

root@mykafka-zookeeper-0:/# curl mykafka-kafka:9092
curl: (56) Recv failure: Connection reset by peer

But I can't access mykafka-zookeeper:2181. That name resolves to the cluster IP, but the attempt to TCP connect hangs until I ctrl-C:

root@mykafka-zookeeper-0:/# curl -v mykafka-zookeeper:2181
* Rebuilt URL to: mykafka-zookeeper:2181/
*   Trying 10.109.43.48...
^C

Similarly, I can shell into the kafka pod:

> kubectl exec -it mykafka-kafka-0 -- /bin/bash

Connecting to the Zookeeper pod by the service name works fine:

root@mykafka-kafka-0:/# curl mykafka-zookeeper:2181
curl: (52) Empty reply from server

Connecting to localhost kafka works fine:

root@mykafka-kafka-0:/# curl localhost:9092
curl: (56) Recv failure: Connection reset by peer

But connecting to the Kafka pod by the service name doesn't work and I must ctrl-C the curl attempt:

curl -v mykafka-kafka:9092
* Rebuilt URL to: mykafka-kafka:9092/
* Hostname was NOT found in DNS cache
*   Trying 10.108.143.59...
^C

Can anyone explain why using I can only connect to a Kubernetes service from outside the service and not from within the service?