Why sometimes kubernetes has 2 internal endpoints for a service, sometimes 4? why do the internal endpoints always come in pairs?
This is based on my loose understanding of things and an assumption. The assumption is that this seems to be the case when the cluster is deployed to GKE.
Since I don't have Kafka manager installed, I will use the example of Kubernetes service, which has a similar port configuration in the console. This service is of type ClusterIP
Name         Cluster IP     Internal Endpoints
Kubernetes   10.11.240.1    kubernetes:443 TCP
                            kubernetes:0 TCP
The port 0 is added by GKE Ingress to randomly select a port for forwarding, as explained here and also related discussion here
In case of NodePort service, it is a different story.
Name         Cluster IP     Internal Endpoints
hello-web    10.11.249.126  helloweb:8080 TCP
                            helloweb:30193 TCP
This can be also seen in the service description. Since service is already exposed on a nodeport, there is no need to additionally expose on a random port.
$kubectl describe service helloweb-backend -n default
Name:           helloweb-backend
Type:           NodePort
IP:             10.11.249.126
Port:           <unset> 8080/TCP
NodePort:       <unset> 30193/TCP
Endpoints:      10.8.3.3:8080