Configure Kubernetes on Docker Desktop for Windows to Listen to localhost port 80

5/30/2020

I have Windows 10 machine running Hyper-V; the Hyper-V is hosting Docker Desktop for Windows and a Ubuntu Desktop 20.04 LTS VM. I have a Deployment that creates 3 Kubernetes Pods listening on 3 separate ports (2828, 2829, 2830) and 3 NodePort Services with containerPort=80 and named NodePorts (31828, 31829, 31830). From within both Windows and Ubuntu I can browse to "localhost:31828" and see the expected output. I can also browse from Windows to the Ubuntu VM using the Ubuntu VM's IP:NodePort, and I can browse from Ubuntu Desktop to the Windows machine using the Windows IP:NodePort, so the NodePort services seem to be working on both VMs.

My Windows machine is a development environment. All I want to do is to browse to "localhost" from Windows and run my web application on my Kubernetes Docker Desktop for Windows, but this has proved to be an incredibly frustrating endeavor.

I install ingress-nginx using Helm and what I think are the recommended switches on both machines, taken from Bare Metal Considerations Over a NodePort Service and ingress-nginx Configuration and also from lots of trial and error:

helm install --name ingress-nginx ingress-nginx/ingress-nginx --set controller.kind=DaemonSet,controller.service.type=NodePort,controller.hostNetwork=true,controller.dnsPolicy=ClusterFirstWithHostNet,controller.reportNodeInternalIp=true,controller.publishService.enabled=false,controller.service.nodePorts.http=30080

Then I set up an Ingress:

apiVersion:  extensions/v1beta1
kind: Ingress
metadata:
  name: all3-services-ingress
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  backend:
    serviceName: httpdefaultworld
    servicePort: 80
  rules:
  - http:
      paths:
      - path: /goodbye
        backend:
          serviceName: httpgoodbyeworld
          servicePort: 80
      - path: /hello
        backend:
          serviceName: httphelloworld
          servicePort: 80
      - path: /default
        backend:
          serviceName: httpdefaultworld
          servicePort: 80
      - backend:
          serviceName: httpdefaultworld
          servicePort: 80

From both Windows and Ubuntu, I can correctly browse to the Ubuntu VM (192.168.1.133) using its IP Address and the Ingress paths, with and without specifying the 30080 NodePort in the URL. These URLs all work as expected from both my Windows machine and from within the Ubuntu Desktop VM:

    192.168.1.133
    192.168.1.133:30080
    192.168.1.133/hello
    192.168.1.133:30080/hello

In addition, "localhost" works as the URL while browsing from the Ubuntu Desktop VM, with or without the NodePort:

    localhost
    localhost:30080
    localhost/hello
    localhost:30080/hello

But on my Docker for Windows machine (192.168.1.122) I must always use the NodePort in the URL. These URLs work from the Windows machine:

    192.168.1.122:30080/hello
    localhost:30080/hello

and using the Windows IP and NodePort works from the Ubuntu VM:

    192.168.1.122:30080/hello

But these URLs, without the NodePort, do not work as expected on the Windows machine:

    192.168.1.122/hello
    localhost/hello

And using the Windows IP without the NodePort while browsing from the Ubuntu VM also does not work:

    192.168.1.122/hello

Two things I notice: On Windows, the Ingress' Address is some random local IP:

NAME                    HOSTS   ADDRESS        PORTS   AGE
all3-services-ingress   *       192.168.65.3   80      17h

while on Ubuntu the Ingress' Address is the IP of the VM itself:

NAME                    HOSTS   ADDRESS         PORTS   AGE
all3-services-ingress   *       192.168.1.133   80      15h

Also, on Ubuntu, the ingress-nginx-controller service's Endpoints match the Ingress and the VMs' IP Address:

Type:                     NodePort
IP:                       10.107.253.21
Port:                     http  80/TCP
TargetPort:               http/TCP
NodePort:                 http  30080/TCP
Endpoints:                192.168.1.133:80
Port:                     https  443/TCP
TargetPort:               https/TCP
NodePort:                 https  30553/TCP
Endpoints:                192.168.1.133:443
Session Affinity:         None
External Traffic Policy:  Cluster

while on the Windows machine the ngress-nginx-controller service Endpoints are set to the same random IP contained in the Ingress' Address.

Again, the .yaml files I use to create the Pods, Services, Ingress controller and ingress are exactly the same on the Windows and the Ubuntu Kubernetes. Also, nothing is holding on to port 80 on my Windows Machine, not even the PID 4 System process (which I stopped by executing "net stop http").

1 - My immediate concern how to configure Kubernetes or my Kubernetes objects on my Windows machine so that I do not need to use the NodePort in the URL when browsing to the Kubernetes servcies running on my Windows Kubernetes Docker for Desktop. What do I need to do different? Can I / should I somehow "force" the Windows IP Address to be used as the Ingress Address and the ingress-nginx-controller service's Endpoints? Am I completely off-base here using NodePort versus LoadBalancer on my local development machine?

2 - ON Ubuntu Desktop, I am running both the browser and Kubernetes on the VM itself, while on my Windows machine, the browser is running on the physical machine while Kubernetes is running in Docker in a Hyper-V VM. But I assume that Docker (in its default setup) is "sharing" the localhost network connection with Windows (like a sort of implied "controller.hostNetwork=true"). Is this an incorrect assumption? Is this rally some kind of "virtual network configuration" issue?

3 - Kubernetes on the Ubuntu VM was set up using the default configuration of weave.net as its CNI, while the Kubernetes running on Docker Desktop for Windows does not require a CNI. Why does Kubernetes running on Docker Desktop for Windows not require a CNI? I assume that Kubernetes is using and sharing Docker Desktop's "internal" networking as its CNI, while on Ubuntu there is no "Docker Desktop" running so it needs an explicit CNI. If some could point me to some documentation describing why and how Kubernetes Docker Desktop for Windows does not need a CNI, that would be appreciated. Is the lack of a CNI on the Windows Kubernetes installation somehow the cause my "NodePort Required" problem?

Any and all help is appreciated!

-- Steve
bare-metal-server
docker-for-windows
kubernetes-ingress
kubernetes-service
nginx

0 Answers