I have Windows 10 machine running Hyper-V; the Hyper-V is hosting Docker Desktop for Windows and a Ubuntu Desktop 20.04 LTS VM. I have a Deployment that creates 3 Kubernetes Pods listening on 3 separate ports (2828, 2829, 2830) and 3 NodePort Services with containerPort=80 and named NodePorts (31828, 31829, 31830). From within both Windows and Ubuntu I can browse to "localhost:31828" and see the expected output. I can also browse from Windows to the Ubuntu VM using the Ubuntu VM's IP:NodePort, and I can browse from Ubuntu Desktop to the Windows machine using the Windows IP:NodePort, so the NodePort services seem to be working on both VMs.
My Windows machine is a development environment. All I want to do is to browse to "localhost" from Windows and run my web application on my Kubernetes Docker Desktop for Windows, but this has proved to be an incredibly frustrating endeavor.
I install ingress-nginx using Helm and what I think are the recommended switches on both machines, taken from Bare Metal Considerations Over a NodePort Service and ingress-nginx Configuration and also from lots of trial and error:
helm install --name ingress-nginx ingress-nginx/ingress-nginx --set controller.kind=DaemonSet,controller.service.type=NodePort,controller.hostNetwork=true,controller.dnsPolicy=ClusterFirstWithHostNet,controller.reportNodeInternalIp=true,controller.publishService.enabled=false,controller.service.nodePorts.http=30080
Then I set up an Ingress:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: all3-services-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
backend:
serviceName: httpdefaultworld
servicePort: 80
rules:
- http:
paths:
- path: /goodbye
backend:
serviceName: httpgoodbyeworld
servicePort: 80
- path: /hello
backend:
serviceName: httphelloworld
servicePort: 80
- path: /default
backend:
serviceName: httpdefaultworld
servicePort: 80
- backend:
serviceName: httpdefaultworld
servicePort: 80
From both Windows and Ubuntu, I can correctly browse to the Ubuntu VM (192.168.1.133) using its IP Address and the Ingress paths, with and without specifying the 30080 NodePort in the URL. These URLs all work as expected from both my Windows machine and from within the Ubuntu Desktop VM:
192.168.1.133
192.168.1.133:30080
192.168.1.133/hello
192.168.1.133:30080/hello
In addition, "localhost" works as the URL while browsing from the Ubuntu Desktop VM, with or without the NodePort:
localhost
localhost:30080
localhost/hello
localhost:30080/hello
But on my Docker for Windows machine (192.168.1.122) I must always use the NodePort in the URL. These URLs work from the Windows machine:
192.168.1.122:30080/hello
localhost:30080/hello
and using the Windows IP and NodePort works from the Ubuntu VM:
192.168.1.122:30080/hello
But these URLs, without the NodePort, do not work as expected on the Windows machine:
192.168.1.122/hello
localhost/hello
And using the Windows IP without the NodePort while browsing from the Ubuntu VM also does not work:
192.168.1.122/hello
Two things I notice: On Windows, the Ingress' Address is some random local IP:
NAME HOSTS ADDRESS PORTS AGE
all3-services-ingress * 192.168.65.3 80 17h
while on Ubuntu the Ingress' Address is the IP of the VM itself:
NAME HOSTS ADDRESS PORTS AGE
all3-services-ingress * 192.168.1.133 80 15h
Also, on Ubuntu, the ingress-nginx-controller service's Endpoints match the Ingress and the VMs' IP Address:
Type: NodePort
IP: 10.107.253.21
Port: http 80/TCP
TargetPort: http/TCP
NodePort: http 30080/TCP
Endpoints: 192.168.1.133:80
Port: https 443/TCP
TargetPort: https/TCP
NodePort: https 30553/TCP
Endpoints: 192.168.1.133:443
Session Affinity: None
External Traffic Policy: Cluster
while on the Windows machine the ngress-nginx-controller service Endpoints are set to the same random IP contained in the Ingress' Address.
Again, the .yaml files I use to create the Pods, Services, Ingress controller and ingress are exactly the same on the Windows and the Ubuntu Kubernetes. Also, nothing is holding on to port 80 on my Windows Machine, not even the PID 4 System process (which I stopped by executing "net stop http").
1 - My immediate concern how to configure Kubernetes or my Kubernetes objects on my Windows machine so that I do not need to use the NodePort in the URL when browsing to the Kubernetes servcies running on my Windows Kubernetes Docker for Desktop. What do I need to do different? Can I / should I somehow "force" the Windows IP Address to be used as the Ingress Address and the ingress-nginx-controller service's Endpoints? Am I completely off-base here using NodePort versus LoadBalancer on my local development machine?
2 - ON Ubuntu Desktop, I am running both the browser and Kubernetes on the VM itself, while on my Windows machine, the browser is running on the physical machine while Kubernetes is running in Docker in a Hyper-V VM. But I assume that Docker (in its default setup) is "sharing" the localhost network connection with Windows (like a sort of implied "controller.hostNetwork=true"). Is this an incorrect assumption? Is this rally some kind of "virtual network configuration" issue?
3 - Kubernetes on the Ubuntu VM was set up using the default configuration of weave.net as its CNI, while the Kubernetes running on Docker Desktop for Windows does not require a CNI. Why does Kubernetes running on Docker Desktop for Windows not require a CNI? I assume that Kubernetes is using and sharing Docker Desktop's "internal" networking as its CNI, while on Ubuntu there is no "Docker Desktop" running so it needs an explicit CNI. If some could point me to some documentation describing why and how Kubernetes Docker Desktop for Windows does not need a CNI, that would be appreciated. Is the lack of a CNI on the Windows Kubernetes installation somehow the cause my "NodePort Required" problem?
Any and all help is appreciated!