I've created a Kubernetes cluster with Windows pool using
gcloud beta container clusters create test-cluster --enable-ip-alias --num-nodes=1 --release-channel=rapid
gcloud container node-pools create rpp2-pool --cluster=test-cluster --image-type=WINDOWS_LTSC --enable-autoupgrade --machine-type=n1-standard-2
gcloud container clusters get-credentials test-clusterAnd deployed a container based on mcr.microsoft.com/windows/servercore:ltsc2019 into it.
When I kubectl exec -it win-webserver powershell into the pod I cannot ping google.com. I can however ping google.com from the VM instance on which the pod is running.
When I kubectl run an image based on linux (busybox) ping works from there.
I've applied these instructions but it still doesn't work: https://cloud.google.com/compute/docs/containers/#mtu_failures
EDIT: I can access other pods in the cluster by their pod names and their service names (BUT only when they are deployed to the same node, look at EDIT2 below). I can ping the neighbor linux VM (default-pool) as well. Cannot ping Default Gateway of the Windows VM though - 10.132.0.1 - not sure if that should be the case.
On the Windows VM Instance:
ipconfig
Windows IP Configuration
Ethernet adapter vEthernet (Ethernet):
Connection-specific DNS Suffix . : europe-west1-b.c.rpp2-261008.internal
Link-local IPv6 Address . . . . . : fe80::2d21:4bd7:fd85:2533%14
IPv4 Address. . . . . . . . . . . : 10.132.0.7
Subnet Mask . . . . . . . . . . . : 255.255.240.0
Default Gateway . . . . . . . . . : 10.132.0.1
Ethernet adapter vEthernet (cbr0):
Connection-specific DNS Suffix . :
Link-local IPv6 Address . . . . . : fe80::1111:61b8:97de:83f8%21
IPv4 Address. . . . . . . . . . . : 10.44.1.2
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . :
Ethernet adapter vEthernet (nat):
Connection-specific DNS Suffix . :
Link-local IPv6 Address . . . . . : fe80::5c57:5e77:1a8a:eccc%9
IPv4 Address. . . . . . . . . . . : 172.21.48.1
Subnet Mask . . . . . . . . . . . : 255.255.240.0
Default Gateway . . . . . . . . . :The pod yaml file i used with kubectl apply:
apiVersion: v1
kind: Service
metadata:
name: win-webserver
labels:
app: win-webserver
spec:
ports:
# the port that this service should serve on
- port: 80
targetPort: 80
selector:
app: win-webserver
type: NodePort
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: win-webserver
name: win-webserver
spec:
replicas: 1
selector:
matchLabels:
app: win-webserver
template:
metadata:
labels:
app: win-webserver
name: win-webserver
spec:
containers:
- name: windowswebserver
image: mcr.microsoft.com/windows/servercore:ltsc2019
command:
- powershell.exe
- -command
- "<#code used from https://gist.github.com/wagnerandrade/5424431#> ; $$listener = New-Object System.Net.HttpListener ; $$listener.Prefixes.Add('http://*:80/') ; $$listener.Start() ; $$callerCounts = @{} ; Write-Host('Listening at http://*:80/') ; while ($$listener.IsListening) { ;$$context = $$listener.GetContext() ;$$requestUrl = $$context.Request.Url ;$$clientIP = $$context.Request.RemoteEndPoint.Address ;$$response = $$context.Response ;Write-Host '' ;Write-Host('> {0}' -f $$requestUrl) ; ;$$count = 1 ;$$k=$$callerCounts.Get_Item($$clientIP) ;if ($$k -ne $$null) { $$count += $$k } ;$$callerCounts.Set_Item($$clientIP, $$count) ;$$ip=(Get-NetAdapter | Get-NetIpAddress); $$header='<html><body><H1>Windows Container Web Server</H1>' ;$$callerCountsString='' ;$$callerCounts.Keys | % { $$callerCountsString+='<p>IP {0} callerCount {1} ' -f $$ip[1].IPAddress,$$callerCounts.Item($$_) } ;$$footer='</body></html>' ;$$content='{0}{1}{2}' -f $$header,$$callerCountsString,$$footer ;Write-Output $$content ;$$buffer = [System.Text.Encoding]::UTF8.GetBytes($$content) ;$$response.ContentLength64 = $$buffer.Length ;$$response.OutputStream.Write($$buffer, 0, $$buffer.Length) ;$$response.Close() ;$$responseStatus = $$response.StatusCode ;Write-Host('< {0}' -f $$responseStatus) } ; "
nodeSelector:
beta.kubernetes.io/os: windows
EDIT2: I noticed i can ping other pods in the cluster by their name but ONLY if they've been deployed to the same node:
NAME READY STATUS RESTARTS AGE IP
win-webserver-75bc4c4c6f-5w9q5 1/1 Running 0 8m34s 10.52.2.4
win-webserver-75bc4c4c6f-d5wlv 1/1 Running 0 22h 10.52.1.4
win-webserver-75bc4c4c6f-pjz57 1/1 Running 0 8m34s 10.52.2.5From the first pod I can ping only the third one by its name not the second, hovewer I can ping the IP 10.52.1.4 fine.
Found the answer here: https://kubernetes.io/docs/setup/production-environment/windows/intro-windows-in-kubernetes/#troubleshooting
3. My Windows Pods do not have network connectivity
If you are using virtual machines, ensure that MAC spoofing is enabled on all the VM network adapter(s).
4. My Windows Pods cannot ping external resources
Windows Pods do not have outbound rules programmed for the ICMP protocol today. However, TCP/UDP is supported. When trying to demonstrate connectivity to resources outside of the cluster, please substitute ping <IP> with corresponding curl <IP> commands.
If you are still facing problems, most likely your network configuration in cni.conf deserves some extra attention. You can always edit this static file. The configuration update will apply to any newly created Kubernetes resources.
One of the Kubernetes networking requirements (see Kubernetes model) is for cluster communication to occur without NAT internally. To honor this requirement, there is an ExceptionList for all the communication where we do not want outbound NAT to occur. However, this also means that you need to exclude the external IP you are trying to query from the ExceptionList. Only then will the traffic originating from your Windows pods be SNAT’ed correctly to receive a response from the outside world. In this regard, your ExceptionList in cni.conf should look as follows:
"ExceptionList": [
"10.244.0.0/16", # Cluster subnet
"10.96.0.0/12", # Service subnet
"10.127.130.0/24" # Management (host) subnet
]
wget google.com -UseBasicParsing returns 200 OK.