I'm trying to get a windows (server 2019) based docker container running inside a kubernetes cluster, using the mcr.microsoft.com/windows/servercore:1809
image from here
The node is recognised by the cluster and shown as ready.
When I try to deploy any pod to the cluster it runs without problems, but it's not possible to access the exposed ports associated with the service.
It appears that there's an issue with running kube-proxy
on the windows worker node. The log for that node show:
Failed create pod sandbox: rpc error: code = Unknown desc = failed to start sandbox container for pod "kube-proxy-nwclf": Error response from daemon: network host not found
Any ideas on where I'm going wrong here?
UPDATE: Just noticed that the kube-proxy pod on the windows machine has a cluster based ip address (10.244.5.2
) rather than one from my local network (192.168.1.X
). Is this relevant in terms of working out what the problem is?
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
kube-system kube-proxy-9jq8w 1/1 Running 1 15d 192.168.1.9 k8s-mr <none> <none>
kube-system kube-proxy-h5gx7 0/1 ContainerCreating 0 13s 10.244.5.2 wins2019-worker <none> <none>
Detail from describe pod shown below.
user@k8s-mr:~/kube_projects/testdemo-4.1/app$ kubectl -n kube-system describe pod kube-proxy-nwclf
Name: kube-proxy-nwclf
Namespace: kube-system
Priority: 2000001000
PriorityClassName: system-node-critical
Node: wins2019-worker/192.168.1.10
Start Time: Wed, 17 Apr 2019 13:35:30 +0100
Labels: controller-revision-hash=b7775b676
k8s-app=kube-proxy
pod-template-generation=1
Annotations: <none>
Status: Pending
IP: 192.168.1.10
Controlled By: DaemonSet/kube-proxy
Containers:
kube-proxy:
Container ID:
Image: k8s.gcr.io/kube-proxy:v1.14.0
Image ID:
Port: <none>
Host Port: <none>
Command:
/usr/local/bin/kube-proxy
--config=/var/lib/kube-proxy/config.conf
--hostname-override=$(NODE_NAME)
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Environment:
NODE_NAME: (v1:spec.nodeName)
Mounts:
/lib/modules from lib-modules (ro)
/run/xtables.lock from xtables-lock (rw)
/var/lib/kube-proxy from kube-proxy (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-proxy-token-f4tqx (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
kube-proxy:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: kube-proxy
Optional: false
xtables-lock:
Type: HostPath (bare host directory volume)
Path: /run/xtables.lock
HostPathType: FileOrCreate
lib-modules:
Type: HostPath (bare host directory volume)
Path: /lib/modules
HostPathType:
kube-proxy-token-f4tqx:
Type: Secret (a volume populated by a Secret)
SecretName: kube-proxy-token-f4tqx
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations:
CriticalAddonsOnly
node.kubernetes.io/disk-pressure:NoSchedule
node.kubernetes.io/memory-pressure:NoSchedule
node.kubernetes.io/network-unavailable:NoSchedule
node.kubernetes.io/not-ready:NoExecute
node.kubernetes.io/pid-pressure:NoSchedule
node.kubernetes.io/unreachable:NoExecute
node.kubernetes.io/unschedulable:NoSchedule
Events:
Type Reason Age From Message
------
Normal Scheduled 70s default-scheduler Successfully assigned kube-system/kube-proxy-nwclf to wins2019-worker
Normal SandboxChanged 2s (x12 over 61s) kubelet, wins2019-worker Pod sandbox changed, it will be killed and re-created.
Warning FailedCreatePodSandBox 0s (x13 over 62s) kubelet, wins2019-worker Failed create pod sandbox: rpc error: code = Unknown desc = failed to start sandbox container for pod "kube-proxy-nwclf": Error response from daemon: network host not found
Fixed it now, I had misunderstood that kube-proxy
should not run at all on the windows node.
I had to amend the kubernetes master and apply a NodeSelector
to ensure that the appropriate pods are setup for each node based on its OS.
wget https://raw.githubusercontent.com/Microsoft/SDN/master/Kubernetes/flannel/l2bridge/manifests/node-selector-patch.yml
kubectl patch ds/kube-proxy --patch "$(cat node-selector-patch.yml)" -n=kube-system