So, I was experimenting and added a Service with an ExternalIP set to the end hosts ipv6 address.

This is a configuration that's accepted, but not supported, in Kubernetes, and causes the kube-proxy to do fail.

From the kube-proxy logs:

-A KUBE-SERVICES -m comment --comment "default/frontend: external IP" -m tcp -p tcp -d 207.154.225.168/32 --dport 80 -j KUBE-MARK-MASQ
-A KUBE-SERVICES -m comment --comment "default/frontend: external IP" -m tcp -p tcp -d 207.154.225.168/32 --dport 80 -m physdev ! --physdev-is-in -m addrtype ! --src-type LOCAL -j KUBE-SVC-GYQQTB6TY565JPRW
-A KUBE-SERVICES -m comment --comment "default/frontend: external IP" -m tcp -p tcp -d 207.154.225.168/32 --dport 80 -m addrtype --dst-type LOCAL -j KUBE-SVC-GYQQTB6TY565JPRW
-A KUBE-SERVICES -m comment --comment "default/frontend: external IP" -m tcp -p tcp -d 2a03:b0c0:3:d0::43bb:4001/32 --dport 80 -j KUBE-MARK-MASQ
-A KUBE-SERVICES -m comment --comment "default/frontend: external IP" -m tcp -p tcp -d 2a03:b0c0:3:d0::43bb:4001/32 --dport 80 -m physdev ! --physdev-is-in -m addrtype ! --src-type LOCAL -j KUBE-SVC-GYQQTB6TY565JPRW

E0502 07:38:39.913815 1 proxier.go:1312] Failed to execute iptables-restore: exit status 2 (iptables-restore v1.4.21: host/network 2a03:b0c0:3:d0::43bb:4001' not found Error occurred at line: 53 Tryiptables-restore -h' or 'iptables-restore --help' for more information.

This is the problem: There is no "default/frontend" service defined. I have explicitly deleted it,

kubectl get services --namespace=default
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes 10.3.0.1 443/TCP 3d

However, kube-proxy on all my nodes still attempts to add these, which causes all services created after this misstep to break.

So, the question is, how do I purge this ghost of a service from my cluster?