calico network dependency on killall.sh in k3s

9/2/2021

I have a k3s cluster that have system pods with calico policy applied:

kube-system   pod/calico-node-xxxx                          
kube-system   pod/calico-kube-controllers-xxxxxx   
kube-system   pod/metrics-server-xxxxx
kube-system   pod/local-path-provisioner-xxxxx
kube-system   pod/coredns-xxxxx
app-system    pod/my-app-xxxx

I ran /usr/local/bin/k3s-killall.sh to clean up containers and networks. Will this clean/remove/reset my calico networking also? (though after killall.sh the iptables of calico still present)

Quoting from the killall.sh link:

The killall script cleans up containers, K3s directories, and networking components while also removing the iptables chain with all the associated rules.

It says that networking component will also be cleaned up though but is it kubernetes networking or any networking applied to cluster?

-- solveit
calico
k3s
kubernetes
kubernetes-networkpolicy
project-calico

1 Answer

9/2/2021

When you install k3s based on the instructions here it won't install Calico CNI by default. There is a need to install Calico CNI separately.

To answer you question, let's analyse /usr/local/bin/k3s-killall.sh file, especially the part with iptables command:

...
iptables-save | grep -v KUBE- | grep -v CNI- | iptables-restore

As can see, this command only removes iptables chains starting with KUBE or CNI.

If you run command iptables -S on cluster setup with k3s and Calico CNI you can see that chains used by Calico are starting with cali-:

 iptables -S
-A cali-FORWARD -m comment --comment "cali:vjrMJCRpqwy5oRoX" -j MARK --set-xmark 0x0/0xe0000
-A cali-FORWARD -m comment --comment "cali:A_sPAO0mcxbT9mOV" -m mark --mark 0x0/0x10000 -j cali-from-hep-forward
...

Briefly answering your questions:

I ran /usr/local/bin/k3s-killall.sh to clean up containers and networks. Will this clean/remove/reset my calico networking also ?

No. There will be still some of Calico CNI components for example earlier mentioned iptables chains but also network interfaces:

 ip addr
 6: calicd9e5f8ac65@if4: <BROADCAST,MULTICAST,UP,LOWER_UP>
 7: cali6fcd2eeafde@if4: <BROADCAST,MULTICAST,UP,LOWER_UP>
 ...

It says that networking component will also be cleaned up though, but is it kubernetes networking or any networking applied to cluster ?

Those are network components provided by k3s by default like earlier mentioned KUBE- and CNI- iptables chains. To get more information what exatcly k3s-killall.sh script does, I'd recommend reading it's code (k3s-killall.sh script starting from # --- create killall script ---, line 575).

-- Mikolaj S.
Source: StackOverflow