I installed a kubernetes v1.16 cluster with two nodes, and enabled "IPv4/IPv6 dual-stack", following this guide. For "dual-stack", I set --network-plugin=kubenet to kubelet.
Now, the pods have ipv4 and ipv6 address, and each node has a cbr0 gw with both ipv4 and ipv6 address. But when I ping from a node to the cbr0 gw of other node, it failed.
I tried to add route as follow manually: "ip route add [podCIDR of other node] via [ipaddress of other node]"
After I added the route on two nodes, I can ping cbr0 gw successful with ipv4. But "adding route manually" does not seem to be a correct way.
When I use kubenet, how should I config to ping from a node to the cbr0 gw of other node?
Kubenet is a requirement for enabling IPv6 and as you stated, kubenet have some limitations and here we can read:
Kubenet is a very basic, simple network plugin, on Linux only. It does not, of itself, implement more advanced features like cross-node networking or network policy. It is typically used together with a cloud provider that sets up routing rules for communication between nodes, or in single-node environments.
I would like to highlight that kubenet is not creating routes automatically for you.
Based on this information we can understand that in your scenario this is the expected behavior and there is no problem happening. If you want to keep going in this direction you need to create routes manually.
It's important to remember this is an alpha feature (WIP).
There is also some work being done to make it possible to bootstrap a Kubernetes cluster with Dual Stack using Kubeadm, but it's not usable yet and there is no ETA for it.
There are some examples of IPv6 and dual-stack setups with other networking plugins in this repository, but it still require adding routes manually.
This project serves two primary purposes: (i) study and validate ipv6 support in kubernetes and associated plugins (ii) provide a dev environment for implementing and testing additional functionality (e.g.dual-stack)