I have the following configuration to setup the cluster using Rancher (RKE).
rancher-config.yml
nodes:
- address: 192.168.88.204
internal_address: 172.16.22.12
user: dockeruser
role: [controlplane,worker,etcd]
- address: 192.168.88.203
internal_address: 172.16.32.37
user: dockeruser
role: [controlplane,worker,etcd]
- address: 192.168.88.202
internal_address: 172.16.42.73
user: dockeruser
role: [controlplane,worker,etcd]
services:
etcd:
snapshot: true
creation: 6h
retention: 24h
According Rancher Networking, I already open the following port for all nodes(192.168.88.204, 192.168.88.203, 192.168.88.202) as firewall-services.
node-firewall.xml
<?xml version="1.0" encoding="utf-8"?>
<service>
<port port="2376" protocol="tcp"/>
<port port="2379" protocol="tcp"/>
<port port="2380" protocol="tcp"/>
<port port="8472" protocol="udp"/>
<port port="9099" protocol="tcp"/>
<port port="10250" protocol="tcp"/>
<port port="443" protocol="tcp"/>
<port port="6443" protocol="tcp"/>
<port port="8472" protocol="udp"/>
<port port="6443" protocol="tcp"/>
<port port="10254" protocol="tcp"/>
<port port="30000-32767" protocol="tcp"/>
</service>
-> commmend
firewall-offline-cmd --new-service-from-file=node-firewall.xml --name=node-firewall
firewall-cmd --reload
firewall-cmd --add-service node-firewall
My RKE is installed on 192.168.88.151. For RKE ->
rancher-firewall.xml
<?xml version="1.0" encoding="utf-8"?>
<service>
<port port="80" protocol="tcp"/>
<port port="433" protocol="tcp"/>
<port port="22" protocol="tcp"/>
<port port="2376" protocol="tcp"/>
<port port="6443" protocol="tcp"/>
</service>
firewall-offline-cmd --new-service-from-file=rancher-firewall.xml --name=rancher-firewall
firewall-cmd --reload
firewall-cmd --add-service rancher-firewall
So, I run the following commend to up my RKE
rke up --config ./rancher-config.yml
log is
[root@localhost ~]# rke up --config ./rancher-config.yml
INFO[0000] Building Kubernetes cluster
INFO[0000] [dialer] Setup tunnel for host [192.168.88.204]
INFO[0000] [dialer] Setup tunnel for host [192.168.88.203]
INFO[0000] [dialer] Setup tunnel for host [192.168.88.202]
INFO[0001] [network] Deploying port listener containers
INFO[0001] [network] Port listener containers deployed successfully
INFO[0001] [network] Running etcd <-> etcd port checks
INFO[0001] [network] Successfully started [rke-port-checker] container on host [192.168.88.202]
INFO[0001] [network] Successfully started [rke-port-checker] container on host [192.168.88.204]
INFO[0001] [network] Successfully started [rke-port-checker] container on host [192.168.88.203]
FATA[0016] [network] Host [192.168.88.202] is not able to connect to the following ports:
[172.16.22.12:2379, 172.16.22.12:2380, 172.16.32.37:2379, 172.16.32.37:2380, 172.16.42.73:2380, 172.16.42.73:2379].
Please check network policies and firewall rules
My question is how to open the port for the internal_address
for all nodes in kubernates
cluster?
That is incorrect.
The internal_address provides the ability to have nodes with multiple addresses set a specific address to use for inter-host communication on a private network. If the internal_address is not set, the address is used for inter-host communication.
Per https://rancher.com/docs/rke/v0.1.x/en/config-options/nodes/#internal-address
You likely have a firewall issue.
Check your active zones and what interfaces are in those zones.
firewall-cmd --get-active-zones
May be it is lack of my experience. I just share what I found. internal_address
is have to be ip-address of (Gateway) of docker
. To know the ip-address of docker for each node (192.168.88.204, 192.168.88.203, 192.168.88.202).
Run the commend docker network ls
. You might be get following network information.
NETWORK ID NAME DRIVER SCOPE
aa13d08f2676 bridge bridge local
02eabe818790 host host local
1e5bb430d790 none null local
And run the commend docker network inspect bridge
to get ip-addres of bridge
. you will get the following similer info.
[
{
"Name": "bridge",
"Id": "aa13d08f2676e40df5a82521fccc4e402ef6b04f82bcd414cd065a1859b3799d",
"Created": "2019-01-31T21:32:02.381082005-05:00",
"Scope": "local",
"Driver": "bridge",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "172.17.0.0/16",
"Gateway": "172.17.0.1"
}
]
},
....
...
..
.
]
and configure rancher-config.yml
as below and run rke up --config ./rancher-config.yml
again
nodes:
- address: 192.168.88.204
internal_address: 172.17.0.1
...
...
..
..