Can't access services on master node via public IP (but I can SSH?)

5/24/2019

I've been toying around with kubernetes and have run into an issue. The core of my problem is that while I can access the services on the master node by curling localhost, attempting to access the same via the public ip and port doing the same on another machine (or web browser) hangs forever.

I've configured the cluster using a terraform script (runs as root):

#!/bin/bash -v

exec 3>&1 4>&2
trap 'exec 2>&4 1>&3' 0 1 2 3
exec 1>startup_log.out 2>&1

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -

add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"

cat << EOF | tee /etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF

apt-get update
apt-get install -y docker-ce
apt-mark hold docker-ce
apt-get install -y kubelet kubeadm kubectl kubernetes-cni
apt-mark hold kubelet kubeadm kubectl

kubeadm init --token=${k8stoken} --pod-network-cidr=10.244.0.0/16

mkdir -p /home/ubuntu/.kube
cp -i /etc/kubernetes/admin.conf /home/ubuntu/.kube/config
chown -R $(id -u ubuntu):$(id -g ubuntu) /home/ubuntu/.kube/

usermod -aG docker ubuntu

echo "net.bridge.bridge-nf-call-iptables=1" | tee -a /etc/sysctl.conf
sysctl -p

runuser -l ubuntu -c '\
   kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/bc79dd1505b0c8681ece4de4c0d86c5cd2643275/Documentation/kube-flannel.yml
'

After which I SSH to the master node (works fine) and run:

$ git clone https://github.com/linuxacademy/robot-shop.git
$ kubectl create namespace robot-shop
$ kubectl -n robot-shop create -f ~/robot-shop/K8s/descriptors/
$ ubuntu@ip-10-0-100-167:~$ kubectl -n robot-shop get svc
NAME        TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)              AGE
cart        ClusterIP   10.105.152.135   <none>        8080/TCP             15s
catalogue   ClusterIP   10.97.111.197    <none>        8080/TCP             15s
dispatch    ClusterIP   None             <none>        55555/TCP            15s
mongodb     ClusterIP   10.107.178.183   <none>        27017/TCP            15s
mysql       ClusterIP   10.110.254.52    <none>        3306/TCP             15s
payment     ClusterIP   10.99.195.138    <none>        8080/TCP             15s
rabbitmq    ClusterIP   10.99.70.232     <none>        5672/TCP,15672/TCP   15s
ratings     ClusterIP   10.98.80.21      <none>        80/TCP               15s
redis       ClusterIP   10.101.232.84    <none>        6379/TCP             15s
shipping    ClusterIP   10.106.246.97    <none>        8080/TCP             15s
user        ClusterIP   10.109.120.146   <none>        8080/TCP             15s
web         NodePort    10.97.162.113    <none>        8080:30080/TCP       15s

Services appear to startup fine but I can't access anything on this host externally. I've tried following this debugging guide, and identified a discrepancy that when I type:

$ iptables-save | grep 

I get nothing. But the doc doesn't make a recommendation as to a fix. Has anyone seen this issue before? I'm concerned my config file has a problem with it but I can't figure out what!

-- Answorth
kubernetes
terraform

1 Answer

5/29/2019

As @Ashworth mentioned in the comment issue has been solved after appropriate service port enabled in the particular security group for the relevant master node instance.

-- mk_sta
Source: StackOverflow