New AKS cluster unreachable via network (including dashboard)

3/29/2018

Yesterday I spun up an Azure Kubernetes Service cluster running a few simple apps. Three of them have exposed public IPs that were reachable yesterday.

As of this morning I can't get the dashboard tunnel to work or the LoadBalancer IPs themselves.

I was asked by the Azure twitter account to solicit help here.

I don't know how to troubleshoot this apparent network issue - only az seems to be able to touch my cluster.

dashboard error log

❯❯❯ make dashboard ~/c/azure-k8s (master) az aks browse --resource-group=akc-rg-cf --name=akc-237 Merged "akc-237" as current context in /var/folders/9r/wx8xx8ls43l8w8b14f6fns8w0000gn/T/tmppst_atlw Proxy running on http://127.0.0.1:8001/ Press CTRL+C to close the tunnel... error: error upgrading connection: error dialing backend: dial tcp 10.240.0.4:10250: getsockopt: connection timed out

service+pod listing

❯❯❯ kubectl get services,pods ~/c/azure-k8s (master) NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE azure-vote-back ClusterIP 10.0.125.49 <none> 6379/TCP 16h azure-vote-front LoadBalancer 10.0.185.4 40.71.248.106 80:31211/TCP 16h hubot LoadBalancer 10.0.20.218 40.121.215.233 80:31445/TCP 26m kubernetes ClusterIP 10.0.0.1 <none> 443/TCP 19h mti411-web LoadBalancer 10.0.162.209 52.168.123.30 80:30874/TCP 26m

NAME READY STATUS RESTARTS AGE azure-vote-back-7556ff9578-sjjn5 1/1 Running 0 2h azure-vote-front-5b8878fdcd-9lpzx 1/1 Running 0 16h hubot-74f659b6b8-wctdz 1/1 Running 0 9s mti411-web-6cc87d46c-g255d 1/1 Running 0 26m mti411-web-6cc87d46c-lhjzp 1/1 Running 0 26m

http failures

❯❯❯ curl --connect-timeout 2 -I http://40.121.215.233 ~/c/azure-k8s (master) curl: (28) Connection timed out after 2005 milliseconds ❯❯❯ curl --connect-timeout 2 -I http://52.168.123.30 ~/c/azure-k8s (master) curl: (28) Connection timed out after 2001 milliseconds

-- Daniel J. Pritchett
azure
azure-container-service
kubernetes

2 Answers

4/3/2018

@daniel Did rebooting the agent VM's solve your issue or are you still seeing issues?

-- sauryadas_
Source: StackOverflow

6/8/2018

If you are getting getsockopt: connection timed out while trying to access to your AKS Dashboard, I think deleting tunnelfront pod will help as once you delete the tunnelfront pod, this will trigger creation of new tunnelfront by Master. Its something I have tried and worked for me.

-- Arjun Adhikari - MSFT
Source: StackOverflow