We had an issue where part of the dashboard would load forever and eventually time out talking to port 9090.

No indications of issues in the docker logs for the dashboard or the kubelet logs on the masters or workers.

I had to force the kubectl delete of the dashboard with "--cascade=false" because the delete would also hang.

This is leaving us with a stale replica set which usually expires on it's own but the replica set seems to think that it's still in use:

NAMESPACE     NAME                                      DESIRED   CURRENT   READY     AGE
kube-system   rs/dnsmasq-1410718522                     1         1         1         20d
kube-system   rs/dnsmasq-autoscaler-1155841093          1         1         1         20d
kube-system   rs/grafana-1857700562                     1         1         1         17d
kube-system   rs/heapster-3338981674                    1         1         1         17d
kube-system   rs/influxdb-3065341217                    1         1         1         17d
kube-system   rs/kubedns-3830354952                     1         1         1         20d
kube-system   rs/kubedns-autoscaler-54374881            1         1         1         20d
kube-system   rs/kubernetes-dashboard-head-3335363610   1         1         1         1h

The dashboard never deploys:

NAMESPACE     NAME                          DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
kube-system   deploy/dnsmasq                1         1         1            1           20d
kube-system   deploy/dnsmasq-autoscaler     1         1         1            1           20d
kube-system   deploy/grafana                1         1         1            1           17d
kube-system   deploy/heapster               1         1         1            1           17d
kube-system   deploy/influxdb               1         1         1            1           17d
kube-system   deploy/kubedns                1         1         1            1           20d
kube-system   deploy/kubedns-autoscaler     1         1         1            1           20d
kube-system   deploy/kubernetes-dashboard   0         0         0            0           1d

After almost a day the dashboard says its up with no errors but clearly it's not:

kubectl logs kubernetes-dashboard-head-3335363610-v37xz --namespace=kube-system
Using HTTP port: 8443
Creating API server client for https://10.233.0.1:443
Successful initial request to the apiserver, version: v1.5.3+coreos.0
Creating in-cluster Heapster client
Using service account token for csrf signing

From the browser:

 {
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {},
  "status": "Failure",
  "message": "no endpoints available for service \"kubernetes-dashboard\"",
  "reason": "ServiceUnavailable",
  "code": 503
}

None of the masters or workers have a dashboard container in docker.

We were able to get nginx to underplay with "--cascade=true" and redeploy but the kubernetes-dashboard and cockpit both hang the same way.