I'm having a problem with kubectl proxy on a fresh install.

When I browse to http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/ I get a 503 response. It seems that the proxy isn't able to reach the kubernetes API even though other commands can.

Kubernetes runs in DC/OS with package 1.3.1-1.10.8. Both kubectl and Kubernetes are version 1.10.8. There's a load balancer configured in dc/os to expose the API.

The LB definition is from the kubernetes on dcos help page. I added "HAPROXY_0_VHOST": "k8s-proxy.dcos.<domain>.com" to the labels.

$ kubectl cluster-info
Kubernetes master is running at https://k8s-proxy.dcos.<domain>.com
KubeDNS is running at https://k8s-proxy.dcos.<domain>.com/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

I ran kubectl proxy in verbose output mode to see what call it was trying to make. It got a 503 response.

$ kubectl proxy --insecure-skip-tls-verify=true --alsologtostderr=true -v=99
I0327 12:26:45.461259   19980 loader.go:357] Config loaded from file U:\/.kube/config
Starting to serve on 127.0.0.1:8001
I0327 12:26:56.200819   19980 proxy_server.go:98] /api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/ matched ^.*
I0327 12:26:56.200819   19980 proxy_server.go:98] localhost matched ^localhost$
I0327 12:26:56.200819   19980 proxy_server.go:138] Filter accepting GET /api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/ localhost
I0327 12:26:56.200819   19980 upgradeaware.go:237] Request was not an upgrade
I0327 12:26:56.200819   19980 round_trippers.go:387] curl -k -v -XGET  -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8" -H "Cache-Control: max-age=0" -H "User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36" -H "Authorization: Bearer <my_token>" -H "X-Forwarded-For: 127.0.0.1" -H "Accept-Language: en-US,en;q=0.9" -H "Dnt: 1" -H "Accept-Encoding: gzip, deflate, br" -H "Upgrade-Insecure-Requests: 1" https://k8s-proxy.dcos.<domain>.com/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/
I0327 12:26:56.313141   19980 round_trippers.go:406] GET https://k8s-proxy.dcos.<domain>.com/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/ 503 Service Unavailable in 112 milliseconds
I0327 12:26:56.313141   19980 round_trippers.go:412] Response Headers:
I0327 12:26:56.313141   19980 round_trippers.go:415]     Cache-Control: no-cache
I0327 12:26:56.313141   19980 round_trippers.go:415]     Content-Type: text/html

In the same shell I tried running the curl that the proxy runs. It got a 200 instead of a 503.

$ curl -k -v -XGET  -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8" -H "Cache-Control: max-age=0" -H "User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36" -H "Authorization: Bearer <my_token>" -H "X-Forwarded-For: 127.0.0.1" -H "Accept-Language: en-US,en;q=0.9" -H "Dnt: 1" -H "Accept-Encoding: gzip, deflate, br" -H "Upgrade-Insecure-Requests: 1" https://k8s-proxy.dcos.<domain>.com/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/

[...]
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Cache-Control: no-store
< Content-Encoding: gzip
< Content-Type: text/html; charset=utf-8
< Date: Wed, 27 Mar 2019 19:30:24 GMT
< Last-Modified: Fri, 24 Aug 2018 05:39:29 GMT
< Content-Length: 529
[...]

I expected to be able to access my cluster but valid requests are returning a 503. Other kubectl commands work fine. This is not a problem that's specific to the dashboard.