kubectl timeout inside kube-addon-manager

11/19/2018

I was debugging a issue from my cluster, seems kubectl commands timeout inside the kube-addon-manager pod, while the equivalent curl command works fine.

bash-4.3# kubectl get node --v 10
I1119 16:35:55.506867      54 round_trippers.go:386] curl -k -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.10.5 (linux/amd64) kubernetes/32ac1c9" http://localhost:8080/api
I1119 16:36:25.507550      54 round_trippers.go:405] GET http://localhost:8080/api  in 30000 milliseconds
I1119 16:36:25.507959      54 round_trippers.go:411] Response Headers:
I1119 16:36:25.508122      54 cached_discovery.go:124] skipped caching discovery info due to Get http://localhost:8080/api: dial tcp: i/o timeout

Equivalent curl command output

bash-4.3# curl -k -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.10.5 (linux/amd64) kubernetes/32ac1c9" http://localhost:8080/api
Note: Unnecessary use of -X or --request, GET is already inferred.
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET /api HTTP/1.1
> Host: localhost:8080
> Accept: application/json, */*
> User-Agent: kubectl/v1.10.5 (linux/amd64) kubernetes/32ac1c9
>
< HTTP/1.1 200 OK
< Content-Type: application/json
< Date: Mon, 19 Nov 2018 16:43:00 GMT
< Content-Length: 134
<
{"kind":"APIVersions","versions":["v1"],"serverAddressByClientCIDRs":[{"clientCIDR":"0.0.0.0/0","serverAddress":"172.16.1.13:6443"}]}
* Connection #0 to host localhost left intact

Also tried to run a docker container with host network mode, kubectl command still timeout.

kube-addon-manager.yaml

apiVersion: v1
kind: Pod
metadata:
  name: kube-addon-manager
  namespace: kube-system
  annotations:
    scheduler.alpha.kubernetes.io/critical-pod: ''
  labels:
    component: kube-addon-manager
spec:
  hostNetwork: true
  containers:
  - name: kube-addon-manager
    image: gcr.io/google-containers/kube-addon-manager:v8.6
    imagePullPolicy: IfNotPresent
    command:
    - /bin/bash
    - -c
    - /opt/kube-addons.sh
    resources:
      requests:
        cpu: 5m
        memory: 50Mi
    volumeMounts:
    - mountPath: /etc/kubernetes/
      name: addons
      readOnly: true
  volumes:
  - name: addons
    hostPath:
      path: /etc/kubernetes/
-- flynn
client-go
docker
kubectl
kubernetes

1 Answer

11/20/2018

Seems like in your config you are trying to talk to port 8080 which is the insecure port in the kube-apiserver.

You can try starting your kube-apiserver with this option:

--insecure-port

The default for the insecure port is 8080. Note that this option might be deprecated in the future.

Also, keep in mind the the kube-addon-manager is part of the legacy add-ons.

-- Rico
Source: StackOverflow