Has anyone run into this issue.
Autoscaling seems to have stopped working and I get the following error from any HPA I create
Conditions: Type Status Reason Message ---- ------ ------ ------- AbleToScale True SucceededGetScale the HPA controller was able to get the target's current scale ScalingActive False FailedGetResourceMetric the HPA was unable to compute the replica count: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) ScalingLimited True TooFewReplicas the desired replica count is increasing faster than the maximum scale rate Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning FailedGetResourceMetric 53s (x17244 over 5d) horizontal-pod-autoscaler unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io)
I checked the metric server logs and I get the following
0529 14:54:33.749323 1 heapster.go:71] /metrics-server --source=kubernetes.summary_api:'' I0529 14:54:33.751589 1 heapster.go:72] Metrics Server version v0.2.1 I0529 14:54:33.752620 1 configs.go:61] Using Kubernetes client with master "https://10.96.0.1:443" and version I0529 14:54:33.752642 1 configs.go:62] Using kubelet port 10255 I0529 14:54:33.754452 1 heapster.go:128] Starting with Metric Sink I0529 14:54:34.036240 1 serving.go:308] Generated self-signed cert (apiserver.local.config/certificates/apiserver.crt, apiserver.local.config/certificates/apiserver.key) I0529 14:54:34.474364 1 heapster.go:101] Starting Heapster API server... [restful] 2018/05/29 14:54:34 log.go:33: [restful/swagger] listing is available at https:///swaggerapi [restful] 2018/05/29 14:54:34 log.go:33: [restful/swagger] https:///swaggerui/ is mapped to folder /swagger-ui/ I0529 14:54:34.476366 1 serve.go:85] Serving securely on 0.0.0.0:443 I0529 15:07:44.217338 1 logs.go:41] http: TLS handshake error from 10.36.0.71:40660: tls: first record does not look like a TLS handshake I0529 15:09:38.647544 1 logs.go:41] http: TLS handshake error from 10.36.0.71:41350: tls: first record does not look like a TLS handshake
Not sure what is causing this as I did work up until last week.
kube version 1.10.3