"Failed to scrape node" `
err="GET \"https://10.128.0.17:10250/stats/summary?
only_cpu_and_memory=true\": bad status code \"403 Forbidden\""
node="gke-zipydev-cluster-zipy-pool-b4bfa53a-t575"
I1215 10:33:03.405180 1 server.go:188] "Failed probe"
probe="metric-storage-ready" err="not metrics to serve"
E1215 10:33:10.513042 1 scraper.go:139] "Failed to scrape
node" err="GET \"https://10.128.0.16:10250/stats/summary?
only_cpu_and_memory=true\": bad status code \"403 Forbidden\""
node="gke-zipydev-cluster-zipy-pool-b4bfa53a-sg4t"please help if anyone faced same issue.
The privileges for the metrics server are not correctly added as the “403“ error is because access to the requested resource is forbidden.
The Metrics Server requires the “CAP_NET_BIND_SERVICE” capability in order to bind to a privileged ports as non-root as this applies even if you use the --secure-port flag to change the port that Metrics Server binds to to a non-privileged port. Refer Security context for information.
As described in the Github link, Granting metrics-server necessary permissions to access(query/read) nodes/stat API resource is the workaround to solve this issue. You can grant metrics-server necessary permissions by using the below configuration file.
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: system:metrics-server
rules:
- apiGroups:
- ""
resources:
- nodes/stats
- nodes
verbs:
- get
- list
NOTE: Check your metrics-server has a recent version if you installed it manually. In order to update your metrics-server deployment, you can refer to the Github link and select the version which suits you.
Refer stackpost for more information about 403 forbidden errors.