I wanted to setup some monitoring and tried this: https://github.com/giantswarm/kubernetes-prometheus
It sets up all the Pods and services but I get an error message:
Error from server (Forbidden): error when creating
"https://raw.githubusercontent.com/giantswarm/kubernetes-prometheus/master/manifests-all.yaml":
clusterroles.rbac.authorization.k8s.io "prometheus" is forbidden:
attempt to grant extra privileges: [PolicyRule{APIGroups:[""],
Resources:["nodes"], Verbs:["get"]} PolicyRule{APIGroups:[""],
Resources:["nodes"], Verbs:["list"]} PolicyRule{APIGroups:[""],
Resources:["nodes"], Verbs:["watch"]} PolicyRule{APIGroups:[""],
Resources:["nodes/proxy"], Verbs:["get"]} PolicyRule{APIGroups:[""],
Resources:["nodes/proxy"], Verbs:["list"]} PolicyRule{APIGroups:[""],
Resources:["nodes/proxy"], Verbs:["watch"]} PolicyRule{APIGroups:[""],
Resources:["services"], Verbs:["get"]} PolicyRule{APIGroups:[""],
Resources:["services"], Verbs:["list"]} PolicyRule{APIGroups:[""],
Resources:["services"], Verbs:["watch"]} PolicyRule{APIGroups:[""],
Resources:["endpoints"], Verbs:["get"]} PolicyRule{APIGroups:[""],
Resources:["endpoints"], Verbs:["list"]} PolicyRule{APIGroups:[""],
Resources:["endpoints"], Verbs:["watch"]} PolicyRule{APIGroups:[""],
Resources:["pods"], Verbs:["get"]} PolicyRule{APIGroups:[""],
Resources:["pods"], Verbs:["list"]} PolicyRule{APIGroups:[""],
Resources:["pods"], Verbs:["watch"]} PolicyRule{APIGroups:[""],
Resources:["configmaps"], Verbs:["get"]}
PolicyRule{NonResourceURLs:["/metrics"], Verbs:["get"]}]
user=&{xxx [system:authenticated]
map[user-assertion.cloud.google.com:[xxx]]}
ownerrules=[PolicyRule{APIGroups:["authorization.k8s.io"],
Resources:["selfsubjectaccessreviews" "selfsubjectrulesreviews"],
Verbs:["create"]} PolicyRule{NonResourceURLs:["/api" "/api/*" "/apis"
"/apis/*" "/healthz" "/swagger-2.0.0.pb-v1" "/swagger.json"
"/swaggerapi" "/swaggerapi/*" "/version"], Verbs:["get"]}
PolicyRule{NonResourceURLs:["/openapi"], Verbs:["get"]}
PolicyRule{NonResourceURLs:["/openapi/*"], Verbs:["get"]}
PolicyRule{NonResourceURLs:["/version/"], Verbs:["get"]}]
ruleResolutionErrors=[]
I would appreciate any help. I suppose I need to grant extra rights, but I dont know how. Cluster version is: 1.10.9-gke.5
Thank you
EDIT: With the changes @Rico proposed I am getting an slightly different error message:
Error from server (Forbidden): error when creating "manifests-all.yaml": clusterroles.rbac.authorization.k8s.io "prometheus" is forbidden: attempt to grant extra privileges:
[PolicyRule{APIGroups:["*"], Resources:["*"], Verbs:["*"]} PolicyRule{NonResourceURLs:["/metrics"], Verbs:["get"]}] user=&{xxx [system:authenticated] map[user-assertion.cloud.google.com:[xxx]]} ownerrules=
[PolicyRule{APIGroups:["authorization.k8s.io"], Resources:["selfsubjectaccessreviews" "selfsubjectrulesreviews"], Verbs:["create"]}
PolicyRule{NonResourceURLs:["/api" "/api/*" "/apis" "/apis/*" "/healthz" "/swagger-2.0.0.pb-v1" "/swagger.json" "/swaggerapi" "/swaggerapi/*" "/version"], Verbs:["get"]} PolicyRule{NonResourceURLs:["/openapi"], Verbs:["get"]}
PolicyRule{NonResourceURLs:["/openapi/*"], Verbs:["get"]}
PolicyRule{NonResourceURLs:["/version/"], Verbs:["get"]}] ruleResolutionErrors=[]
You need to set up your prometheus
ClusterRole
to be more permissive. You can start with full permissions to see if it works initially and then work your way down:
cat <<EOF
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: prometheus
rules:
- apiGroups: ["*"]
resources: ["*"]
verbs: ["*"]
EOF | kubectl apply -f -