I am deploying prometheus which needs persistent volume(i have also tried with other statefulset), but persistent volume is not created and persistent volume clam shows the flowing error after kubectl describe -n {namespace} {pvc-name}.
Type: Warning
Reason: ProvisioningFailed
From: persistentvolume-controller
Message: (combined from similar events): Failed to provision volume with StorageClass "gp2": error querying for all zones: error listing AWS instances: "UnauthorizedOperation: You are not authorized to perform this operation.\n\tstatus code: 403, request id: d502ce90-8af0-4292-b872-ca04900d41dc"
kubectl get sc
NAME PROVISIONER AGE
gp2 (default) kubernetes.io/aws-ebs 7d17h
kubectl describe sc gp2
Name: gp2
IsDefaultClass: Yes
Annotations: kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"storage.k8s.io/v1","kind":"StorageClass","metadata":{"annotations":{"storageclass.kubernetes.io/is-default-class":"true"},"name":"gp2"},"parameters":{"fsType":"ext4","type":"gp2"},"provisioner":"kubernetes.io/aws-ebs","volumeBindingMode":"WaitForFirstConsumer"}
,storageclass.kubernetes.io/is-default-class=true
Provisioner: kubernetes.io/aws-ebs
Parameters: fsType=ext4,type=gp2
AllowVolumeExpansion: <unset>
MountOptions: <none>
ReclaimPolicy: Delete
VolumeBindingMode: WaitForFirstConsumer
Events: <none>
K8s versions(aws eks):
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.2", GitCommit:"c97fe5036ef3df2967d086711e6c0c405941e14b", GitTreeState:"clean", BuildDate:"2019-10-15T23:41:55Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"14+", GitVersion:"v1.14.8-eks-b7174d", GitCommit:"b7174db5ee0e30c94a0b9899c20ac980c0850fc8", GitTreeState:"clean", BuildDate:"2019-10-18T17:56:01Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"}
helm version
version.BuildInfo{Version:"v3.0.0", GitCommit:"e29ce2a54e96cd02ccfce88bee4f58bb6e2a28b6", GitTreeState:"clean", GoVersion:"go1.13.4"}
I solved this problem by adding AmazonEKSClusterPolicy
and AmazonEKSServicePolicy
to the eks cluster role.
Check your IAM policies, and ensure that you are using the correct access keys. Also check IAM role has permission to provision dynamic storage