I've been trying for the last 3 days to setup cert-manager on a K8S cluster (v1.19.8) in an OpenStack environment with 1 master and 2 nodes. It worked before (like 1 month ago), but since I re-created the cluster, pod ACME challenges cannot be created due to this error:
Status:
Presented: false
Processing: true
Reason: pods "cm-acme-http-solver-" is forbidden: PodSecurityPolicy: unable to admit pod: []
State: pending
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Started 8m25s cert-manager Challenge scheduled for processing
Warning PresentError 3m18s (x7 over 8m23s) cert-manager Error presenting challenge: pods "cm-acme-http-solver-" is forbidden: PodSecurityPolicy: unable to admit pod: []
I've tried different versions of the ingress-nginx, different versions of cert-manager, different versions of k8s, but to no avail. I'm getting crazy..., please help. Many thanks :)
kubectl create namespace ingress-nginx && \
helm install ingress-nginx ingress-nginx/ingress-nginx -n ingress-nginx && \
kubectl create namespace cert-manager && \
helm install cert-manager jetstack/cert-manager \
--namespace cert-manager \
--version v1.1.0 \
--set installCRDs=true
Issuer
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: letsencrypt-prod
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: email@example.com
preferredChain: "ISRG Root X1"
privateKeySecretRef:
name: letsencrypt-prod
solvers:
- http01:
ingress:
class: nginx
Ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: main-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/from-to-www-redirect: "true"
cert-manager.io/issuer: "letsencrypt-prod"
spec:
tls:
- hosts:
- host.com
secretName: the-secret-name
rules:
- host: host.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: api-nginx
port:
number: 80
A year late, but adding another solution in case it helps others finding this. I had the same issue of the challenge pod being blocked by PSP, but really didn't want to have to recreate/reconfigure my cluster, so I eventually solved the issue by adding this to the helm chart values.yaml: https://github.com/cert-manager/cert-manager/blob/master/deploy/charts/cert-manager/values.yaml
global:
podSecurityPolicy:
enabled: true
useAppArmor: false
In my case, this is part of a Gitlab deployment so I added it under the certmanager key, as follows:
certmanager:
install: true
global:
podSecurityPolicy:
enabled: true
useAppArmor: false
(tags for search: gitlab helm chart certmanager PodSecurityPolicy "unable to admit pod" blocked)
After some debugging and much help from the hosting provider, we found the problem and the solution.
We were using the latest (from master) version of Magnum/OpenStack, which got an update that installed by default a PodSecurityPolicy controller. That prevented ACME pods to be created by cert-manager.
Recreating the cluster without a policy controller solved the issue:
openstack coe cluster create \
--cluster-template v1.kube1.20.4 \
--labels \
admission_control_list="NodeRestriction,NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,PersistentVolumeClaimResize,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,RuntimeClass" \
--merge-labels
...