Gitlab CI Pipeline: Cannot create pods in the namespace

11/20/2018

I have a kubernetes cluster (rancherOS & RKE) that has a running gitlab runner pod. Connection to my GitLab instance works fine.

If I activate the pipeline, it directly fails with this error:

Running with gitlab-runner 11.4.2 (cf91d5e1)
  on Kubernetes Runner e5e25776
Using Kubernetes namespace: gitlab-managed-apps
Using Kubernetes executor with image ubuntu:latest ...
ERROR: Job failed (system failure): pods is forbidden: User "system:serviceaccount:gitlab-managed-apps:default" cannot create pods in the namespace "gitlab-managed-apps"

This here is my gitlab-runner deployment yaml:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: gitlab-runner
  namespace: gitlab-managed-apps
spec:
  replicas: 1
  selector:
    matchLabels:
      name: gitlab-runner
  template:
    metadata:
      labels:
        name: gitlab-runner
    spec:
      containers:
      - args:
        - run
        image: gitlab/gitlab-runner:latest
        imagePullPolicy: Always
        name: gitlab-runner
        securityContext:
          privileged: true
        volumeMounts:
        - mountPath: /etc/gitlab-runner
          name: config
        - mountPath: /etc/ssl/certs
          name: cacerts
          readOnly: true
      restartPolicy: Always
      volumes:
      - configMap:
          name: gitlab-runner
        name: config
      - hostPath:
          path: /usr/share/ca-certificates/mozilla
        name: cacerts
      hostNetwork: true

I tried to add a security context with the parameter "privileged: true" but that does not help..

Has anyone an idea on how to grant the gitlab-runner deployment the right permission to create other pods in the namespace "gitlab-managed-apps"?

Thanks a lot :)

-- user7436888
continuous-integration
gitlab
kubernetes
pipeline

2 Answers

3/9/2020

Your service account lacks permissions. A similar issue has happened to me during secrets creation.

You can grant access without having to fulfill any extra files, just with the help of kubectl. You should create a role binding, namely, grant a role to the default service account in a namespace. A full description is provided here.

In your case the command will look like this:

kubectl create rolebinding default-view --clusterrole=edit --serviceaccount=gitlab-managed-apps:default --namespace=gitlab-managed-apps
-- Mariia Abramyk
Source: StackOverflow

11/20/2018

In your deployment yaml you didn't add spec.template.spec.serviceAccountName, which means it uses the default serviceaccount named default in your deployment namespace named gitlab-managed-apps. And it has no rbac rule to create pods according to the error you specified.

For details, see https://kubernetes.io/docs/reference/access-authn-authz/rbac/.

There are more than one way to resolve this. Here is one:

First create a rbac rule and bind it to a serviceaccount. Bellow is an example:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: gitlab
  namespace: gitlab-managed-apps
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: gitlab-managed-apps
  name: gitlab
rules:
- apiGroups: [""] # "" indicates the core API group
  resources: ["pods"]
  verbs: ["create"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: gitlab
  namespace: gitlab-managed-apps
subjects:
- kind: ServiceAccount
  name: gitlab # Name is case sensitive
  apiGroup: ""
roleRef:
  kind: Role #this must be Role or ClusterRole
  name: gitlab # this must match the name of the Role or ClusterRole you wish to bind to
  apiGroup: rbac.authorization.k8s.io

Then edit your deployment yaml to add this serviceaccount:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: gitlab-runner
  namespace: gitlab-managed-apps
spec:
  replicas: 1
  selector:
    matchLabels:
      name: gitlab-runner
  template:
    metadata:
      labels:
        name: gitlab-runner
    spec:
      serviceAccountName: gitlab
      containers:
      - args:
        - run
        image: gitlab/gitlab-runner:latest
        imagePullPolicy: Always
        name: gitlab-runner
        securityContext:
          privileged: true
        volumeMounts:
        - mountPath: /etc/gitlab-runner
          name: config
        - mountPath: /etc/ssl/certs
          name: cacerts
          readOnly: true
      restartPolicy: Always
      volumes:
      - configMap:
          name: gitlab-runner
        name: config
      - hostPath:
          path: /usr/share/ca-certificates/mozilla
        name: cacerts
      hostNetwork: true

Then deploy your gitlab instances and other things those you need.

-- Shudipta Sharma
Source: StackOverflow