K
Q

Filbeat 7.3.2 serviceaccount with clusterrole issue

5/3/2021

My Kubernetes user is not admin in the cluster. So I just cannot create a cluster role binding for filebeat service account. I am using auto discover in filebeat. Can someone help how can I achieve this without clusterrole.

apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat-config
  namespace: logging
  labels:
    k8s-app: filebeat
    kubernetes.io/cluster-service: "true"
data:
  filebeat.yml: |-
    setup.dashboards.enabled: false
    setup.template.enabled: true
    setup.template.settings:
      index.number_of_shards: 1
    filebeat.modules:
      - module: system
        syslog:
          enabled: true
          #var.paths: ["/var/log/syslog"]
        auth:
          enabled: true
          #var.paths: ["/var/log/authlog"]
    filebeat.autodiscover:
      providers:
        - type: kubernetes
          templates:
            - condition:
                equals:
                  kubernetes.namespace: microsrv-test
              config:
                - type: docker
                  json.keys_under_root: true
                  json.add_error_key: true
                  json.message_key: log
                  containers:
                    ids:
                      - "${data.kubernetes.container.id}"
    processors:
      - drop_event:
          when.or:
              - and:
                  - regexp:
                      message: '^\d+\.\d+\.\d+\.\d+ '
                  - equals:
                      fileset.name: error
              - and:
                  - not:
                      regexp:
                          message: '^\d+\.\d+\.\d+\.\d+ '
                  - equals:
                      fileset.name: access
      - add_cloud_metadata:
      - add_kubernetes_metadata:
      - add_docker_metadata:
    output.elasticsearch:
      hosts: ["elasticsearch:9200"]
---

apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: filebeat
  namespace: logging
  labels:
    k8s-app: filebeat
    kubernetes.io/cluster-service: "true"
spec:
  template:
    metadata:
      labels:
        k8s-app: filebeat
        kubernetes.io/cluster-service: "true"
    spec:
      serviceAccountName: filebeat
      terminationGracePeriodSeconds: 30
      containers:
      - name: filebeat
        image: docker.elastic.co/beats/filebeat:7.3.2
        imagePullPolicy: Always
        args: [
          "-c", "/etc/filebeat.yml",
          "-e",
        ]
        env:
        - name: ELASTICSEARCH_HOST
          value: elasticsearch
        - name: ELASTICSEARCH_PORT
          value: "9200"
        - name: KIBANA_HOST
          value: kibana
        - name: KIBANA_PORT
          value: "5601"
        securityContext:
          runAsUser: 0
        volumeMounts:
        - name: config
          mountPath: /etc/filebeat.yml
          readOnly: true
          subPath: filebeat.yml
        - name: data
          mountPath: /usr/share/filebeat/data
        - name: varlog
          mountPath: /var/log
          readOnly: true
        - name: varlibdockercontainers
          mountPath: /var/lib/docker/containers
          readOnly: true
        - name: dockersock
          mountPath: /var/run/docker.sock
      volumes:
      - name: config
        configMap:
          defaultMode: 0600
          name: filebeat-config
      - name: varlog
        hostPath:
          path: /var/log
      - name: varlibdockercontainers
        hostPath:
          path: /var/lib/docker/containers
      - name: dockersock
        hostPath:
          path: /var/run/docker.sock
      - name: data
        emptyDir: {}

---

Cluster Roles and role bindings

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: filebeat
subjects:
- kind: ServiceAccount
  name: filebeat
  namespace: logging
roleRef:
  kind: ClusterRole
  name: filebeat
  apiGroup: rbac.authorization.k8s.io

---

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: filebeat
  labels:
    k8s-app: filebeat
rules:
- apiGroups: [""] # "" indicates the core API group
  resources:
  - namespaces
  - pods
  verbs:
  - get
  - watch
  - list
  
---

apiVersion: v1
kind: ServiceAccount
metadata:
  name: filebeat
  namespace: logging
  labels:
    k8s-app: filebeat

I have tried creating non cluster role and rolebinding as below, 

apiVersion: v1
kind: ServiceAccount
metadata:
  name: filebeat
  namespace: logging
  labels:
    k8s-app: filebeat
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: filebeat
  namespace: logging
  labels:
    k8s-app: filebeat
rules:
  - apiGroups: [""] # "" indicates the core API group
    resources:
      - pods
    verbs:
      - get
      - watch
      - list
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: filebeat
  namespace: logging
subjects:
  - kind: ServiceAccount
    name: filebeat
    namespace: logging
roleRef:
  kind: Role
  name: filebeat
  apiGroup: rbac.authorization.k8s.io

but I am getting error.

Performing a resource sync err kubernetes api: Failure 403 pods is forbidden: User "system:serviceaccount:xxxxx:filebeat" cannot list resource "pods" in API group "" at the cluster scope for *v1.PodList|

-- riteshmaurya
elastic-stack
elasticsearch
filebeat
kubernetes
rbac

Similar Questions

How check all my deployments in Kubernetes are finished
Is it necessary to run Django behind a reverse proxy when deployed on Kubernetes
Trying to configure Nginx for Rancher 2.X - Migrating from Rancher 1.x
Comparison of ways to connect to Google Cloud SQL database from GKE
Is there a way/config where I could change default revisionHistoryLimit from 10 to another limit?

1 Answer

5/6/2021

Unfortunately, it will not work the way you want it and the error you are getting indicates it perfectly:

Performing a resource sync err kubernetes api: Failure 403 pods is forbidden: User "system:serviceaccount:xxxxx:filebeat" cannot list resource "pods" in API group "" at the cluster scope for *v1.PodList|

Notice the most important part which is: at the cluster scope. You could also check whether an action is allowed by executing the kubectl auth can-i command. More about that can be found in the Authorization Overview.

This brings us to the differences between Role and ClusterRole:

An RBAC Role or ClusterRole contains rules that represent a set of permissions. Permissions are purely additive (there are no "deny" rules).

A Role always sets permissions within a particular namespace; when you create a Role, you have to specify the namespace it belongs in.

ClusterRole, by contrast, is a non-namespaced resource. The resources have different names (Role and ClusterRole) because a Kubernetes object always has to be either namespaced or not namespaced; it can't be both.

ClusterRoles have several uses. You can use a ClusterRole to:

  • define permissions on namespaced resources and be granted within individual namespace(s)

  • define permissions on namespaced resources and be granted across all namespaces

  • define permissions on cluster-scoped resources

If you want to define a role within a namespace, use a Role; if you want to define a role cluster-wide, use a ClusterRole.

And between RoleBinding and ClusterRoleBinding:

A role binding grants the permissions defined in a role to a user or set of users. It holds a list of subjects (users, groups, or service accounts), and a reference to the role being granted. A RoleBinding grants permissions within a specific namespace whereas a ClusterRoleBinding grants that access cluster-wide.

A RoleBinding may reference any Role in the same namespace. Alternatively, a RoleBinding can reference a ClusterRole and bind that ClusterRole to the namespace of the RoleBinding. If you want to bind a ClusterRole to all the namespaces in your cluster, you use a ClusterRoleBinding.

So it is impossible to get the cluster scope permissions by using Role and RoleBinding.

You will most likely have to ask your Admin to help you solve this issue.

-- Wytrzymały Wiktor
Source: StackOverflow