K
Q

nginx setup in kubernetes with RBAC enabled

4/26/2017

From Kubernetes v1.6, RBAC authorize feature is enabled by default. This implies that the deployments/configurations I had for v1.5, are no longer working.

One of the key components to which I needed to grant access is to nginx, otherwise a message like to following can be seen on the logs

F0425 15:08:07.246596       1 main.go:116] no service with name kube-system/default-http-backend found: the server does not allow access to the requested resource (get services default-http-backend)
-- aitorhh
kubernetes
nginx
rbac

Similar Questions

Deployment - node selector per replica
Kubernetes cluster role error: at least one verb must be specified
How to mount volume into the source code of the app?
Pod cannot get IP in PodCIDR, get the docker ip

1 Answer

4/26/2017

UPDATED: kubernetes/nginx has the documentation updated here and for RBAC details, here

OLD:

In order to support RBAC, we need two things:

  • define the servciceAccount/ClusterRole/ClusterRoleBindings
  • set a serviceAccount for the nginx deployment

Here are the files I use to set it up:

nginx-roles.yml

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nginx
  namespace: kube-system
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: nginx-role
rules:
- apiGroups: [""]
  resources: ["secrets", "configmaps", "services", "endpoints"]
  verbs:
    - get
    - watch
    - list
    - proxy
    - use
    - redirect
- apiGroups: [""]
  resources: ["events"]
  verbs:
    - redirect
    - patch
    - post
- apiGroups:
    - "extensions"
  resources:
    - "ingresses"
  verbs:
    - get
    - watch
    - list
    - proxy
    - use
    - redirect
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: nginx-role
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: nginx-role
subjects:
- kind: ServiceAccount
  name: nginx
  namespace: kube-system

nginx-ingress-controller.yml with nodeSelector: kubecluster-amd-1 and default-http-backend used

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  labels:
    k8s-app: nginx-ingress-controller
  namespace: kube-system
spec:
  replicas: 1
  template:
    metadata:
      labels:
        k8s-app: nginx-ingress-controller
    spec:
      serviceAccount: nginx
      hostNetwork: true
      nodeSelector:
          kubernetes.io/hostname: kubecluster-amd-1
      terminationGracePeriodSeconds: 60
      containers:
      - image: gcr.io/google_containers/nginx-ingress-controller:0.9.0-beta.4
        name: nginx-ingress-controller
        readinessProbe:
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
        livenessProbe:
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 20
          timeoutSeconds: 1
        ports:
        - containerPort: 80
          hostPort: 80
        - containerPort: 443
          hostPort: 443
        - containerPort: 5683
          hostPort: 5683
          protocol: UDP
        - containerPort: 5684
          hostPort: 5684
          protocol: UDP
        - containerPort: 53
          hostPort: 53
          protocol: UDP
        env:
          - name: POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: POD_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
        args:
        - /nginx-ingress-controller
        - --default-backend-service=$(POD_NAMESPACE)/default-http-backend
-- aitorhh
Source: StackOverflow