Kubernetes Dashboard Unauthorized with Keycloack/Dex

8/23/2019

I'm having trouble getting kubernetes to allow access to the the dashboard. I am using dex/keycloack and am able to login successfully but then I receive an Unauthorized message from the UI. The browser console shows:

Couldn't get available api versions from server: Unauthorized\n","status":500

The kubernetes dashboard logs shows multiple errors, I have metrics-server installed but not heapster so I'm not as worried about the second one:

2019/08/23 12:15:11 Getting application global configuration
2019/08/23 12:15:11 Application configuration {"serverTime":1566562511915}
2019/08/23 12:15:12 [2019-08-23T12:15:12Z] Incoming HTTP/1.1 GET /api/v1/settings/global request from 192.168.2.12:38700: {}
2019/08/23 12:15:12 Cannot find settings config map: Unauthorized
2019/08/23 12:15:12 Cannot restore settings config map: Unauthorized
2019/08/23 12:15:12 [2019-08-23T12:15:12Z] Outcoming response to 192.168.2.12:38700 with 200 status code
2019/08/23 12:15:12 [2019-08-23T12:15:12Z] Incoming HTTP/1.1 GET /api/v1/systembanner request from 192.168.2.12:38700: {}
2019/08/23 12:15:12 [2019-08-23T12:15:12Z] Outcoming response to 192.168.2.12:38700 with 200 status code
2019/08/23 12:15:12 [2019-08-23T12:15:12Z] Incoming HTTP/1.1 GET /api/v1/login/status request from 192.168.2.12:38700: {}
2019/08/23 12:15:12 [2019-08-23T12:15:12Z] Outcoming response to 192.168.2.12:38700 with 200 status code
2019/08/23 12:15:12 [2019-08-23T12:15:12Z] Incoming HTTP/1.1 GET /api/v1/rbac/status request from 192.168.2.12:38700: {}
2019/08/23 12:15:12 Couldn't get available api versions from server: Unauthorized
2019/08/23 12:15:12 [2019-08-23T12:15:12Z] Outcoming response to 192.168.2.12:38700 with 500 status code
2019/08/23 12:15:12 [2019-08-23T12:15:12Z] Incoming HTTP/1.1 GET /api/v1/overview/default?filterBy=&itemsPerPage=10&name=&page=1&sortBy=d,creationTimestamp request from 192.168.2.12:38700: {}
2019/08/23 12:15:12 Getting config category
2019/08/23 12:15:12 Non-critical error occurred during resource retrieval: Unauthorized

2019/08/23 11:47:25 Metric client health check failed: the server could not find the requested resource (get services heapster). Retrying in 30 seconds.

I also see this in the keycloack logs which seems like it might be the issue but I haven't been able to determine why it would happen.

1.5665671075671058e+09  warn    unable to parse the access token, using id token only   {"error": "malformed JWS, only 1 segments"}

I'm using the following service account, role and role-binding but I've also tried binding the k8s-app: kubernetes-dashboard account to the cluster-admin role with no luck.

# ------------------- Dashboard Service Account ------------------- #

apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kube-system

---
# ------------------- Dashboard Role & Role Binding ------------------- #

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: kubernetes-dashboard-minimal
  namespace: kube-system
rules:
  # Allow Dashboard to create 'kubernetes-dashboard-key-holder' secret.
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["create"]
  # Allow Dashboard to create 'kubernetes-dashboard-settings' config map.
- apiGroups: [""]
  resources: ["configmaps"]
  verbs: ["create"]
  # Allow Dashboard to get, update and delete Dashboard exclusive secrets.
- apiGroups: [""]
  resources: ["secrets"]
  resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs"]
  verbs: ["get", "update", "delete"]
  # Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map.
- apiGroups: [""]
  resources: ["configmaps"]
  resourceNames: ["kubernetes-dashboard-settings"]
  verbs: ["get", "update"]
  # Allow Dashboard to get metrics from heapster.
- apiGroups: [""]
  resources: ["services"]
  resourceNames: ["heapster"]
  verbs: ["proxy"]
- apiGroups: [""]
  resources: ["services/proxy"]
  resourceNames: ["heapster", "http:heapster:", "https:heapster:"]
  verbs: ["get"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: kubernetes-dashboard-minimal
  namespace: kube-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: kubernetes-dashboard-minimal
subjects:
- kind: ServiceAccount
  name: kubernetes-dashboard
  namespace: kube-system

---
-- tweeks200
keycloak
kubernetes
openid-connect

1 Answer

8/23/2019

This turns out to be a case of me not reading the documentation. I missed the section where I had to add these to the kube-api server.

--oidc-issuer-url=https://dex.k8s.example.com/dex: External Dex endpoint
--oidc-client-id=loginapp: ID for our Login Application
--oidc-username-claim=name: Map to nameAttr Dex configuration. This will be used by Kubernetes RBAC to authorize users based on their name.
--oidc-groups-claim=groups: This will be used by Kubernetes RBAC to authorize users based on their groups.
-- tweeks200
Source: StackOverflow