Applying API Server App ID to k8s cluster spec

11/7/2018

Team,

I already have a cluster running and I need to update the OIDC value. is there a way I can do it without having to recreate the cluster?

ex: below is my cluster info and I need to update the oidcClientID: spn:

How can I do this as I have 5 masters running?

kubeAPIServer:
    storageBackend: etcd3
    oidcClientID: spn:45645hhh-f641-498d-b11a-1321231231
    oidcUsernameClaim: upn
    oidcUsernamePrefix: "oidc:"
    oidcGroupsClaim: groups
    oidcGroupsPrefix: "oidc:"
-- AhmFM
azure-kubernetes
kubectl
kubernetes

1 Answer

11/8/2018

You update your kube-apiserver on your masters one by one (update/restart). If your cluster is setup correctly, when you get to the active kube-apiserver it should automatically failover to another kube-apiserver master in standby.

You can add the oidc options in the /etc/kubernetes/manifests/kube-apiserver.yaml pod manifest file.

apiVersion: v1
kind: Pod
metadata:
  annotations:
    scheduler.alpha.kubernetes.io/critical-pod: ""
  creationTimestamp: null
  labels:
    component: kube-apiserver
    tier: control-plane
  name: kube-apiserver
  namespace: kube-system
spec:
  containers:
  - command:
    - kube-apiserver
    - --authorization-mode=Node,RBAC
    - --advertise-address=172.x.x.x
    - --allow-privileged=true
    - --client-ca-file=/etc/kubernetes/pki/ca.crt
    - --oidc-client-id=...
    - --oidc-username-claim=...
    - --oidc-username-prefix=...
    - --oidc-groups-claim=...
    - --oidc-groups-prefix=...
...

Then you can restart your kube-apiserver container, if you are using docker:

$ sudo docker restart <container-id-for-kube-apiserver>

Or if you'd like to restart all the components on the master:

$ sudo systemctl restart docker

Watch for logs on the kube-apiserver container

$ sudo docker logs -f <container-id-for-kube-apiserver>

Make sure you never have less running nodes than your quorum which should be 3 for your 5 master cluster, to be safe. If for some reason your etcd cluster falls out of quorum you will have to recover by recreating the etcd cluster and restoring from a backup.

-- Rico
Source: StackOverflow