Waiting for pods: apiserver get stuck

3/14/2019

I am trying to implement auditing policy My yaml

~/.minikube/addons$ cat audit-policy.yaml 
# Log all requests at the Metadata level.
apiVersion: audit.k8s.io/v1beta1
kind: Policy
rules:
- level: Metadata

Pods got stuck

minikube start  --extra-config=apiserver.Authorization.Mode=RBAC --extra-config=apiserver.Audit.LogOptions.Path=/var/logs/audit.log   --extra-config=apiserver.Audit.PolicyFile=/etc/kubernetes/addons/audit-policy.yaml
  minikube v0.35.0 on linux (amd64)
  Tip: Use 'minikube start -p <name>' to create a new cluster, or 'minikube delete' to delete this one.
  Restarting existing virtualbox VM for "minikube" ...
⌛  Waiting for SSH access ...
  "minikube" IP address is 192.168.99.101
  Configuring Docker as the container runtime ...
✨  Preparing Kubernetes environment ...
    ▪ apiserver.Authorization.Mode=RBAC
    ▪ apiserver.Audit.LogOptions.Path=/var/logs/audit.log
    ▪ apiserver.Audit.PolicyFile=/etc/kubernetes/addons/audit-policy.yaml
  Pulling images required by Kubernetes v1.13.4 ...
  Relaunching Kubernetes v1.13.4 using kubeadm ... 
⌛  Waiting for pods: apiserver

Why?

I can do this

minkub start

Then I go for minikube ssh

$ sudo bash
$ cd /var/logs
bash: cd: /var/logs: No such file or directory
ls
cache  empty  lib  lock  log  run  spool  tmp

How to apply extra-config?

-- Richard Rublev
kubernetes

1 Answer

4/9/2019

I don't have good news. Although you made some mistakes with the /var/logs it does not matter in this case, as there seems to be no way of implement auditing policy in Minikube (I mean there is, few ways at least but they all seem to fail).

You can try couple of ways presented in GitHub issues and other links I will provide, but I tried probably all of them and they do not work with current Minikube version. You might try to make this work with earlier versions maybe, as it seems like at some point it was possible with the way you have provided in your question, but as for now in the updated version it is not. Anyway I have spend some time on trying the ways from the links and couple of my own ideas but no success, maybe you will be able to find the missing piece.

You can find more information in this documents:

Audit Logfile Not Created

Service Accounts and Auditing in Kubernetes

fails with -extra-config=apiserver.authorization-mode=RBAC and audit logging: timed out waiting for kube-proxy

How do I enable an audit log on minikube?

Enable Advanced Auditing Webhook Backend Configuration

-- aurelius
Source: StackOverflow