Lets say I have a pod with one volume created from a secret, which comes to life in ~5sec.
Now if I create a bigger amounts of pods and secrets everything getting slower. Which is kind of normal I guess.
Lets say I create 20pod with 500secret. It takes ~150sec to get into running state. I get a ton of warning like:
Warning FailedMount 45s (x1 over 1m) kubelet Unable to attach or mount volumes: unmounted volumes=[abc-secrets-vol], unattached volumes=[abc-secrets-vol default-token-cp4nw nginx-config]: timed out waiting for the conditionBut nothing serious , by the end everything will work fine.
Now the questions:<br>
volume creation OR volume attach (I excluded the secret creation because they got created pretty fast) ?The kubelet handles this. There is no reason for anyone to have load-tested such a weird edge case because why would you ever do that? The slowdown is likely the Kubelet getting access to all the data because that has to wait for the scheduler (because of NodeRestriction) and then it might be hitting a client-side rate limit or APF rate limit after that? Honestly not sure, again we don't load test every ludicrous scenario because there are an unlimited number of those :) Nor do we put in options to control them without good reason.