Edits to config-autoscaler configmap automatically revert a few minutes after changes applied

8/30/2019

I'm trying to tune the autoscaler for a Google Kubernetes Engine cluster with the cloud-run addon enabled. When I edit the configmap The changes are accepted by the API server. However, after a few minutes, the configmap reverts to the original version. Is there any way to tune the autoscaler while using the cloud-run cluster addon?

Steps to reproduce:

  1. Edit the configmap
kubectl edit cm config-autoscaler -n knative-serving

# ...

configmap/config-autoscaler configured
  1. Check the results:
kubectl get -n knative-serving configmap config-autoscaler -o json | jq '.data'

#=>

{
  "container-concurrency-target-default": "1",
  "container-concurrency-target-percentage": "0.5",
  "enable-scale-to-zero": "false",
  "max-scale-up-rate": "1000",
  "panic-threshold-percentage": "200.0",
  "panic-window-percentage": "10.0",
  "scale-to-zero-grace-period": "90s",
  "stable-window": "600s",
  "target-burst-capacity": "-1",
  "tick-interval": "2s"
}
  1. Wait a few minutes and check again:
kubectl get -n knative-serving configmap config-autoscaler -o json | jq '.data'

#=>

{
  "_example": "################################\n#                              #\n#    EXAMPLE CONFIGURATION     #\n#                              #\n################################\n# This block is not actually functional configuration,\n# but serves to illustrate the available configuration\n# options and document them in a way that is accessible\n# to users that `kubectl edit` this config map.\n#\n# These sample configuration options may be copied out of\n# this example block and unindented to be in the data block\n# to actually change the configuration.\n# The Revision ContainerConcurrency field specifies the maximum number\n# of requests the Container can handle at once. Container concurrency\n# target percentage is how much of that maximum to use in a stable\n# state. E.g. if a Revision specifies ContainerConcurrency of 10, then\n# the Autoscaler will try to maintain 7 concurrent connections per pod\n# on average. A value of 0.7 is chosen because the Autoscaler panics\n# when concurrency exceeds 2x the desired set point. So we will panic\n# before we reach the limit.\ncontainer-concurrency-target-percentage: \"1.0\"\n# The container concurrency target default is what the Autoscaler will\n# try to maintain when the Revision specifies unlimited concurrency.\n# Even when specifying unlimited concurrency, the autoscaler will\n# horizontally scale the application based on this target concurrency.\n#\n# A value of 100 is chosen because it's enough to allow vertical pod\n# autoscaling to tune resource requests. E.g. maintaining 1 concurrent\n# \"hello world\" request doesn't consume enough resources to allow VPA\n# to achieve efficient resource usage (VPA CPU minimum is 300m).\ncontainer-concurrency-target-default: \"100\"\n# When operating in a stable mode, the autoscaler operates on the\n# average concurrency over the stable window.\nstable-window: \"60s\"\n# When observed average concurrency during the panic window reaches\n# panic-threshold-percentage the target concurrency, the autoscaler\n# enters panic mode. When operating in panic mode, the autoscaler\n# scales on the average concurrency over the panic window which is\n# panic-window-percentage of the stable-window.\npanic-window-percentage: \"10.0\"\n# Absolute panic window duration.\n# Deprecated in favor of panic-window-percentage.\n# Existing revisions will continue to scale based on panic-window\n# but new revisions will default to panic-window-percentage.\npanic-window: \"6s\"\n# The percentage of the container concurrency target at which to\n# enter panic mode when reached within the panic window.\npanic-threshold-percentage: \"200.0\"\n# Max scale up rate limits the rate at which the autoscaler will\n# increase pod count. It is the maximum ratio of desired pods versus\n# observed pods.\nmax-scale-up-rate: \"10\"\n# Scale to zero feature flag\nenable-scale-to-zero: \"true\"\n# Tick interval is the time between autoscaling calculations.\ntick-interval: \"2s\"\n# Dynamic parameters (take effect when config map is updated):\n# Scale to zero grace period is the time an inactive revision is left\n# running before it is scaled to zero (min: 30s).\nscale-to-zero-grace-period: \"30s\"\n"
}

Is there something that I'm missing or is it not possible to edit the configmap the governs the knative serving autoscaler? If not, what are my alternatives?

-- Faun
google-cloud-run
google-kubernetes-engine
knative
knative-serving
kubernetes

1 Answer

9/1/2019

You can't make changes to this confgiMap as the GKE reconciler will revert the changed you make back.

Any resource with the addonmanager.kunernetes.io/mode: reconcile will have any changes made revert back as part of the managed components of GKE

-- Patrick W
Source: StackOverflow