Superset on Kubernetes with Helm not using custom values file

5/13/2021

I'm trying to install Apache Superset on a Kubernetes Cluster (AWS EKS) using Helm and following the official procedure described here.

bash-3.2$ helm repo add superset https://apache.github.io/superset
"superset" has been added to your repositories

bash-3.2$ helm search repo superset
NAME             	CHART VERSION	APP VERSION	DESCRIPTION                                       
superset/superset	0.1.2        	1.0        	Apache Superset is a modern, enterprise-ready b...

Since I want to use RDS and ElastiCache for database and cache respectively, instead of the bundled postgresql and redis, I need to override several values in the default values.yaml so I made a copy of the default values

bash-3.2$ helm show values superset/superset > custom-values.yaml

edited several sections such as

postgresql:
  ##
  ## Use the PostgreSQL chart dependency.
  ## Set to false if bringing your own PostgreSQL.
  enabled: false

[...]

  ##
  ## If you are bringing your own PostgreSQL, you should set postgresHost and
  ## also probably service.port, postgresqlUsername, postgresqlPassword, and postgresqlDatabase
  postgresHost: myproject.cluster-xxxxxxxxx.us-east-2.rds.amazonaws.com

and others, and installed my release with

bash-3.2$ helm upgrade --install --values custom-values.yaml superset superset/superset

as described in the guide.

The first three pods are created:

bash-3.2$ kubectl get pods
NAME                               READY   STATUS     RESTARTS   AGE
superset-7d96fc8787-vr6c6          0/1     Init:0/1   0          3m4s
superset-init-db-xqmd9             0/1     Init:0/1   0          3m3s
superset-worker-7fff4f497b-cnqs5   0/1     Init:0/1   0          3m4s

but then nothing happens. Logging in into the init-container of the superset-init pod I discovered that the process waiting for the database to become available is stuck because it is using the default environment variables instead of those I provided in the custom-values.yaml file:

bash-3.2$ kubectl exec -it --container wait-for-postgres superset-init-db-xqmd9 -- sh
/ # 
/ # env
REDIS_PORT=6379
KUBERNETES_PORT=tcp://172.20.0.1:443
KUBERNETES_SERVICE_PORT=443
HOSTNAME=superset-init-db-xqmd9
DB_PORT=5432
SUPERSET_PORT=tcp://172.20.72.4:8088
SUPERSET_SERVICE_PORT=8088
SUPERSET_PORT_8088_TCP_ADDR=172.20.72.4
SHLVL=1
HOME=/root
DB_NAME=superset
SUPERSET_PORT_8088_TCP_PORT=8088
SUPERSET_PORT_8088_TCP_PROTO=tcp
SUPERSET_PORT_8088_TCP=tcp://172.20.72.4:8088
TERM=xterm
KUBERNETES_PORT_443_TCP_ADDR=172.20.0.1
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
KUBERNETES_PORT_443_TCP_PORT=443
KUBERNETES_PORT_443_TCP_PROTO=tcp
DB_PASS=superset
SUPERSET_SERVICE_PORT_HTTP=8088
KUBERNETES_SERVICE_PORT_HTTPS=443
KUBERNETES_PORT_443_TCP=tcp://172.20.0.1:443
REDIS_HOST=superset-redis-headless
KUBERNETES_SERVICE_HOST=172.20.0.1
PWD=/
DB_HOST=superset-postgresql
SUPERSET_SERVICE_HOST=172.20.72.4
DB_USER=superset

EDIT:

some of the custom values in the custom-values.yaml file are actually used to override the defaults. E.g:

service:
  type: LoadBalancer

instead of

service:
  type: ClusterIP

and also postgresql and redis pods are not created when I set them as enabled: false but for some reason other custom values are not applied or passed to the secret storing env variables for the pods.

What am I doing wrong?

-- MariusPontmercy
apache-superset
kubernetes
kubernetes-helm

1 Answer

5/13/2021

According to the values.yaml of superset, I do see if you are bringing your own Postgres instance, from the above question as I've understood, You have to change the values of these

supersetNode:
  connections:
    # Change incase bringing your own redis and then also make `redis.enabled`:false
    redis_host: '{{ template "superset.fullname" . }}-redis-headless'
    redis_port: "6379"
    # You need to change below configuration incase bringing own pg instance and as you made `postgresql.enabled`:false that's correct incase bringing own pg instance
    db_host: <YOUR RDS PG HOST>
    db_port: "5432"
    db_user: <YOUR DB USER>
    db_pass: <YOUR DB PASS>
    db_name: <YOUR DB NAME | postgres>
-- Saikat Chakrabortty
Source: StackOverflow