My service need a cfg file which need to be changed before containers start running. So it is not suitable to pack the cfg into docker image. I need to copy from cluster to container, and then the service in container start and reads this cfg.
How can I do this?
You could use ConfigMap. Create a config map resource and config your container to load the config map accordingly. You container will then be able to load those variables from its environment variable.
Here is the reference: https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/
I think for your use-case , Init Containers might be the best fit. Init Containers are like small scripts that you can run before starting your own containers in kubernetes pod , they must exit. You can have this config file updated in shared Persistent Volume between your Init container and your container.
Following article gives a nice example as to how this can be done
UPDATE :
I found another answer from stackoverflow which might be related and give you a better approach in handling this
can i use a configmap created from an init container in the pod
An example
ouput
NAME READY STATUS RESTARTS AGE
cassandra-0 1/1 Running 0 70s
[root@green--1 ~]# k exec -it cassandra-0 -n green -- /bin/bash
root@cassandra-0:/# ls /config/cassandra/
cassandra.yaml
How the directory is shared
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: cassandra
namespace: green
labels:
app: cassandra
spec:
serviceName: cassandra
replicas: 1
selector:
matchLabels:
app: cassandra
template:
metadata:
labels:
app: cassandra
spec:
terminationGracePeriodSeconds: 1800
initContainers:
- name: copy
image: busybox:1.28
command: ["/bin/sh", "-c", "cp /config/cassandra.yaml /config/cassandra/"]
volumeMounts:
- name: tmp-config
mountPath: /config/cassandra/
- name: cassandraconfig
mountPath: /config/
containers:
- name: cassandra
#image: gcr.io/google-samples/cassandra:v13
image: cassandra:3.11.6
imagePullPolicy: Always
#command: [ "/bin/sh","-c","su cassandra && mkdir -p /etc/cassandra/ && cp /config/cassandra/cassandra.yaml /etc/cassandra/" ]
ports:
- containerPort: 7000
name: intra-node
- containerPort: 7001
name: tls-intra-node
- containerPort: 7199
name: jmx
- containerPort: 9042
name: cql
resources:
limits:
cpu: "500m"
memory: 4Gi
requests:
cpu: "500m"
memory: 4Gi
securityContext:
capabilities:
add:
- IPC_LOCK
lifecycle:
postStart:
exec:
command:
- /bin/sh
- -c
- "cp /config/cassandra/cassandra.yaml /etc/cassandra/"
preStop:
exec:
command:
- /bin/sh
- -c
- nodetool drain
env:
- name: MAX_HEAP_SIZE
value: 1G
- name: HEAP_NEWSIZE
value: 100M
- name: CASSANDRA_SEEDS
value: "cassandra-0.cassandra.green.svc.cluster.local"
- name: CASSANDRA_CLUSTER_NAME
value: "green"
- name: CASSANDRA_DC
value: "ee1-green"
- name: CASSANDRA_RACK
value: "Rack1-green"
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
readinessProbe:
exec:
command:
- /bin/bash
- -c
- /ready-probe.sh
initialDelaySeconds: 15
timeoutSeconds: 55
volumeMounts:
- name: cassandradata
mountPath: /cassandra_data
- name: tmp-config
mountPath: /config/cassandra/
volumes:
- name: cassandraconfig
configMap:
name: cassandraconfig
- name: tmp-config
emptyDir: {}
# 1 Creating a volume to move date from init container to main container without making mount ReadOnly
# These are converted to volume claims by the controller
# and mounted at the paths mentioned above.
volumeClaimTemplates:
- metadata:
name: cassandradata
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: rook-ceph-block
resources:
requests:
storage: 5Gi