I'm trying to deploy Spinnaker into a Kubernetes cluster. To do the trick, I use Halyard which uses Helm. While I'm trying to run my Helm pod I have the following output:
Cluster "default" set.
Context "default" created.
User "user" set.
Context "default" modified.
Switched to context "default".
Creating /home/spinnaker/.helm
Creating /home/spinnaker/.helm/repository
Creating /home/spinnaker/.helm/repository/cache
Creating /home/spinnaker/.helm/repository/local
Creating /home/spinnaker/.helm/plugins
Creating /home/spinnaker/.helm/starters
Creating /home/spinnaker/.helm/cache/archive
Creating /home/spinnaker/.helm/repository/repositories.yaml
Adding stable repo with URL: https://kubernetes-charts.storage.googleapis.com
Adding local repo with URL: http://127.0.0.1:8879/charts
$HELM_HOME has been configured at /home/spinnaker/.helm.
Tiller (the Helm server-side component) has been upgraded to the current version.
Happy Helming!
All seems correct. But my pod raise a CrashLoopBackOff event without any other error and my pod restart again for no apparent reason.
The dockerfile I'm using to build the helm docker image is the following:
FROM gcr.io/spinnaker-marketplace/halyard:stable
ARG GCP_SPINNAKER_GCR_KEY
# install helm
WORKDIR /home/spinnaker
# get helm
RUN curl https://raw.githubusercontent.com/kubernetes/helm/master/scripts/get > get_helm.sh
RUN sed -i 's/\/usr\/local\/bin/\/home\/spinnaker/g' get_helm.sh
# sudo user workaround
RUN sed -i 's/sudo //g' get_helm.sh
RUN chmod u+x get_helm.sh
# add the current folder to the path
ENV PATH="/home/spinnaker:${PATH}"
# install helm
RUN ./get_helm.sh
# importing the configuration script
ADD shell/halyard-configure.sh .
# auhtorize the spinnaker user to execute the configuration script
USER root
RUN chown -R spinnaker halyard-configure.sh
USER spinnaker
# create the gcp key directory for docker registry
RUN mkdir -p ~/.gcp
RUN echo $GCP_SPINNAKER_GCR_KEY | base64 -d > ~/.gcp/gcr-account.json
ENTRYPOINT [ "./halyard-configure.sh" ]
CMD "/opt/halyard/bin/halyard"And here is the content of the halyard-configure.sh shell script:
#!/usr/bin/env bash
set -e
# configure kubectl
kubectl config set-cluster default --server=https://kubernetes.default --certificate-authority=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt
kubectl config set-context default --cluster=default
token=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
kubectl config set-credentials user --token=$token
kubectl config set-context default --user=user
kubectl config use-context default
# configure helm
helm init --service-account tiller --upgrade
Your entrypoint script needs to end with the magic line exec "$@".
In Docker in general, a container startup launches the container entrypoint, passing it the command as parameters. (A Kubernetes pod spec refers to these parts as "command" and "args".) Once the entrypoint completes, the container exits. Since your entrypoint script just runs kubectl config and helm init commands which all complete promptly, the container exits almost immediately; when it does, Kubernetes restarts it; and when it has to restart it more than two or three times, it goes into CrashLoopBackOff state.
The usual way to get around this is to set up the entrypoint script to do any required first-time setup, then exec the command that was passed to it as parameters. Then the command (in your case, /opt/halyard/bin/halyard) winds up being "the main container process", and has the magic process ID 1, and will receive signals at container termination time.
Also note that there is a reasonably standard pattern for accessing the Kubernetes API from a pod that involves configuring a service account for the pod and using an official API, or else launching a kubectl proxy sidecar. You might be able to use that in place of the manual setup steps you have here. (I've never tried to launch Helm from inside a Kubernetes pod, though.)