I am interested in know what is the best way to automate YAML
deployments in Kubernetes
I have a cert-manager
helm installation and a kong YAML
manifest and an Ingress
resource created, installed and working in a Kubernetes cluster
This deployment provides me TLS
encryption in order to get the https
protocol to my application service which also is installed via helm (I’ve created a helm chart for it)
My objective is to find the way of executing those YAML
files that I have created, and some helm commands of an automatic way and not execute manually the steps and the process.
Maybe is important to keep in mind that all these cert-manager, kong and Ingress deployment is associated to my helm chart service application which I've created, so that I have been deploying the following:
I've created a helm chart of my service application and I've installed it via helm install ...
command
I've installed kong
and kong-ingress-controller
from a YAML manifest using Postgres
like external service.
I've installed cert-manager
via helm adding its helm repo add jetstack https://charts.jetstack.io
and executing
helm install \
--name cert-manager \
--namespace cert-manager \
--version v0.7.0 \
jetstack/cert-manager
Ingress
and KongIngress
resources to allow the access to my helm chart application service and allow to kong manage the ingress operations and other actions like the routes, certificates and service creation, and also associate somethings like basic-auth
plugin to my service applicationAll this process was made via kubectl apply ...
executing YAML files and also helm ....
command from a CLI.
This mean, that currently this process is highly dependent of an human events, and I would like to automatize somethings like the kong
and cert-manager
installation process and even if is possible the Ingress
creation.
Do I am right when I think that this may be possible?
What is the best way to do it?
I have been reading and looking in some places, and some people also have orient me in the following alternatives:
From my helm chart application ?
I have been reading that helm has something named hooks
The hooks
itself describe the following:
A kind of mechanism to allow chart developers to intervene at certain points in a release's life cycle. For example, you can use hooks to:
Load a ConfigMap or Secret during install before any other charts are loaded.
Execute a Job to back up a database before installing a new chart, and then execute a second job after the upgrade in order to restore data.
Run a Job before deleting a release to gracefully take a service out of rotation before removing it
Could be the helm hooks approach a good option? I think that with them I can automatize or at least that the helm chart
application take over the secrets creation and some pre/post-deployment
.
Can I consider the kong
, cert-manager
and Ingress
resources operations (commands and YAML files execution) like operations or actions to be managed by helm hooks?
There is a Helm and Kubernetes providers available from terraform. I could explore that possibility of install these software packages from terraform scripts.
This terraform approach could be useful to install my helm chart application
and the cert-manager
helm
Kong also like terraform provider
Currently there are some third party repositories to work with kong from terraform. Terraform Provider Kong it's more proper to me due to I am using Azure Kubernetes service, and the different resource configurations which are possible to implement looks very good. I am not sure if I would have to inject manually somethings like certificates
, route
resources when I work with the kong-ingress-controller
and cert-manager communication, but even this terraform kong provider allow me the possibility of import a existing routes terraform import kong_route.<route_identifier> <route_id>
Could be apply it this to certificates
and other resources?
What about of automate deployments via python? How are going the things with this alternative? Is possible do it interacting from python to kubernetes and executing the commands operations which I have described above (kubectl
and helm
to create resources from YAML files and helm charts available in local and remote repositories)
In affirmative case ... How to can I address this option?
I have found this option which use ansible but it involve some other things like Automate broker ansible and Kubernetes service catalog, which are unknown for me at the moment.
Can I create those resources (Ingress
and kong and cert-manager installation and configuration) via jupyter notebooks, writing down directly to Kubernetes cluster? Is this possible? How to can I do it?
So, I think that the terraform approach (using the helm
and kong
providers would be very useful tools from a infrastructure as a code perspective, but I am not sure if I would be choose it, maybe could there is less complex alternatives?
Somebody has performed automation of resources and things in Kubernetes cluster. I suppose that it is a normal or expected process to do, although is my first time.
If someone can point me in the right approach to solve my particular scenario I will be highly grateful. :)
A possible solution is to use initContainer that will wait for other services to be UP.
Typically, for a given A and B application, you will use helm or event better helmfile to deploy them both at the same time. But for the pod of B, you will add an initContainer, that will check say every 10 seconds if service A is started. By doing this, you are sure that service B will only start when service A is fully started and ready. You can have a look at https://kubernetes.io/docs/concepts/workloads/pods/init-containers/. This requires to configure livenessProbe and readynessProbe.
Note that this will not really order chart installation, simply ensure that each service will wait for dependendancies before start. And you can install many chart at the same time using helmfile (https://github.com/roboll/helmfile).
Unfortunately you cannot control the order of installation of dependent Helm charts. Helm groups together the chart's templates and all of its dependencies' templates and applies them as if they came from the same chart (see here for more details on the ordering).
You could write a Python script that installs the charts one after the other using PyHelm (full disclosure - I'm one of the maintainers of the package).
In addition, you can use Helm hooks in order to control the order of the installation - that can take care of your additional yamls (e.g. using the post-install/post-upgrade hooks).