I have a specific workflow to set up a fresh Kubernetes cluster on Google Cloud. And I want to automate the process with Terraform. Those are the steps:
Create cluster
gcloud beta container --project "my-google-project" clusters create "cluster-name" --zone "europe-west3-b"Setup Helm repos
helm repo add stable https://kubernetes-charts.storage.googleapis.com/
helm repo add jetstack https://charts.jetstack.io/
helm repo update
Install NGINX Ingress
kubectl create clusterrolebinding cluster-admin-binding --clusterrole cluster-admin --user $(gcloud config get-value account)
helm install nginx-ingress stable/nginx-ingressInstall Cert-Manager
kubectl apply --validate=false -f https://raw.githubusercontent.com/jetstack/cert-manager/v0.13.0/deploy/manifests/00-crds.yaml
kubectl create namespace cert-manager
helm install cert-manager jetstack/cert-manager --namespace cert-managerThe first step will probably look like this:
resource "google_container_cluster" "primary" {
name = "cluster-name"
location = "europe-west3-b"
initial_node_count = 3
master_auth {
username = ""
password = ""
client_certificate_config {
issue_client_certificate = false
}
}
node_config {
oauth_scopes = [
"https://www.googleapis.com/auth/logging.write",
"https://www.googleapis.com/auth/monitoring",
]
metadata = {
disable-legacy-endpoints = "true"
}
}
}
But I have no idea how to approach steps 2 - 4.
While Terraform makes sense for building and provisioning cloud infrastructure for things like Kubernetes to run on, it doesn't necessarily make sense to be used to configure said infrastructure after deployment. I think most infrastructure designs would consider applications deployed onto a provisioned cluster as configurations to said cluster. The semantics here are surely a bit nuanced but I maintain that a tool like Ansible is better suited to deploy applications to your cluster after provisioning.
So my advice would be to define a handful of Ansible Roles. Perhaps:
create_cluster
deploy_helm
install_nginx_ingress
install_cert_manager
Within each respective role, define the tasks and variables that are required to be used as per the Galaxy schema. Lastly, define a Playbook that Ansible uses to include or import these roles. This would allow you to provision your infrastructure and deploy all of the required applications to it in a single command:
ansible-playbook playbook.yml