I have to create a k8s job via Terraform and somehow organize the mechanism of waiting for this job to be completed (since Terraform fails to do this https://github.com/terraform-providers/terraform-provider-kubernetes/issues/534). I figured out nothing better than using a null resource with a command that waits for k8s job (namely kubectl wait
). This stuff will be executed in a Docker container on a CI. Moreover, I need to go through the Bastion to get to the k8s cluster. I use an SSH tunnel for that:
provider "ssh" {
port = ....
}
provider "kubernetes" {
config_context = "..."
config_context_cluster = "..."
host = "api.${k8s_host}:${data.ssh_tunnel.k8s.port}"
}
data "ssh_tunnel" "k8s" {
host = "bastion.....com"
local_address = "localhost:0"
remote_address = "api.${k8s_host}:443"
}
All k8s resources successfully create so I assume that SSH tunnel works fine. But how to use it for a null resource? Here is it:
resource "null_resource" "wait" {
provisioner "local-exec" {
connection {
type = "ssh"
bastion_host = data.ssh_tunnel.k8s.host
bastion_private_key = file("~/.ssh/id_rsa")
bastion_port = data.ssh_tunnel.k8s.port
host = "api.${k8s_host}"
port = 443
}
command = "kubectl wait ...."
}
triggers = {
job_ids = join(", ", kubernetes_job.a-job.*.id)
}
}
But no luck, I got "The connection to the server api.${k8s_host} was refused - did you specify the right host or port?"
So there are two questions: 1. How to wait for a job in a different way 2. If 1 is impossible (I'm sure that it is) how to reuse the SSH tunnel in the right way.
P.S Yes, I read the documentation https://www.terraform.io/docs/provisioners/connection.html but I definitely do smth wrong.
I figured out how to reuse SSH tunnel opened by Terraform:
resource "null_resource" "wait" {
provisioner "local-exec" {
command = "kubectl wait --server=https://api.${k8s_host}:${data.ssh_tunnel.k8s.port} --for=condition=complete --timeout=3000s job/a-job"
}
triggers = {
job_ids = join(", ", kubernetes_job.a-job.*.id)
}
}