I am at wits end searching for several days now, I am no terraform expert but I cant seem to find anyway to get all the GCP Vm instance List of external ips inside of a Kubernetes cluster as i then need to use this list to add them to allowed list for Database access.
Can someone point me to an example or what
tried something like this but cant use * =\
data "google_compute_instance" "instances" {
}
output "instance_ids" {
value = "${data.google_compute_instance.instances.*.network_interface.0.access_config.0.nat_ip }"
}
Rather than think about the reality that GKE nodes are implemented as Compute Engines, maybe think about GKE nodes in their Kubernetes concept. As a test, I created a cluster with 3 nodes and then ran:
kubectl get nodes -o wideThe result was
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
gke-c1-default-pool-5b6b460a-59nb Ready <none> 68s v1.13.11-gke.14 10.128.0.42 35.222.104.41 Container-Optimized OS from Google 4.14.138+ docker://18.9.7
gke-c1-default-pool-5b6b460a-ggh9 Ready <none> 68s v1.13.11-gke.14 10.128.0.41 35.192.152.130 Container-Optimized OS from Google 4.14.138+ docker://18.9.7
gke-c1-default-pool-5b6b460a-j8nn Ready <none> 67s v1.13.11-gke.14 10.128.0.40 104.197.68.223 Container-Optimized OS from Google 4.14.138+ docker://18.9.7Please pay attention to the column called EXTERNAL-IP. I then compared these against the Compute Engine VM instance exposed public IP and found them to be identical.
Now that we see that examining the nodes using kubectl returns the information we desire, we can now potentially use a Terraform equivalent to this command. For example, the Kubernetes Provider.
My terraform knowledge is a little haisy but can you not do the following?
As mentioned on this page: https://www.terraform.io/docs/provisioners/local-exec.html
resource "gcp_instance" "web" {
# ...
provisioner "local-exec" {
command = "echo gcloud compute instances list --format=\"value(External IP)\" --filter=\"name~'gke-$cluster_name-$pool_name*' --format=\"value(External_IP)\" "} >> public_ips.txt"
}
}
then read it from the file?
I came up with one way to do this using Terraform, but it's slightly hacky as it involves a bit of string manipulation on some of the URLs due to the GKE cluster metadata returning a list of Instance Group Managers and not Instance Groups (despite what the attribute is labelled). However it does work for the cluster I tested it on.
If it helps, here's a Terraform example that outputs the external IPs for all the nodes compute instances:
provider "google" {
version = "~> 2"
project = "my-project"
}
// GKE cluster details
data "google_container_cluster" "my_cluster" {
name = "my-cluster-name"
location = "my-location"
}
// GKE node instance group details
data "google_compute_instance_group" "node_instance_groups" {
for_each = toset(data.google_container_cluster.my_cluster.node_pool[0].instance_group_urls)
self_link = replace(each.key, "instanceGroupManagers", "instanceGroups")
}
// GKE node compute instance details
data "google_compute_instance" "nodes" {
for_each = toset(flatten([for x in data.google_compute_instance_group.node_instance_groups : x.instances[*]]))
self_link = each.key
}
// Return the external IPs for all GKE node instances
output "external_ips" {
value = [for x in data.google_compute_instance.nodes : x.network_interface[0].access_config[0].nat_ip]
}