Pre-deploying Kubernetes loadbalancer with terraform on DigitalOcean?

2/10/2019

I'm learning about creating a k8s cluster on DO using terraform, I've been trying to take the ID of the single K8s node I've created, and reference it from the loadbalancer.

The main reasoning for this is so that I can declare the FQDN in the .tf file.

First, here is the cluster declaration:

variable "digitalocean_token" {}

provider "digitalocean" {
  token = "${var.digitalocean_token}"
}

resource "digitalocean_kubernetes_cluster" "foo" {
  name    = "foo"
  region  = "nyc1"
  version = "1.12.1-do.2"

  node_pool {
    name       = "woker-pool"
    size       = "s-1vcpu-2gb"
    node_count = 1
  }
}

And here is the load balancer declaration:

resource "digitalocean_loadbalancer" "foo" {
  name = "k8s-lb.nyc1"
  region = "nyc1"

  forwarding_rule {
    entry_port = 80
    entry_protocol = "http"

    target_port     = 80
    target_protocol = "http"
  }
  droplet_ids = ["${digitalocean_kubernetes_cluster.foo.node_pool.0.id}"]
}

output "loadbalancer_ip" {
  value = "${digitalocean_loadbalancer.foo.ip}"
}

resource "digitalocean_record" "terraform" {
  domain = "example.com" # "${digitalocean_domain.example.name}"
  type   = "A"
  name   = "terraform"
  value  =  "${digitalocean_loadbalancer.foo.ip}"
}

# Output the FQDN for the record
output "fqdn" {
  value = "${digitalocean_record.terraform.fqdn}"
}

I'm guessing that maybe the digitalocean_loadbalancer resources is only setup to work with individual droplets?


Here are the output errors: when I run terraform apply:

* output.loadbalancer_ip: Resource 'digitalocean_loadbalancer.foo' not found for variable 'digitalocean_loadbalancer.foo.ip'

* digitalocean_record.terraform: Resource 'digitalocean_loadbalancer.foo' not found for variable 'digitalocean_loadbalancer.foo.ip'

* digitalocean_loadbalancer.foo: droplet_ids.0: cannot parse '' as int: strconv.ParseInt: parsing "d4292e64-9c0a-4afb-83fc-83f239bcb4af": invalid syntax

Pt. 2

I added a digitalocean_droplet resource, to see what kind of id was passed to the load balancer.

resource "digitalocean_droplet" "web" {
  name      = "web-1"
  size      = "s-1vcpu-1gb"
  image     = "ubuntu-18-04-x64"
  region    = "nyc1"
}

digitalocean_kubernetes_cluster.foo.node_pool.0.id = '6ae6a787-d837-4e78-a915-cb52155f66fe'

digitalocean_droplet.web.id = 132533158

-- denski
digital-ocean
kubernetes
load-balancing
terraform

2 Answers

2/10/2019

You're using the wrong attribute reference for your load balancer droplet ids.

droplet_ids = ["${digitalocean_kubernetes_cluster.foo.node_pool.0.id}"]

This will use the node_pool id linked here

What you actually need to do is use the node_pool nodes id, which is referenced here

droplet_ids = "${digitalocean_kubernetes_cluster.foo.node_pool.0.nodes}"

The next problem you're going to have is that this returns a list of maps, and you'll need to build a list of ids from that. I'm not currently sure how to solve that, I'm afraid, but this should move you along hopefully.

It seems from your answer however, that what you want to do is update DNS for your loadbalancer.

You can do this external-dns using the digitalocean provider

Simply deploy this as a pod, specifying the required configuration, and ensure that the arg --source=service is set.

If you want to go a step further, and allow updating DNS with specific hostname, deploy an ingress controller like nginx-ingress and specify ingresses for your applications. The external-dns deployment (if you set --source=ingress) will the hostname from your ingress and update DNS for you.

-- jaxxstorm
Source: StackOverflow

2/10/2019

So, the digitalocean_loadbalancer resource has an optional droplet_tag argument, which can be used to supply a common tag given to the created nodes/droplets.

However, when declaring a load-balancer inside kubernetes, a new one will still be created. So for now at least, it would appear that defining the domain/CNAME record with terraform isn't possible on digitalocean

-- denski
Source: StackOverflow