How can I replace variables in annotation via Kustomize?

9/21/2021

Any ideas how can I replace variables via Kustomize? I simply want to use a different ACCOUNT_ID and IAM_ROLE_NAME for each overlay.

apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::${ACCOUNT_ID}:role/${IAM_ROLE_NAME}

Thanks in advance!

-- cosmos-1905-14
kubernetes
kustomize

1 Answer

9/21/2021

Kustomize doesn't use "variables". The way you would typically handle this is by patching the annotation in an overlay. That is, you might start with a base directory that looks like:

base
├── kustomization.yaml
└── serviceaccount.yaml

Where serviceaccount.yaml contains your ServiceAccount manifest:

apiVersion: v1
kind: ServiceAccount
metadata:
    name: my-service-account
    annotions:
      eks.amazonaws.com/role-arn: "THIS VALUE DOESN'T MATTER"

And kustomization.yaml looks like:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: my-namespace

resources:
  - serviceaccount.yaml

Then in your overlays, you would replace the eks.amazonaws.com/role-arn annotation by using a patch. For example, if you had an overlay called production, you might end up with this layout:

.
├── base
│   ├── kustomization.yaml
│   └── serviceaccount.yaml
└── overlay
    └── production
        ├── kustomization.yaml
        └── patch_aws_creds.yaml

Where overlay/production/patch_aws_creds.yaml looks like:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-service-account
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::1234:role/production-role

And overlay/production/kustomization.yaml looks like:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - ../../base

patches:
  - patch_aws_creds.yaml

With this in place, running...

kustomize build overlay/production

...would generate output using your production role information, and so forth for any other overlays you choose to create.


If you don't like the format of the strategic merge patch, you can use a json patch document instead. Here's what it would look like inline in your kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - ../../base

patches:
  - target:
      version: v1
      kind: ServiceAccount
      name: my-service-account
    patch: |-
      - op: replace
        path: /metadata/annotations/eks.amazonaws.com~1role-arn
        value: arn:aws:iam::1234:role/production-role

I don't think this really gets you anything, though.

-- larsks
Source: StackOverflow