How if I interact with different kubernetes clusters in different terminals sessions with out having to switch contexts all the the time?

10/29/2019

I am testing role differences right now so I have a context for each role setup.

Terminal session Admin, I want to be able to use context Admin in one session so I can update the rules as needed.

In terminal session User, I want to be able to test that role via its context.

(Note: I am on EKS so roles map to IAM roles)

-- Josh Beauregard
amazon-eks
aws-eks
kubectl
kubernetes

4 Answers

10/29/2019

You can create a copy of your context file that is located under ~/.kube/config, and in 2 different shells, point to 2 different config files using export KUBECONFIG=/path/to/kubeconfig1 on the first and export KUBECONFIG=/path/to/kubeconfig2 on the second. You can edit those files to have 2 different context selected.

To easily select contexts/switch between them, you can use kubectx, as suggested by Blokje5.

-- Utku Ă–zdemir
Source: StackOverflow

10/29/2019

I always like kubectx as a way to quickly switch context. If you correctly setup your contexts with the aws-iam-authenticator, like so:

users:
- name: kubernetes-admin
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1alpha1
      command: aws-iam-authenticator
      args:
        - "token"
        - "-i"
        - "<cluster_id>"
        - "-r"
        - "<admin_role_arn>"
- name: kubernetes-user
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1alpha1
      command: aws-iam-authenticator
      args:
        - "token"
        - "-i"
        - "<cluster_id>"
        - "-r"
        - "<user_role_arn>"

This should allow you to easily switch contexts. (Note: This assumes an assume-role type situation. You can also pass AWS_PROFILE to the aws-iam-authenticator instead.)

-- Blokje5
Source: StackOverflow

10/29/2019

Here are some tips for managing multiple kubectl contexts:

  • Use asdf to manage multiple kubectl versions
  • Set the KUBECONFIG env var to change between multiple kubeconfig files
  • Use kube-ps1 to keep track of your current context/namespace
  • Use kubectx and kubens to change fast between clusters/namespaces
  • Use aliases to combine them all together

Take a look at this article, it explains how to accomplish this: Using different kubectl versions with multiple Kubernetes clusters (Disclaimer: I wrote the mentioned article)

I also recommend this reads: Mastering the KUBECONFIG file and Configure Access to Multiple Clusters

-- Eduardo Baitello
Source: StackOverflow

10/29/2019

Well, I am an idiot.

There is no answer in the --help output for kubectl, however, there is output for this in the man page.

All one has to do is throw the --context flag into their command.

-- Josh Beauregard
Source: StackOverflow