K
Q

Question

AWS / Heptio Authenticator from an EC2 instance

8/28/2018

So I've followed the AWS article Deploying the Heptio Authenticator to kops, and I was able to successfully able to get things running. From my laptop, which has a fully configured AWSCLI setup, I can run any kubectl command since I'm an administrator. The token is obtained by using the following command in my ~/.kube/config file:

users:
- name: mycluster-exec
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1alpha1
      args:
      - token
      - --cluster-id
      - mycluster
      - --role
      - arn:aws:iam::<account-number>:role/KubernetesAdministrator
      command: aws-iam-authenticator
      env: null

Now what I'd like to do is essentially described in the GitHub issue EKS heptio authentication using IAM without AWSCLI. I'm not using EKS, but the principle is the same. I have EC2 instances that are running my CI system's build agents, and I would like those build agents to NOT have static, hard-coded credentials (i.e. a static AWS ID and Secret Key). I'd prefer those nodes to also use the aws-iam-authenticator binary to temporarily grab credentials as needed to deploy / make changes to my Kubernetes clusters.

I created a Role / Instance profile called KubernetesCIRole, and attached that IAM role to my EC2 build agent nodes. I then added the following to the ConfigMap:

apiVersion: v1
data:
  config.yaml: |
    clusterID: mycluster
    server:
      mapRoles:
      - roleARN: arn:aws:iam::<account-number>:role/KubernetesAdministrator
        username: kubernetes-admin
        groups:
        - system:masters
      - roleARN: arn:aws:iam::<account-number>:role/KubernetesCIRole
        username: kubernetes-admin
        groups:
        - system:masters
kind: ConfigMap
metadata:
  labels:
    k8s-app: heptio-authenticator-aws
  name: heptio-authenticator-aws
  namespace: kube-system

However, when I configure the ~/.kube/config on that EC2 build agent machine, and then run something simple like kubectl --v=10 get pods, I receive the following:

I0828 10:16:30.605964    5196 loader.go:359] Config loaded from file /home/ubuntu/.kube/config
I0828 10:16:30.606744    5196 loader.go:359] Config loaded from file /home/ubuntu/.kube/config
I0828 10:16:30.607704    5196 loader.go:359] Config loaded from file /home/ubuntu/.kube/config
...
I0828 10:16:33.092683    5196 round_trippers.go:386] curl -k -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.11.2 (linux/amd64) kubernetes/bb9ffb1" 'https://api.mycluster.k8s.mycompany.com/api?timeout=32s'
I0828 10:16:33.605698    5196 round_trippers.go:405] GET https://api.mycluster.k8s.mycompany.com/api?timeout=32s 401 Unauthorized in 512 milliseconds
I0828 10:16:33.605727    5196 round_trippers.go:411] Response Headers:
I0828 10:16:33.605734    5196 round_trippers.go:414]     Content-Type: application/json
I0828 10:16:33.605749    5196 round_trippers.go:414]     Www-Authenticate: Basic realm="kubernetes-master"
I0828 10:16:33.605758    5196 round_trippers.go:414]     Content-Length: 129
I0828 10:16:33.605767    5196 round_trippers.go:414]     Date: Tue, 28 Aug 2018 10:16:33 GMT
I0828 10:16:33.608443    5196 request.go:897] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Unauthorized","reason":"Unauthorized","code":401}
I0828 10:16:33.610858    5196 cached_discovery.go:111] skipped caching discovery info due to Unauthorized
F0828 10:16:33.610901    5196 helpers.go:119] error: the server doesn't have a resource type "pods"

Why doesn't this work? If I've specified in the Heptio / AWS IAM Authenticator ConfigMap that a given role has cluster administrator access (or anything else, for that matter), shouldn't I be able to authenticate?

Thanks for the help in advance!

-- Scott Crooks

amazon-iam

amazon-web-services

authentication

kubectl

kubernetes

1 Answer

8/29/2018

Finally figured it out. The AWS documentation Managing Users or IAM Roles for your Cluster has the part that's needed:

First, you need to create an IAM role (and corresponding instance profile) which has the following basic permissions, and attach it to your EC2 instance:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "ec2.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Then you need to update the ConfigMap with something like the following:

apiVersion: v1
kind: ConfigMap
metadata:
  name: aws-auth
  namespace: kube-system
data:
  mapRoles: |
    - rolearn: <ARN of instance role (not instance profile)>
      username: system:node:{{EC2PrivateDNSName}}
      groups:
        - system:masters

The key part is the username system:node:{{EC2PrivateDNSName}}. I guess the {{EC2PrivateDNSName}} serves as a placeholder for any EC2 instance with the corresponding rolearn attached to it.

-- Scott Crooks

Source: StackOverflow

KQ

AWS / Heptio Authenticator from an EC2 instance

Similar Questions

1 Answer

K
Q