Unable to write to Kinesis from EKS container

8/4/2019

I'm running a container on EKS in account A and need to write to Kinesis stream (firehose) in another account (account B).

I'm using boto3, when running locally I use the user IAM credentials set using aws configure.

But when deployed to EKS it's unable to write to that stream because it can't find it. I suspect that I need to somehow set up IAM role on account A and use it in EKS on account B, but I just can't find a way.

Any help will be great...

client = boto3.client('firehose')
client.put_record_batch(DeliveryStreamName=self.kinesis_stream_name, Records=records)
-- ElinN
amazon-eks
amazon-kinesis
amazon-web-services
kubernetes

1 Answer

8/4/2019
  • You would need to enable cross account permission between account A(Kinesis) and B(EKS) in aws. Create an IAM role to access Kinesis.

  • Cross Account Permissions AWS

  • So when your container hosts (EC2 instances) get the instance profile from above created cross account IAM role.Your containers will be able to access account A(Kinesis). Look Here

  • Go to IAM -> Roles -> ecsInstanceRole ---> now update this role with your newly created policies. ecsInstanceRole is a default role created for all EKS container instances (spot or on-demand) refer.

  • After adding the roles, add the assume-role api call too in logic so that the containers will assume the role for Kinesis in another account.

  • Boto example from aws docs:

import boto3

# Create IAM client
sts_default_provider_chain = boto3.client('sts')

print('Default Provider Identity: : ' + sts_default_provider_chain.get_caller_identity()['Arn'])

role_to_assume_arn='arn:aws:iam::123456789012:role/roleName'
role_session_name='test_session'

response=sts_default_provider_chain.assume_role(
    RoleArn=role_to_assume_arn,
    RoleSessionName=role_session_name
)

creds=response['Credentials']

sts_assumed_role = boto3.client('sts',
    aws_access_key_id=creds['AccessKeyId'],
    aws_secret_access_key=creds['SecretAccessKey'],
    aws_session_token=creds['SessionToken'],
)

print('AssumedRole Identity: ' + sts_assumed_role.get_caller_identity()['Arn'])
  • For an end to end example see , just replace the roles accordingly (in your case ecsInstanceRole -> would be updated, instead of new role)

PS: Also just to test you could use the actual programmatic access Access+Secret Keys for AWS account A in your boto3 api calls. But not at all recommended.

-- sanster_23
Source: StackOverflow