I have my company S3 (companys3) bucket with multiple files for example file1, file2 and file3. And client S3 bucket (clients3) with some files that i don't know.
What I want is the solution for opening only file2 from companys3 to clients3.
I found solutions about how to copy/clone whole buckets. But couldn't find any that copy only specific files.
Till this time wi copy files through Kubernetes pods, but files become too large to handle this way (ower 20GB one file), so I am searching to solution that allows us to quit using Kubernetes pods ad transfer clients.
Suppose you have SOURCE and DESTINATION bucket. You need to delegate permission properly.
SOURCE S3
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "DelegateS3Access",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::DESTINATION_BUCKET_ACCOUNT_NUMBER:root"
},
"Action": [
"s3:ListBucket",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::SOURCE_BUCKET_NAME/*",
"arn:aws:s3:::SOURCE_BUCKET_NAME"
]
}
]
}
DESTINATION S3
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::SOURCE_BUCKET_NAME",
"arn:aws:s3:::SOURCE_BUCKET_NAME/*"
]
},
{
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:PutObject",
"s3:PutObjectAcl"
],
"Resource": [
"arn:aws:s3:::DESTINATION_BUCKET_NAME",
"arn:aws:s3:::DESTINATION_BUCKET_NAME/*"
]
}
]
}
After that you will use AWS CLI
aws s3 sync s3://SOURCE-BUCKET-NAME s3://DESTINATION-BUCKET-NAME --source-region SOURCE-REGION-NAME --region DESTINATION-REGION-NAMEYou also need to attach an IAM policy to the source bucket to add access for the destination account. And then you can copy the bucket content with AWS CLI Check this guide
You can use S3 command line (awscli).
aws s3 cp s3://COMPANY_BUCKET/filename s3://CLIENT_BUCKET/filename