I’m trying to recursively download a S3 folder from an EC2 instance. I have
role with a policy that includes: s3:ListBucket
among other things attached to
the EC2 instance and can download individual files but keep getting the
following error when trying to do so recursively:
“An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied”
The command being used that was erroring:
aws s3 cp s3://<bucket>/<path>/ ~/temp --recursive --debug
Note: aws s3 ls s3://<bucket>
also did not work.
However the following did work:
aws s3 cp s3://<bucket>/<folder>/very-important.txt ~/temp
The current policy resource section has:
"Resource": "arn:aws:s3:::<bucket>/*"
The reason is there are different resource types that this document will spell out:
https://docs.aws.amazon.com/service-authorization/latest/reference/list_amazons3.html
So when doing a policy the permission may require for example a “bucket”
resource type or a “object” resource type. For example: ListObjects
requires a
“bucket” resource and GetObject
requires a “object” resource type. So when
crafting the policy you have to take that into consideration likely separating
out the permissions objects such that each has the appropriate resource type
associated with it.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [ "s3:GetObject" ],
"Resource": [ "arn:aws:s3:::example-devops/*" ]
},
{
"Effect": "Allow",
"Action": [ "s3:ListObjects" ],
"Resource": [
"arn:aws:s3:::example-devops"
]
}
]
}