AWS S3 ListMultipartUploads : access denied

Question

I have followed this blog in order to setup my AWS IAM and S3 accounts with Web Identity Federation. I am able to authenticate and receive session credentials and tokens all fine. I am also able to Download and Upload objects. However, I am getting:

access denied

on the following ListMultipartUploads request:

var request = new ListMultipartUploadsRequest()
{
    BucketName = bucketName,
    Prefix = $"{UserId}/"
};

var response = await s3Client.ListMultipartUploadsAsync(request);

The access policy attached to my IAM role is:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:AbortMultipartUpload",
                "s3:DeleteObject",
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Resource": "arn:aws:s3:::mybucket/${myidentityprovider:userId}/*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:ListBucketMultipartUploads"
            ],
            "Resource": [
                "arn:aws:s3:::mybucket"
            ],
            "Condition": {
                "StringLike": {
                    "s3:prefix": "${myidentityprovider:userId}/"
                }
            }
        }
    ]
}

As you can see, I have the permission "s3:ListBucketMultipartUploads", so the user should be able to perform ListMultiPartUploads on their buckets. What am I doing wrong?

Goldengoldenberg

Perhaps try the prefix without the ending slash? (Based on reading Allow a user to get a list of objects in a bucket according to a specific prefix) – Erlene

I tried it without the prefix slash but still it still returns access denied. – Goldengoldenberg

Does it work correctly if you remove the prefix condition? – Erlene

Yes it does, but obviously I want the user to only be able to list multipart uploads that they have done. – Goldengoldenberg

While I can successfully use s3:ListBucket with s3:prefix, all my experiments have failed when using s3:ListBucketMultipartUploads with s3:prefix -- and that's just by directly specifying the prefix rather than using ${myidentityprovider:userId}. The documentation suggests that it should work, but I can't achieve it. – Erlene

Hmmm. Could this be a bug? How can we move forward with this? – Goldengoldenberg

Somebody else might find a solution, otherwise you can raise a support case with AWS if you subscribe to Support. Otherwise, post it to their Forums but there's no guaranteed response there. – Erlene

You've probably already checked, but is there a bucket policy or an ACL that may be getting in the way? – Turbo

There's no bucket policy and I can't see any ACL's that would be causing issues. The identity provider is set up under my developer AWS IAM account, as per the blog linked in the question. – Goldengoldenberg

Enable cloudtrail and see if that gives you any more details? I found that that helps point me in the right direction when debugging API calls. – Dismantle

Ok, i'll give that a go and report back – Goldengoldenberg

Have you tried adding an asterisk on the end of your "StringLike" condition? Think of it like a SQL like where you have to add a '%' to match anything that starts with the prefix. I've seen lots of examples that show that pattern. See "Block 3" in this link: aws.amazon.com/blogs/security/… – Conservatism

Thanks for the idea. It produces the same result, unfortunately. – Goldengoldenberg

Cloud trail doesn't seem to be listing anything interesting, nothing regarding errors for list multi part etc – Goldengoldenberg

You might want to try the AWS policy simulator policysim.aws.amazon.com/home/index.jsp. It can be tricky to use, but has helped me enormously with problems such as this one. – Unvoice

I see that in your request the prefix is $"{UserId}/" while in your policy the prefix is "${myidentityprovider:userId}/" This might be the problem – Gautea

Answer 1

Per AWS documentation, s3Prefix is not a valid condition keys for ListBucketMultipartUploads. I'm running into the same issue as well, and it's unfortunate because when using Spark to write to S3 with the recommended s3a committers, this permission is required. Would love to see what others have come up with to workaround the issue.

Wanettawanfried

"when using Spark to write to S3, this permission is required. ". Really? I know s3a committers will use it to clean up, but this should be optional/recovered from. Have you got a stack trace? – Cannelloni

I think you're right. It's only required when s3a committers are used. I've updated my comment. Thanks for the correction! – Wanettawanfried

looking at the code, there's a way to turn off abort() of all pending uploads to a destination path, but on committer abort the active committer will still try to cancel everything. But if it is blocked, it should warn and continue. you just need to set up a rule to delete pending uploads after a few days. See issues.apache.org/jira/browse/HADOOP-17318 – Cannelloni

Answer 2

I see an error in your prefix statement,

It needs to be an array,

"s3:prefix": ["${myidentityprovider:userId}/*"]

{
"Version": "2012-10-17",
"Statement": [
    {
        "Effect": "Allow",
        "Action": [
            "s3:AbortMultipartUpload",
            "s3:DeleteObject",
            "s3:GetObject",
            "s3:PutObject"
        ],
        "Resource": "arn:aws:s3:::mybucket/${myidentityprovider:userId}/*"
    },
    {
        "Effect": "Allow",
        "Action": [
            "s3:ListBucket",
            "s3:ListBucketMultipartUploads"
        ],
        "Resource": [
            "arn:aws:s3:::mybucket"
        ],
        "Condition": {
            "StringLike": {
                "s3:prefix": ["${myidentityprovider:userId}/*"]
            }
        }
    }
]}

Amanuensis