Correct S3 Policy For Pre-Signed URLs

Question

I need to issue pre-signed URLs for allowing users to GET and PUT files into a specific S3 bucket. I created an IAM user and use its keys to create the pre-signed URLs, and added a custom policy embedded in that user (see below). When I use the generated URL, I get an AccessDenied error with my policy. If I add the FullS3Access policy to the IAM user, the file can be GET or PUT with the same URL, so obviously, my custom policy is lacking. What is wrong with it?

Here's the custom policy I am using that is not working:

{
    "Statement": [
        {
            "Action": [
                "s3:ListBucket"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::MyBucket"
            ]
        },
        {
            "Action": [
                "s3:AbortMultipartUpload",
                "s3:CreateBucket",
                "s3:DeleteBucket",
                "s3:DeleteBucketPolicy",
                "s3:DeleteObject",
                "s3:GetBucketPolicy",
                "s3:GetLifecycleConfiguration",
                "s3:GetObject",
                "s3:ListBucket",
                "s3:ListBucketMultipartUploads",
                "s3:ListMultipartUploadParts",
                "s3:PutBucketPolicy",
                "s3:PutLifecycleConfiguration",
                "s3:PutObject"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::MyBucket/*"
            ]
        }
    ]
}

Follett

I'm having what appears to be the same problem. Did you find a solution? – Sharla

might be helpful: radishlogic.com/aws/s3/… – Sarcous

Answer 1

Here is an IAM policy that works for my presigned S3 URLs.

{
    "Version": "2012-10-17",
    "Statement": [
      {
        "Sid": "VisualEditor0",
        "Effect": "Allow",
        "Action": [
            "s3:PutObject",
            "s3:GetObject"
        ],
        "Resource": "arn:aws:s3:::mydocs/*"
      }
    ]
}

I wonder if your problem is in the Resource part. Were your GET requests for bucket MyBucket always?

Addendum

Mine was the resources part. I was using multiple buckets that I forgot about until you mentioned the resources. I was only allowing one bucket. – Finbur

Just s3:GetObject should be sufficient – Homopolar

@Homopolar A pre-signed S3 URL can be used for writing an object to an S3 bucket too. So I included s3:PutObject as well. – Addendum

@BigPumpkin that's a good point, thanks! – Homopolar

I didn't occur to me that a /* was needed at the end of the "Resource" property. Thanks a lot! – Iminourea

Answer 2

I was also working on a feature that used presigned GET and put URLs, specifically by a role associated with an AWS Lambda function. I found a bit of a twist though in that I also needed to also allow permission to use the KMS key that was encrypting the bucket.

I ran across this off-the-beaten-path article the pointed me in the right direction. It's not necessary to allow bucket-level permissions for URL presigning, only a handful of object-level permissions.

In short, my lambda role policy to support presigned URLs looked like the following. Note that the cloudwatch log permission is irrelevant for signing, but generally important for lambda functions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "arn:aws:logs:*:*:*",
      "Effect": "Allow"
    },
    {
      "Action": [
        "s3:GetObject",
        "s3:PutObject"
      ],
      "Resource": "<my-bucket-arn-expression>/*",
      "Effect": "Allow"
    },
    {
      "Sid": "KMSAccess",
      "Action": [
        "kms:Decrypt",
        "kms:DescribeKey",
        "kms:Encrypt",
        "kms:GenerateDataKey*",
        "kms:ReEncrypt*"
      ],
      "Effect": "Allow",
      "Resource": "<my-key-arn>"
    }
  ]
}

If you are using the built-in AES encryption (or no encryption), your policy can be simplified to this:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "arn:aws:logs:*:*:*",
      "Effect": "Allow"
    },
    {
      "Action": [
        "s3:GetObject",
        "s3:PutObject"
      ],
      "Resource": "<my-bucket-arn-expression>/*",
      "Effect": "Allow"
    }
  ]
}

Numinous

You are a legend! Thank you! – Radiology

Fantastic! The kms actions were what I needed. – Albright

Thank you! What's strange is that without the s3 permissions for the lambda role, the presignedUrl is still created (no error) but can't be used. – Usufruct

That's interesting @Usufruct - off the top of my head, I think it's because the signing process is generally client-side: docs.aws.amazon.com/IAM/latest/UserGuide/… If you were to compute a presigned URL without the permissions, then add the permissions, would the same URL then work? – Numinous

Answer 3

Bucket Permissions vs Object Permissions

The following permissions from your policy should be at the Bucket level (arn:aws:s3:::MyBucket), rather than a sub-path within the Bucket (eg arn:aws:s3:::MyBucket/*):

• s3:CreateBucket
• s3:DeleteBucket
• s3:DeleteBucketPolicy
• s3:GetBucketPolicy
• s3:GetLifecycleConfiguration
• s3:ListBucket
• s3:ListBucketMultipartUploads
• s3:PutBucketPolicy
• s3:PutLifecycleConfiguration

See: Specifying Permissions in a Policy

However, that is not the cause of your inability to PUT or GET files.

GET

The fact that your have assigned GetObject permissions means that you should be able to GET an object from the S3 bucket. I tested this by assigning your policy to a User, then using that User's credentials to access an object and it worked correctly.

PUT

I also used your policy to upload via a web form and it worked correctly.

Here is the form I used to upload:

<html><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
    <title>S3 POST Form</title> 

  <style type="text/css"></style></head>

  <body> 
    <form action="https://<BUCKET-NAME>.s3.amazonaws.com/" method="post" enctype="multipart/form-data">
      <input type="hidden" name="key" value="uploads/${filename}">
      <input type="hidden" name="AWSAccessKeyId" value="<ACCESS-KEY>">
      <input type="hidden" name="acl" value="private"> 
      <input type="hidden" name="success_action_redirect" value="http://<BUCKET-NAME>.s3.amazonaws.com/ok.html">
      <input type="hidden" name="policy" value="<ENCODED-POLICY>">
      <input type="hidden" name="signature" value="<SIGNATURE>">
      <input type="hidden" name="Content-Type" value="image/jpeg">
      <!-- Include any additional input fields here -->

      File to upload to S3: 
      <input name="file" type="file"> 
      <br> 
      <input type="submit" value="Upload File to S3"> 
    </form> 

Here is how I generated the Signature:

#!/usr/bin/python
import base64
import hmac, hashlib

policy_document = '{"expiration": "2018-01-01T00:00:00Z", "conditions": [ {"bucket": "<BUCKET-NAME>"}, ["starts-with", "$key", "uploads/"], {"acl": "private"}, {"success_action_redirect": "http://BUCKET-NAME.s3.amazonaws.com/ok.html"}, ["starts-with", "$Content-Type", ""], ["content-length-range", 0, 1048000] ] }'

AWS_SECRET_ACCESS_KEY = "<SECRET-KEY>"

policy = base64.b64encode(policy_document)

signature = base64.b64encode(hmac.new(AWS_SECRET_ACCESS_KEY, policy, hashlib.sha1).digest())

print policy
print
print signature

Jacki

Answer 4

After messing with IAM permissions for about a week, this worked. My goal was to create a presigned_url to read an S3 image (and not expire until the max 7 days).

KMS and S3 are needed.

STS may not be needed but I was messing with the "assume_role" function too.

{
"Version": "2012-10-17",
"Statement": [
    {
        "Sid": "VisualEditor0",
        "Effect": "Allow",
        "Action": [
            "s3:PutObject",
            "s3:GetObject",
            "kms:Decrypt",
            "kms:Encrypt",
            "kms:DescribeKey",
            "kms:ReEncrypt*",
            "kms:GenerateDataKey*"
        ],
        "Resource": [
            "arn:aws:kms:*:<account-number>:key/*",
            "arn:aws:s3:::<bucket-name>/*"
        ]
    },
    {
        "Sid": "VisualEditor1",
        "Effect": "Allow",
        "Action": [
            "sts:GetSessionToken",
            "sts:DecodeAuthorizationMessage",
            "sts:GetAccessKeyInfo",
            "sts:GetCallerIdentity",
            "sts:GetServiceBearerToken"
        ],
        "Resource": "*"
    },
    {
        "Sid": "VisualEditor2",
        "Effect": "Allow",
        "Action": "sts:*",
        "Resource": [
            "arn:aws:iam::<account-number>:<role-arn>",
            "arn:aws:iam::<account-number>:user/<aws-user-name>"
        ]
    }
]

}

here's the function that uses this user credentials

from botocore.config import Config
my_config = Config(
    region_name = 'us-east-2',
    signature_version = 's3v4',
    s3={'addressing_style': 'path'},
)

client = boto3.client('s3', config=my_config,
aws_access_key_id = AWS_ACCESS_KEY_ID,
aws_secret_access_key = AWS_SECRET_ACCESS_KEY
)
presigned_url = client.generate_presigned_url(
    'get_object',
    Params={'Bucket': bucket_name, 'Key': key_name},
    ExpiresIn=604800,
    HttpMethod=None
)

Vinery

This might be not the answer for the question but it gets the work done if all someone need is to have a long lasting presigned url. – Gillenwater

Answer 5

What worked for me is this:

{
    "Action": [
       "s3:PutObject"
    ],
    "Resource": [
       "arn:aws:s3:::<your-bucket-name>/*"
    ],
    "Effect": "Allow"
}

Note: I did not mention other polices here for ex: permissions for CloudWatch logs or XRay.

Sandman

it should be "arn:aws:s3:::<your-bucket-name>/*", note the /* – Homopolar

Answer 6

If you use a custom kms key to encrypt your bucket, you have to ensure that the key policy allows your user / role that you use to generate the presigned-url.
If that identity is not granted the following permissions in the key policy you will also receive an access denied error when trying to download via the presigned-url.

...
"Action": [
  "kms:ReEncrypt*",
  "kms:GenerateDataKeyWithoutPlaintext",
  "kms:GenerateDataKey",
  "kms:Encrypt",
  "kms:DescribeKey",
  "kms:Decrypt"
],
...

Vacla