Can IAM role temporary credentials be used in cloudformation templates?

Question

I'm building a stack that needs access to a private S3 bucket to download the most current version of my application. I'm using IAM roles, a relatively new AWS feature that allows EC2 instances to be assigned specific roles, which are then coupled with IAM policies. Unfortunately, these roles come with temporary API credentials generated at instantiation. It's not crippling, but it's forced me to do things like this cloud-init script (simplified to just the relevant bit):

#!/bin/sh

# Grab our credentials from the meta-data and parse the response
CREDENTIALS=$(curl -s http://169.254.169.254/latest/meta-data/iam/security-credentials/s3access)
S3_ACCESS_KEY=$(echo $CREDENTIALS | ruby -e "require 'rubygems'; require 'json'; puts JSON[STDIN.read]['AccessKeyId'];")
S3_SECRET_KEY=$(echo $CREDENTIALS | ruby -e "require 'rubygems'; require 'json'; puts JSON[STDIN.read]['SecretAccessKey'];")
S3_TOKEN=$(echo $CREDENTIALS | ruby -e "require 'rubygems'; require 'json'; puts JSON[STDIN.read]['Token'];")

# Create an executable script to pull the file
cat << EOF > /tmp/pullS3.rb
require 'rubygems'
require 'aws-sdk'
AWS.config(
    :access_key_id     => "$S3_ACCESS_KEY",
    :secret_access_key => "$S3_SECRET_KEY",
    :session_token     => "$S3_TOKEN")
s3 = AWS::S3.new()
myfile = s3.buckets['mybucket'].objects["path/to/my/file"]
File.open("/path/to/save/myfile", "w") do |f|
    f.write(myfile.read)
end
EOF

# Downloading the file
ruby /tmp/pullS3.rb

First and foremost: This works, and works pretty well. All the same, I'd love to use CloudFormation's existing support for source access. Specifically, cfn-init supports the use of authentication resources to get at protected data, including S3 buckets. Is there anyway to get at these keys from within cfn-init, or perhaps tie the IAM role to an authentication resource?

I suppose one alternative would be putting my source behind some other authenticated service, but that's not a viable option at this time.

Another promising lead is the AWS::IAM::AccessKey resource, but the docs don't suggest it can be used with roles. I'm going to try it anyway.

Chiba

boto, a popular python AWS library, handles this elegantly. See this answer for more details. – Chiba

Answer 1

I'm not sure when support was added, but you can meanwhile use an IAM role for authenticating S3 downloads for files and sources sections in AWS::CloudFormation::Init.

Just use roleName instead of accessKeyId & secretKey (see AWS::CloudFormation::Authentication for details), e.g.:

"Metadata": {
    "AWS::CloudFormation::Init": {
        "download": {
            "files": {
                "/tmp/test.txt": {
                    "source": "http://myBucket.s3.amazonaws.com/test.txt"
                }
             }
        }
    },
    "AWS::CloudFormation::Authentication": {
        "default" : {
            "type": "s3",
            "buckets": [ "myBucket" ],
            "roleName": { "Ref": "myRole" }
        }
    }
}

Tested with aws-cfn-bootstrap-1.3-11

Reconcile

I fail to see how this is the answer. You need to in addition, attach the role to the ec2 instance itself, which grants it permanent access to bucket. – Essa

Answer 2

I managed to get this working. What I used was code from this exchange: https://forums.aws.amazon.com/message.jspa?messageID=319465

The code doesn't use IAM Policies - it uses the AWS::S3::BucketPolicy instead.

Cloud formation code snippet:

"Resources" : {     

"CfnUser" : {
  "Type" : "AWS::IAM::User",
  "Properties" : {
    "Path": "/",
    "Policies": [{
      "PolicyName": "root",
      "PolicyDocument": { "Statement":[{
        "Effect"   : "Allow",
        "Action"   : [
          "cloudformation:DescribeStackResource",
          "s3:GetObject"
        ],
        "Resource" :"*"
      }]}
    }]
  }
},

"CfnKeys" : {
  "Type" : "AWS::IAM::AccessKey",
  "Properties" : {
    "UserName" : {"Ref": "CfnUser"}
  }
},

"BucketPolicy" : {
  "Type" : "AWS::S3::BucketPolicy",
  "Properties" : {
    "PolicyDocument": {
      "Version"      : "2008-10-17",
      "Id"           : "CfAccessPolicy",
      "Statement"    : [{
        "Sid"        : "ReadAccess",
        "Action"     : ["s3:GetObject"],
        "Effect"     : "Allow",
        "Resource"   : { "Fn::Join" : ["", ["arn:aws:s3:::<MY_BUCKET>/*"]]},
        "Principal"  : { "AWS": {"Fn::GetAtt" : ["CfnUser", "Arn"]} }
      }]
    },
    "Bucket" : "<MY_BUCKET>"
  }
},

"WebServer": {  
  "Type": "AWS::EC2::Instance",
  "DependsOn" : "BucketPolicy",
  "Metadata" : {

    "AWS::CloudFormation::Init" : {
      "config" : {

        "sources" : {
          "/etc/<MY_PATH>" : "https://s3.amazonaws.com/<MY_BUCKET>/<MY_FILE>"
        }

      }
    },

    "AWS::CloudFormation::Authentication" : {
      "S3AccessCreds" : {
        "type" : "S3",
        "accessKeyId" : { "Ref" : "CfnKeys" },
        "secretKey" : {"Fn::GetAtt": ["CfnKeys", "SecretAccessKey"]},
        "buckets" : [ "<MY_BUCKET>" ]
      }
    }
  },

  "Properties": {
    "ImageId" : "<MY_INSTANCE_ID>", 
    "InstanceType" : { "Ref" : "WebServerInstanceType" },
    "KeyName" : {"Ref": "KeyName"},
    "SecurityGroups" : [ "<MY_SECURITY_GROUP>" ],

    "UserData"       : { "Fn::Base64" : { "Fn::Join" : ["", [
      "#!/bin/bash\n",

      "# Helper function\n",
      "function error_exit\n",
      "{\n",
      "  cfn-signal -e 1 -r \"$1\" '", { "Ref" : "WaitHandle" }, "'\n",
      "  exit 1\n",
      "}\n",

      "# Install Webserver Packages etc \n",
      "cfn-init -v --region ", { "Ref" : "AWS::Region" },
      "    -s ", { "Ref" : "AWS::StackName" }, " -r WebServer ",
      "         --access-key ", { "Ref" : "CfnKeys" },
      "         --secret-key ", {"Fn::GetAtt": ["CfnKeys", "SecretAccessKey"]}, " || error_exit 'Failed to run cfn-init'\n",

      "# All is well so signal success\n",
      "cfn-signal -e 0 -r \"Setup complete\" '", { "Ref" : "WaitHandle" }, "'\n"

    ]]}}        
  }
}

Obviously replacing MY_BUCKET, MY_FILE, MY_INSTANCE_ID, MY_SECURITY_GROUP with your values.

Np