Upload & fetch media files from AWS S3 in Flutter
Asked Answered
P

4

12

My flutter app is using firebase as a backend but I need to store media files (photos & videos) in my s3 bucket. The mission is to upload the media retrieved from the image picker into s3 & get back the url, which can then be stored as a string in my firebase database.

The problem is a scarcity of aws libraries or api for dart 2. I found 3 in pub, but 2 of them were incompatible with dart 2 & 1 was under development. Has anyone implemented this in flutter using dart 2? Any suggestions are welcome. Thank you.

The packages I found were (pub.dartlang.org) : aws_client, aws_interop, amazon_s3

Prochoras answered 17/12, 2018 at 8:39 Comment(1)
What are the library you have already found? Could you post them so that people doesn't try to find out by themselves, and to avoid them to suggest something you have already seen.Rimester
M
21

There's a few ways to do this. One way is to sign your request with Signature V4 and POST your file to S3.

First, create a policy helper:

import 'dart:convert';
import 'package:amazon_cognito_identity_dart/sig_v4.dart';

class Policy {
  String expiration;
  String region;
  String bucket;
  String key;
  String credential;
  String datetime;
  int maxFileSize;

  Policy(this.key, this.bucket, this.datetime, this.expiration, this.credential,
      this.maxFileSize,
      {this.region = 'us-east-1'});

  factory Policy.fromS3PresignedPost(
    String key,
    String bucket,
    String accessKeyId,
    int expiryMinutes,
    int maxFileSize, {
    String region,
  }) {
    final datetime = SigV4.generateDatetime();
    final expiration = (DateTime.now())
        .add(Duration(minutes: expiryMinutes))
        .toUtc()
        .toString()
        .split(' ')
        .join('T');
    final cred =
        '$accessKeyId/${SigV4.buildCredentialScope(datetime, region, 's3')}';
    final p = Policy(key, bucket, datetime, expiration, cred, maxFileSize,
        region: region);
    return p;
  }

  String encode() {
    final bytes = utf8.encode(toString());
    return base64.encode(bytes);
  }

  @override
  String toString() {
    return '''
{ "expiration": "${this.expiration}",
  "conditions": [
    {"bucket": "${this.bucket}"},
    ["starts-with", "\$key", "${this.key}"],
    {"acl": "public-read"},
    ["content-length-range", 1, ${this.maxFileSize}],
    {"x-amz-credential": "${this.credential}"},
    {"x-amz-algorithm": "AWS4-HMAC-SHA256"},
    {"x-amz-date": "${this.datetime}" }
  ]
}
''';
  }
}

Then, sign your request with your policy helper and upload via http.MultipartRequest:

import 'dart:convert';
import 'dart:io';
import 'package:path/path.dart' as path;
import 'package:async/async.dart';
import 'package:http/http.dart' as http;
import 'package:test/test.dart';
import 'package:amazon_cognito_identity_dart/sig_v4.dart';
import './policy.dart';

void main() {
  const _accessKeyId = 'AKXXXXXXXXXXXXXXXXXX';
  const _secretKeyId = 'xxxxxxxxxxxxxxxxxxxxxxxx/xxxxxxxxxxxxxxx';
  const _region = 'ap-southeast-1';
  const _s3Endpoint =
      'https://bucketname.s3-ap-southeast-1.amazonaws.com';

  final file = File(path.join('/path/to/file', 'square-cinnamon.jpg'));
  final stream = http.ByteStream(DelegatingStream.typed(file.openRead()));
  final length = await file.length();

  final uri = Uri.parse(_s3Endpoint);
  final req = http.MultipartRequest("POST", uri);
  final multipartFile = http.MultipartFile('file', stream, length,
      filename: path.basename(file.path));

  final policy = Policy.fromS3PresignedPost('uploaded/square-cinnamon.jpg',
      'bucketname', _accessKeyId, 15, length,
      region: _region);
  final key =
      SigV4.calculateSigningKey(_secretKeyId, policy.datetime, _region, 's3');
  final signature = SigV4.calculateSignature(key, policy.encode());

  req.files.add(multipartFile);
  req.fields['key'] = policy.key;
  req.fields['acl'] = 'public-read';
  req.fields['X-Amz-Credential'] = policy.credential;
  req.fields['X-Amz-Algorithm'] = 'AWS4-HMAC-SHA256';
  req.fields['X-Amz-Date'] = policy.datetime;
  req.fields['Policy'] = policy.encode();
  req.fields['X-Amz-Signature'] = signature;

  try {
    final res = await req.send();
    await for (var value in res.stream.transform(utf8.decoder)) {
      print(value);
    }
  } catch (e) {
    print(e.toString());
  }
}

Please note that this method requires you to provide your Access Key and Secret Key. If you use a service like Cognito, it is recommended to get a temporary Access Key and Secret Key. Examples using temporary access found here.

Disclaimer: I am the original author of the Signature V4 package.

Mandarin answered 4/3, 2019 at 13:0 Comment(8)
Getting Method Not Allowed error: The specified method is not allowed against this resource.Tyndale
You should be replace "POST" by "PUT" method final req = http.MultipartRequest("PUT", uri);Lichee
Getting some Error : The XML you provided was not well-formed or did not validate against our published schemaIntense
Can we use Signature V24 for object deletion as well. Will you be able to share the exampleWoolworth
I'm able to send contentType alongKelsi
Thanks, This solved the issues I was having with the awsS3 Plugin that does not support Flutter WEB. This solution should be marked as the correct answer.Glazer
Was also having issues with other S3 plugins, but this worked. To avoid having to copy paste this across projects, I wrapped it in a little package pub.dev/packages/aws_s3_upload. Hope that's alright!Tempered
Hey, earlier this code was working fine for me but after update flutter sdk to 3.3 i am getting Failed host lookup: 'bucketname.s3-us-east-1.amazonaws.com'Allineallis
I
5

You can use package amazon_s3_cognito to upload and delete the images to amazon s3.

I am the author of the plugin and we are using this plugin successfully in many of our projects.

        import 'package:amazon_s3_cognito/amazon_s3_cognito.dart';
        import 'package:amazon_s3_cognito/aws_region.dart';

        String uploadedImageUrl = await AmazonS3Cognito.uploadImage(
                  _image.path, BUCKET_NAME, IDENTITY_POOL_ID);


        //Use the below code to upload an image to amazon s3 server
        //I advise using this method for image upload.
        String uploadedImageUrl = await AmazonS3Cognito.upload(
                    _image.path,
                    BUCKET_NAME,
                    IDENTITY_POOL_ID,
                    IMAGE_NAME,
                    AwsRegion.US_EAST_1,
                    AwsRegion.AP_SOUTHEAST_1)

_image.path = path of the image you want to upload (file.path method in flutter)

IMAGE_NAME = this image uploads to s3 server with the name you give here.

        //use below code to delete an image
         String result = AmazonS3Cognito.delete(
                    BUCKET_NAME,
                    IDENTITY_POOL_ID,
                    IMAGE_NAME,
                    AwsRegion.US_EAST_1,
                    AwsRegion.AP_SOUTHEAST_1)

For fetching images you could use cached_network_image package
The CachedNetworkImage can be used directly or through the ImageProvider.

    CachedNetworkImage(
            imageUrl: "http://via.placeholder.com/350x150",
            placeholder: (context, url) => new CircularProgressIndicator(),
            errorWidget: (context, url, error) => new Icon(Icons.error),
         ),`enter code here`


    Future<String> _getFilePath(Asset asset,ListingImage listingImage) async{
       try {
         if (!isUploadCancelled) {
           // getting a directory path for saving
           final directory = await getTemporaryDirectory();
           String path = directory.path;
           File file = File(path + "/temp_" + listingImage.index.toString() + "_"+DateTime.now().microsecondsSinceEpoch.toString());
           listingImage.file = file;
           file = await file.writeAsBytes( asset.imageData.buffer.asUint8List(asset.imageData.offsetInBytes, asset.imageData.lengthInBytes));

           return file.path;
         } else {
           return null;
         }
       } catch(exceptioon) {
         return null;
       }

      }
Intellectualism answered 24/8, 2019 at 9:17 Comment(8)
Wow this looks like a cool plugin. I've never worked with identity pools or aws cognito as I am more familiar with using access key id & secret access key in Node Js. Will try this out...Prochoras
what is IDENTITY_POOL_ID? is it awsSecretAccess or awsAccessKey?Gerrald
Amazon Cognito identity pools provide temporary AWS credentials for users who are guests (unauthenticated) and for users who have been authenticated and received a token. An identity pool is a store of user identity data specific to your account. Please refer here docs.aws.amazon.com/cognito/latest/developerguide/…. Also below link could be useful studytutorial.in/…Intellectualism
Will this work with private s3 buckets with access key and secret key?Tyndale
This is for anyone else that finds this. I had a heck of a time getting it working. Turns out the easiest way to get this work is to create your identity_pool and your s3 bucket on US_EAST_1. I looked thru the android parts this plugin wraps and it appears that EAST was hardcoded. Second for the subregion you can just put US_EAST_1 again. Works like a charm if you stick to those rules.Zoosporangium
Please notice that your plugin is not working with private buckets on iOS image upload.Sisk
This plugin is not working for iOS even if the bucket is public, I have wasted lot of time implementing this.Kelsi
I want to upload json files... how I can I do it with this package?Coryza
N
1

You can use Flutter Multipart, something like this

 // open a bytestream
    var stream = new http.ByteStream(DelegatingStream.typed(_image.openRead()));
    // get file length
    var length = await _image.length();
    // string to uri
    var uri = Uri.parse(apiUrl);
    // create multipart request
    var request = new http.MultipartRequest("POST", uri);
    NetworkUtils.addAuthHeaders(request);
    // multipart that takes file
    var multipartFile = new http.MultipartFile('file', stream, length,
        filename: basename(_image.path),
        contentType: new MediaType("image", "jpg"));
    // add file to multipart
    request.files.add(multipartFile);
    request.fields.addAll(body);
    // send
    var response = await request.send();
    print(response.statusCode);
    // listen for response
    response.stream.transform(utf8.decoder).listen((value) {
      print(value);
    });
  }
Negligee answered 17/12, 2018 at 8:44 Comment(0)
B
1

You can use package amazon_s3_cognito to upload images. But it has issues on the iOS side, you can try my fork.

Put this line in your pubspec.yaml:

amazon_s3_cognito:
  git:
    URL: https://github.com/zhihaozhang/amazon_s3_cognito.git
Bren answered 4/8, 2021 at 8:24 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.