Amazon S3 - Your proposed upload is smaller than the minimum allowed size

Question

I'm having problems when I want to upload an image to my amazon s3 bucket.

I'm trying to upload a jpg image with the size of 238 KB. I've put a try/catch in my code to check what the error was. I always get this error:

Your proposed upload is smaller than the minimum allowed size

I've also tried this with images from 1MB and 2MB, same error ... .

Here's my code:

<?php

// Include the SDK using the Composer autoloader
require 'AWSSDKforPHP/aws.phar';

use Aws\S3\S3Client;
use Aws\Common\Enum\Size;

$bucket = 'mybucketname';
$keyname = 'images';
$filename = 'thelinktomyimage';

// Instantiate the S3 client with your AWS credentials and desired AWS region
$client = S3Client::factory(array(
    'key'    => 'key',
    'secret' => 'secretkey',
));


// Create a new multipart upload and get the upload ID.
$response = $client->createMultipartUpload(array(
    'Bucket' => $bucket,
    'Key'    => $keyname
));

$uploadId = $response['UploadId'];

// 3. Upload the file in parts.
$file = fopen($filename, 'r');
$parts = array();
$partNumber = 1;
while (!feof($file)) {
    $result = $client->uploadPart(array(
        'Bucket'     => $bucket,
        'Key'        => $keyname,
        'UploadId'   => $uploadId,
        'PartNumber' => $partNumber,
        'Body'       => fread($file, 5 * 1024 * 1024),
    ));
    $parts[] = array(
        'PartNumber' => $partNumber++,
        'ETag'       => $result['ETag'],
    );

}

// Complete multipart upload.
try{
    $result = $client->completeMultipartUpload(array(
        'Bucket'   => $bucket,
        'Key'      => $keyname,
        'UploadId' => $uploadId,
        'Parts'    => $parts,
    ));
    $url = $result['Location'];

    fclose($file);
}
catch(Exception $e){
    var_dump($e->getMessage());
}

(I've changed the bucket, keys and image link.)
Has anyone had this before? Searching on the internet didn't help me much.
Also searching to change the minimum upload size didn't provide much help.

UPDATE:
When I tried this with a local image (changed the filename), it worked! How can I make this work with an image that's online? Now I save it in my temp file and then upload it from there. But isn't there a way to store it directly without saving it locally?

Licha

Answer 1

The minimal multipart upload size is 5Mb (1), even if your total size is 100MB, each individual multipart upload (other than the last one) can't be smaller than 5MB. You probably want to use a "normal" upload, not a multipart upload.

(1) http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadUploadPart.html

Pampas

The multipart upload API can upload objects of arbitrary size. The only requirement is that every "part" except the last must be at least 5MB, which is the problem here: fread() is returning a string that is less than 5MB in size. – Neighbor

Thanks, annoyingly it doesn't fail until the complete request is called. It also means I can't use this at all because the upload is generated and size isn't known up front. – Panegyrize

This answer is incorrect. As others have pointed out: the last part (or the only part) can in fact be any size. – Tempo

@Jesse's answer needs to be actually the accepted answer. – Malvie

Answer 2

This error occurs when one of the parts is less than 5MB in size and isn't the last part (the last part can be any size). fread() can return a string shorter than the specified size, so you need to keep calling fread() until you have at least 5MB of data (or you have reached the end of the file) before uploading that part.

So your 3rd step becomes:

// 3. Upload the file in parts.
$file = fopen($filename, 'r');
$parts = array();
$partNumber = 1;
while (!feof($file)) {

    // Get at least 5MB or reach end-of-file
    $data = '';
    $minSize = 5 * 1024 * 1024;
    while (!feof($file) && strlen($data) < $minSize) {
        $data .= fread($file, $minSize - strlen($data));
    }

    $result = $client->uploadPart(array(
        'Bucket'     => $bucket,
        'Key'        => $keyname,
        'UploadId'   => $uploadId,
        'PartNumber' => $partNumber,
        'Body'       => $data, // <= send our buffered part
    ));
    $parts[] = array(
        'PartNumber' => $partNumber++,
        'ETag'       => $result['ETag'],
    );
}

Neighbor

I had exactly the same issue, if you upload one part it can be any size, for multiple parts, 5mb is the minimum. So if you keep your internal buffer at least 5M, then it will work. Thank you @Neighbor – Malvie

Answer 3

I got this error when my part was big enough, but I missed to set size property of the request object.

UploadPartRequest uploadRequest = new UploadPartRequest()
        .withPartSize(actualPartSize)

Jacksmelt