Here's a comprehensive batch solution that copies files from one folder to another using a single call of CommandPool::batch, although under the hood it runs a executeAsync command for each file, so not sure it counts as a single API call.
As I understand you should be able to copy hundreds of thousands of files using this method as there's no way to send a batch to AWS to be processed there, but if you're hosting this at an AWS instance or even running it on Lambda then its "technically" processed at AWS.
Install the SDK:
composer require aws/aws-sdk-php
use Aws\ResultInterface;
use Aws\S3\S3Client;
use Aws\S3\Exception\S3Exception;
use Aws\S3\Exception\DeleteMultipleObjectsException;
$bucket = 'my-bucket-name';
// Setup your credentials in the .aws folder
// See: https://docs.aws.amazon.com/sdk-for-php/v3/developer-guide/guide_credentials_profiles.html
$s3 = new S3Client([
'profile' => 'default',
'region' => 'us-east-2',
'version' => 'latest'
]);
// Get all files in S3
$files = array();
try {
$results = $s3->getPaginator('ListObjects', [
'Bucket' => $bucket,
'Prefix' => 'existing-folder' // Folder within bucket, or remove this to get all files in the bucket
]);
foreach ($results as $result) {
foreach ($result['Contents'] as $object) {
$files[] = $object['Key'];
}
}
} catch (AwsException $e) {
error_log($e->getMessage());
}
if(count($files) > 0){
// Perform a batch of CopyObject operations.
$batch = [];
foreach ($files as $file) {
$batch[] = $s3->getCommand('CopyObject', array(
'Bucket' => $bucket,
'Key' => str_replace('existing-folder/', 'new-folder/', $file),
'CopySource' => $bucket . '/' . $file,
));
}
try {
$results = CommandPool::batch($s3, $batch);
// Check if all files were copied in order to safely delete the old directory
$count = 0;
foreach($results as $result) {
if ($result instanceof ResultInterface) {
$count++;
}
if ($result instanceof AwsException) {
}
}
if($count === count($files)){
// Delete old directory
try {
$s3->deleteMatchingObjects(
$bucket, // Bucket
'existing-folder' // Prefix, folder within bucket, as indicated above
);
} catch (DeleteMultipleObjectsException $exception) {
return false;
}
return true;
}
return false;
} catch (AwsException $e) {
return $e->getMessage();
}
}
