Why is it so much slower to delete objects in AWS S3 than it is to create them?

Question

I have an AWS lambda function that watches an S3 bucket. When an image is uploaded to the bucket the lambda function creates a thumbnail of the image. However I made a mistake in the function and saved the transformed file in the same directory that was being watched, creating an infinite loop.

It ran for roughly a minute before I stopped it, and in that time created 600,000 images. However, deleting those images (using the AWS console) took about 20 minutes.

Why such a difference? I'm interested in the low-level reasons for this.

Sumptuary

The simple explanation is that the delete is a much more expensive db operation as the engine must first locate the row, then remove it. While an insert (or bulk insert with 600,000 rows) operation does not require the seek. A thorough explanation would probably require AWS developers at Amazon to explain the back end implementation at the CPU level. Perhaps someone has that knowledge and is willing to share? – Stichomythia

@JacobH that's the kind of level I'm after, thanks. I'll remove "implementation" from the question, as it may be a more general reason like the one you give rather than something specific to AWS. – Sumptuary

You should at least try to compare apples to apples here and script the delete function instead of using the AWS console. It could be that the console is simply inefficient at this type of operation. – Evangelical

Answer 1

According to this documentation https://docs.aws.amazon.com/AmazonS3/latest/dev/delete-or-empty-bucket.html deletion is an "insertion of a deletion marker". So it might explain why a deletion is so slow compared to an upload which provides only an eventual consistency.

Nisa

Answer 2

Amazon S3 buckets in all Regions provide read-after-write consistency for PUTS of new objects and eventual consistency for overwrite PUTS and DELETES.

According to the above, it should be the opposite you experience in terms of object creation and deletion performance.

However the main cause I assume is due to, using the AWS Web Console, where the "delete" action can take more time for Web Console to reflect. In contrast, if you run the delete operation using AWS CLI, it should be the opposite.

Bencher

Given a +1 because this is totaly true even if the observed behavior is not the one described. It should be true as far as we can understand AWS implementation not going under the hood ... – Nisa

That hasn't been my experience. DELETES are slower than PUTS, at least for small files. – Pulchritude