SQS - why limiting maximum message size?

Question

Is there any reason why I should a lower than maximum limit in Maximum message size in AWS SQS? I'm not able to find anyone good one...

Chalcography

There are always good reasons for rate limiting, which is what this is. Everything on the website you're looking at right now is rate-limited, and I do mean everything. – Spandrel

As long as your system will never have bugs and not exposes to internet you may be fine... also it's hard to see how at least later of these possible for Amazon's services. Otherwise think about single 1Tb message eating into all your other limits... (listing all possible issues with non-rate-limited service is too broad for SO) – Ophiuchus

in a SaaS service like SQS, the system most likely scales based on the maximum payload your messages are set to. Also, SQS bills a 64kb chunk as a request (IE: 256kb msg will be billed as 4 requests) so limiting your payloads can also give you tighter controls/estimations on your billing. Pre-Allocating storage nowadays is becoming a lost art (malloc anyone?) – Leesen

Thank you for your good comments, folks! – Chalcography

I want to confirm that if we set maximum message size to 256KB but my request size is under 64KB so I will be charged for 1 request or for 4 requests? @Leesen – Pocked

@truongnm, My understanding from aws.amazon.com/sqs/pricing/?nc1=h_ls is that you'll be charged for 1 request if the payload is under 64KB. In the doc they said : "for example, an API action with a 256 KB payload is billed as 4 requests". They talk about the "action" not the queue configuration so I conclude that the payload size determines how much you'll be charged. – Spinifex

Answer 1

SQS provides many pro's like bulk message send , delayed messages , polling etc. So since we have all these pro's they definitely need to limit their sizing. But how we handled was ,

• we check the message size and if the message size is above 256kb , we upload the message to s3 with unique id as file name and share the message in queue as { largeFile : true , id : (s3 File Name)} , now the consumer checks whether the largeFile is true , if so fetches from s3 and processes the data , simple :)

• Or if u want only queue go with other message brokers like rabbitmq , where there isn't any size limits.

Studied