How to handle request timeout(408) error in boto?

Question

We are using domain.select() method, that boto provides, to query SimpleDB. For smaller queries(queries involving couple of hours of data), this method works fine. But when I start using multiple threads and longer queries(24 hours of data), it starts timing out, giving following error on stdout:

-------------------------
         4 0 8
...
<?xml version="1.0"?>
<Response><Errors><Error><Code>QueryTimeout</Code><Message>A timeout occurred when attempting to query domain 'd110824' with query expression 'select * from `d110824` where `timestamp` &gt;= '2011-08-24T10:45:56' and `timestamp` &lt; '2011-08-25T10:45:56' and `identifier` = '00063F052C49' order by `timestamp` asc </Message><BoxUsage>0.0055590278</BoxUsage></Error></Errors><RequestID>....</RequestID></Response>

I want to implement a retry mechanism (exponential backoff), when this error is encountered. Boto doesn't throw any exception for this error and simply prints it. To implement a retry mechanism, I need some kind of error code or exception to know that the error has occurred.

Any thoughts on how to achieve this in boto?

Stillage

Answer 1

Boto will retry on a 503, but not on a 408.

There are several things that will make boto retry, including a 503 (service not available), and some types of HTTP errors when trying to connect. It will use exponential backoff, and try up to 5 times by default. You can change the number of retries by setting num_retries in the .boto config file:

[Boto]
num_retries = 3

I don't know why it doesn't retry on a 408. The AWS docs I've seen recommend doing so.

Myrtice

thanks for your reply. Yeah, it does retry on 503, but there is no way to find out, what it does with 408, it doesn't throw any exception as well... boto documentation sucks man. – Stillage

Boto doesn't retry on a 408 response because we were requested, several years ago, not to by AWS. If the AWS documentation now states that it should be retried, then boto should be changed to retry it. I would suggest filing a ticket on github and I will take care of it personally. – Todtoday

BTW, Sujit, any chance you could be more specific about the problems with boto docs? I'm always happy to get suggestions for improvement from the community. Thanks! – Todtoday

Also, could you provide a link to the AWS documentation that says a 408 should be retried? My understanding was that a 408 meant your query was too complex and you should change it before retrying, hence the 400 response (Client Error) rather than a 500 response (Server Error). – Todtoday

Answer 2

I had the same problem with boto and ended up forking it and adding an exponential backoff directly. See https://github.com/datacratic/boto/blob/develop/boto/sdb/queryresultset.py#L83.

Payment