How to solve the 10054 error

Question

I'm using an app provided by some website to collect some data from the website periodly, say, 30s a time. The returned response is then recorded in database.

I use the requests modular by import requests and write codes to catch Exception. The codes for the main function are as following:

def get_response(self):
    try:
       response = requests.get(self.request_url)
       if response.status_code == 200:
          return response.json()
       except Exception as e:
          msg = "Exception is:\n %s \n" % e
          print msg

The above function works quite well for the first several hours. The function can also recover from some exceptions like: ('Connection aborted.', BadStatusLine("''",))
or ('Connection aborted.', error(10053, '')) It omits the exception (by recording a Null in database) and continues to get the response of next period.

However, the function stops working when encoutering a 10054 error.

Exception is:
 ('Connection aborted.', error(10054, '')) 

I check the database to find that all the response is Null after the time that the 10054 error comes. I firstly guess that the website may breakdown, thus no response is received. But when I manually restart the function, it starts to get response again. So that's no realated with breakdown of the website.

I search in stackoverflow and find: Errno 10054 An existing connection was forcibly closed by the remote host. But I don't know how to resolve it.

Could you please provide some solotion to this problem?(ideally speaking) or provide some solution to restart the function without manually restarting? (It looks like once I restart the funtion and it works again.)

Thanks in advance.

Traveler

Did you solve that problem? – Niacin

Answer 1

The web server actively rejected your connection. That's usually because it is congested, has rate limiting or thinks that you are launching a denial of service attack. If you get this from a server, you should sleep a bit before trying again. In fact, if you don't sleep before retry, you are a denial of service attack. The polite thing to do is implement a progressive sleep of, say, (1,2,4,8,16,32) seconds.

Tetramethyldiarsine

Thanks for your thorough explaination. – Traveler

Yet your answer remains one question: Why it works when I manually restart the program even if the time gap is small between the time that the function automally start an access request and the time I manually restart the function after that? Thanks. – Traveler

Answer 2

You can try it. It solved my problem with ConnectionResetError 10054.

session = requests.Session()
session.headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.1.2222.33 Safari/537.36",
    "Accept-Encoding": "*",
    "Connection": "keep-alive"
}
response = session.get(self.request_url)

Santasantacruz

How does one determine their most appropriate user-agent? – Uncrown

Answer 3

Error 10054 occur when the API used extremely so the Host refuse to connect try after some interval it may be solved or by checking the API stat it shows a large number.

Blackthorn

Answer 4

The error message you're seeing indicates that the connection to a remote host was abruptly terminated by that host. This could be due to various reasons:

Network Issues: It's possible that there are network connectivity issues between your machine and the remote host, leading to the connection being forcibly closed. Firewall or Security Software: Sometimes, firewall settings or security software on either your machine or the remote host could interfere with the connection and close it unexpectedly. Server Overload: If the remote host is experiencing a high volume of traffic or is overloaded, it might close connections to free up resources. Intentional Closure: The remote host may have intentionally closed the connection due to some policy or configuration reasons.

Timbering