urllib.error.URLError: <urlopen error [Errno 11002] getaddrinfo failed>？

Question

So, My code is only 4 lines. I am trying to connect to a website, what I am trying to do after that is irrelevant because the error arised without the other codes.

import urllib.request
from bs4 import BeautifulSoup 

html=urllib.request.urlopen('http://python-data.dr-chuck.net/known_by_Fikret.html').read()
soup=BeautifulSoup(html,'html.parser')

and the error(succinctly summarized one):

for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 11002] getaddrinfo failed
During handling of the above exception, another exception occurred:
urllib.error.URLError: <urlopen error [Errno 11002] getaddrinfo failed>

Here is what I have tried.

1. I searched for the Error returned "urlopen error [Errno 11002]” on google and especially on stackoverflow, nothing helpful was returned(In fact there are not much question on this error 11002 was asked).
2. So Then I try to replace the website argument(i.e. "http://python-data.dr-chuck.net/known_by_Fikret.html" ) inside the urlopen function with another website "http://www.pythonlearn.com/code/urllinks.py". And it worked just fine. No error comes up.
3. So I guess this error must have something to do with this particular website per se. The website is somewhat dynamic, by which I mean its content will alter and change into another wholly different thing. But I have no more knowledge than just to descibe what I saw.

and the longer and full version of error:

Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35-32\lib\urllib\request.py", line 1240, in do_open
h.request(req.get_method(), req.selector, req.data, headers)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35-32\lib\http\client.py", line 1083, in request
self._send_request(method, url, body, headers)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35-32\lib\http\client.py", line 1128, in _send_request
self.endheaders(body)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35-32\lib\http\client.py", line 1079, in endheaders
self._send_output(message_body)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35-32\lib\http\client.py", line 911, in _send_output
self.send(msg)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35-32\lib\http\client.py", line 854, in send
self.connect()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35-32\lib\http\client.py", line 826, in connect
(self.host,self.port), self.timeout, self.source_address)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35-32\lib\socket.py", line 693, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35-32\lib\socket.py", line 732, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 11002] getaddrinfo failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:/baiduyundownload/Tempo/Active/Python/Python Examples/Fileanalysis11111.py", line 4, in <module>
html=urllib.request.urlopen('http://python-data.dr-chuck.net/known_by_Fikret.html').read()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35-32\lib\urllib\request.py", line 162, in urlopen
return opener.open(url, data, timeout)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35-32\lib\urllib\request.py", line 465, in open
response = self._open(req, data)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35-32\lib\urllib\request.py", line 483, in _open
'_open', req)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35-32\lib\urllib\request.py", line 443, in _call_chain
result = func(*args)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35-32\lib\urllib\request.py", line 1268, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python35-32\lib\urllib\request.py", line 1242, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 11002] getaddrinfo failed>

Gabey

Answer 1

This means that either your DNS system is not working correctly, or you have to use a proxy on your network and it isn't defined correctly.

If you need to use a proxy, set the environment variable HTTP_PROXY (and optionally, HTTPS_PROXY) to the correct configuration for your network. The format is http://proxy.example.com:80; if your proxy needs a username and password, you should pass it in, like this: http://username:[email protected]:80.

For DNS issues, try looking up the domain from the command line. Open a command prompt and type nslookup python-data.dr-chuck.net and see if it returns to you an IP address.

Heisenberg

Problem solved. And I have to mention that I am in China, and the site I am trying to connect to is blocked by the infamous GFW. I open the lantern(one of the proxy software to bypass the censorship.) and add the environment variable as you suggest. And it worked, albeit in a somewhat lagging manner. Thought this might help other Chinese like me. – Gabey

I am getting the same Errno 11002(I am behind a corporate firewall) where running nslookup python-data.dr-chuck.net gives me DNS Server and Address followed by DNS Request time-out. What should i do? – Solifluction

@Solifluction you need to ask your IT admin to enable access for you through the firewall. – Heisenberg

The firewall has certain hostname and port, I have an ID and Password. I can access and download content through chrome, It is through command line that i cannot install any package. So what access specific is needed for CLI? – Solifluction

Answer 2

This is an issue with DNS; apparently your python program cannot resolve the hostname of the URL you provide. Maybe DNS is badly configured on the host it is running on?

Additionally, may I recommend using the requests library? This is a much nicer and easier to use module to do web requests. You'll find it here: https://pypi.python.org/pypi/requests

Oakie