Why can't I get Python's urlopen() method to work on Windows?

Question

Why isn't this simple Python code working?

import urllib
file = urllib.urlopen('http://www.google.com')
print file.read()

This is the error that I get:

Traceback (most recent call last):
  File "C:\workspace\GarchUpdate\src\Practice.py", line 26, in <module>
    file = urllib.urlopen('http://www.google.com')
  File "C:\Python26\lib\urllib.py", line 87, in urlopen
    return opener.open(url)
  File "C:\Python26\lib\urllib.py", line 206, in open
    return getattr(self, name)(url)
  File "C:\Python26\lib\urllib.py", line 345, in open_http
    h.endheaders()
  File "C:\Python26\lib\httplib.py", line 892, in endheaders
    self._send_output()
  File "C:\Python26\lib\httplib.py", line 764, in _send_output
    self.send(msg)
  File "C:\Python26\lib\httplib.py", line 723, in send
    self.connect()
  File "C:\Python26\lib\httplib.py", line 704, in connect
    self.timeout)
  File "C:\Python26\lib\socket.py", line 514, in create_connection
    raise error, msg
IOError: [Errno socket error] [Errno 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

I've tried it with several different pages but I can never get the urlopen method to execute correctly.

Megaphone

Works fine for me. Might want to check your firewall settings. – Avowal

@Avowal - Was thinking of that... my office has a really strong firewall. But I can navigate to the page in a browser without any problems. – Megaphone

check that your python has access to the firewall. Sorry I don't remember the exact settings, but there are variables in your environment that determine how python gets to the web (firewall/proxy/etc). – Flameout

Is there a proxy involved at all? The documentation states that this method will definitely not work if you're using a proxy that requires authentication. I'm not sure what kind of error would be raised if you tried. See docs.python.org/library/urllib.html#urllib.urlopen for more. – Avowal

Answer 1

Your code is not the problem here.

Do you have any Proxy settings in your IE?

This says the python documentation for urllib.urlopen:

In a Windows environment, if no proxy environment variables are set,
proxy settings are obtained from the registry's Internet Settings
section.

Diazotize

Answer 2

Try using urllib2 if it is feasible to change some lines of code. Set the timeout argument in seconds

For example:

urllib2.urlopen(http://www.abc.com/api, timeout=20)

Here the connection persists for a longer duration. So if for example you are reading an XML file that is too large it avoids incomplete reading.

The above code will never work if the Net connection is slow or it breaks suddenly.

Hecklau

Answer 3

If you have wireshark, check what's being sent out and if there is anything coming back at all. It will help you debug the problem if you can see the GET request being sent.

Also i remember having similar problem like this once, what i did was flush my dns cache

(ipconfig /flushdns) and restarted. It fixed my problem. It doesn't hurt to try i guess.

Lauds

Answer 4

for python 3:

import urllib.request

proxies=urllib.request.ProxyHandler({'http':None})

opener=urllib.request.build_opener(proxies)

urllib.request.install_opener(opener)

j=urllib.request.urlopen(url="https://google.com")

k=j.read()

print(k)

Mustang

Welcome to Stack Overflow! Take a moment to read through the editing help in the help center. Formatting on Stack Overflow is different than other sites. – Smokestack

@Mustang OP is using Python 2.6, does your answer work the same for that version? – Intrench

Answer 5

Your code is correct

import urllib
file = urllib.urlopen('http://www.google.com')
print file.read()

But most likely your internet has a problem or you have not set up the dns properly.

you can use request library insted of urllib.

Pelargonium