"The handshake operation timed out" with urllib, works with requests

First of all, this is for a Gira Homeserver, which is a home automation server. It has Python 2.7, and I can't install external modules.

But for testing and examples, I've been using both python 2.7.15 and python 3.6.8 (but have tried a few other version as well - same result)

What I'm trying to do, is to read content from the webserver of my Philips Android TV. This works fine with browser, it works fine with Curl, and it works fine with Python requests. But it does not work with urllib2, which is what I need to use for this to work with my home automation system.

The TV is providing json output on a https webpage requiring Digest auth.

Urllib example (Python3, to be able to compare with requests):

import urllib.request
import ssl

url="https://192.168.3.100:1926/6/powerstate"
username="6AJeu5Ffdm9dQnum"
password="5a21386d952c2f1fbe66be2471d98c391bb918a6d2130cdf1b6deb2b87872eaa"

ctx = ssl._create_unverified_context()

auth = urllib.request.HTTPDigestAuthHandler(urllib.request.HTTPPasswordMgrWithDefaultRealm())
auth.add_password (None, url, username, password)

opener = urllib.request.build_opener(urllib.request.HTTPSHandler(context=ctx,debuglevel=1), auth)
urllib.request.install_opener(opener)
response = urllib.request.urlopen(url,None,10)

Requests example:

import requests
import ssl

url="https://192.168.3.100:1926/6/powerstate"
username="6AJeu5Ffdm9dQnum"
password="5a21386d952c2f1fbe66be2471d98c391bb918a6d2130cdf1b6deb2b87872eaa"


from requests.auth import HTTPDigestAuth
from requests.packages.urllib3.exceptions import InsecureRequestWarning
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)
r = requests.get(url, auth=HTTPDigestAuth(username, password), timeout=10, verify=False)
print(r.status_code)
print(r.content)

The urllib example times out with following error:

stianj@buick:~$ python3 stack.py
send: b'GET /6/powerstate HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: 192.168.3.100:1926\r\nUser-Agent: Python-urllib/3.6\r\nConnection: close\r\n\r\n'
reply: 'HTTP/1.1 401 Unauthorized\r\n'
header: Date: Wed, 10 Jul 2019 21:36:45 GMT+00:00
header: Accept-Ranges: bytes
header: Server: Restlet-Framework/2.3.12
header: WWW-Authenticate: Digest realm="XTV", domain="/", nonce="MTU2Mjc5NDYwNTI3ODo1NTNlMTFhYzk5MjJjODQyMTYyZjAxZjRhYmYyYzNhMA==", algorithm=MD5, qop="auth"
header: Content-Length: 424
header: Content-Type: text/html; charset=UTF-8
Traceback (most recent call last):
  File "/usr/lib/python3.6/urllib/request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/usr/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/usr/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/usr/lib/python3.6/http/client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "/usr/lib/python3.6/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/usr/lib/python3.6/ssl.py", line 817, in __init__
    self.do_handshake()
  File "/usr/lib/python3.6/ssl.py", line 1077, in do_handshake
    self._sslobj.do_handshake()
  File "/usr/lib/python3.6/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
socket.timeout: _ssl.c:835: The handshake operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "stack.py", line 15, in <module>
    response = urllib.request.urlopen(url,None,10)
  File "/usr/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.6/urllib/request.py", line 532, in open
    response = meth(req, response)
  File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.6/urllib/request.py", line 564, in error
    result = self._call_chain(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 1208, in http_error_401
    host, req, headers)
  File "/usr/lib/python3.6/urllib/request.py", line 1089, in http_error_auth_reqed
    return self.retry_http_digest_auth(req, authreq)
  File "/usr/lib/python3.6/urllib/request.py", line 1103, in retry_http_digest_auth
    resp = self.parent.open(req, timeout=req.timeout)
  File "/usr/lib/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/usr/lib/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 1361, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/usr/lib/python3.6/urllib/request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error _ssl.c:835: The handshake operation timed out>

while the request example works just fine (shows output from webpage. Since I'm using the same Python version for both examples, I'd expect ssl parameters and ciphers etc. to be the same. What is extremely interesting, is that a POST works just fine with urllib. It's just the GET that times out.

I know it's always recommended to use requests these days, but that's not an option for me. Would anyone have an explanation to the handshake error?

Been banging my head at this for a few days...

Recommended topics

Hot tags