urllib.error.URLError: <urlopen error unknown url type: 'https>

Question

(Python 3.4.2) I've got a weird error when I run 'urllib.request.urlopen(url)' inside of a script. If I run it directly in the Python interpreter, it works fine, but not when I run it inside of a script through a bash shell (Linux).

I'm guessing it has something to do with the 'url' string, maybe because I'm creating the string through the 'string.join' method.

import urllib.request
url = "".join((baseurl, other_string, midurl, query))
response = urllib.request.urlopen(url)

The 'url' string prints perfectly, but when I try to create the 'response' string, I get this output:

File "./script.py", line 124, in <module>
    response = urllib.request.urlopen(url)
  File "/usr/lib/python3.4/urllib/request.py", line 153, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.4/urllib/request.py", line 455, in open
    response = self._open(req, data)
  File "/usr/lib/python3.4/urllib/request.py", line 478, in _open
    'unknown_open', req)
  File "/usr/lib/python3.4/urllib/request.py", line 433, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.4/urllib/request.py", line 1244, in unknown_open
    raise URLError('unknown url type: %s' % type)
urllib.error.URLError: <urlopen error unknown url type: 'https>

Python was compiled with SSL support on my computer (these commands work perfectly in the Python interpreter).

I've also tried wrapping the 'url' string with 'repr(url)' and 'str(url)'. I've tried this too:

url = "".join(("'", baseurl, other_string, midurl, query, "'"))

Anyone know what's going on?

-----EDIT-----
I figured it out. My url had a ":" in it, and I guess urllib didn't like that. I replaced it with "%3A" and now it's working.

Teran

Also see urllib HTTPS request: <urlopen error unknown url type: https>, urllib cannot read https, urllib.error.URLError: <urlopen error unknown url type: 'https>, urllib HTTPS request: <urlopen error unknown url type: https>, etc. – Mayflower

If you are using Anaconda, the answers here can help you: #54926896 – Magnify

@jww: The last link you offer has no answers, Im pretty sure you've seen it, as you have a similar comment on it. Perhaps linking to a bunch of other questions without vetting that they contain answers is not helpful. – Burnet

Answer 1

You should use urllib.parse.urlencode(), urllib.parse.urljoin(), etc functions to construct urls instead of manually joining the strings. It would take care of : -> %3A conversion e.g.:

>>> import urllib.parse
>>> urllib.parse.quote(':')
'%3A'

Mccallion

Answer 2

I figured it out. My url had a : in it, and urllib cannot use that character. I replaced it with %3A and now it's working. Web browsers usually convert : to %3A automatically, but urllib requires it to be converted first.

Teran

Answer 3

May be due to openssl-devel if you do not install it.

yum list installed|grep openssl

install it and try again after make.

sudo yum install openssl-devel
./configure
make

It can also be due to that your Python executable is built against a newer OpenSSL version than what your system has installed.

Cowden

Answer 4

people who encountered the error ValueError: unknown url type: 'http or ValueError: unknown url type: b'http

while opening some url like below with urllib.request.Request 'http://localhost/simple_form/insert.php'

just change localhost to 127.0.0.1

looks like Request method is looking for a domain.something in url

Angry