Python's requests "Missing dependencies for SOCKS support" when using SOCKS5 from Terminal

Asked 5/8, 2016 at 16:35 Answered 15/7, 2023 at 19:38

Solved python python-2.7 python-requests socks urllib3

I'm trying to interact with an API from my Python 2.7 shell using a package that relies on Python's requests. Thing is the remote address is blocked by my network (university library).

So to speak to the API I do the following:

~$ ssh -D 8080 [email protected]

And then, in new terminal, in local computer:

~$ export http_proxy=socks5://127.0.0.1:8080 https_proxy=socks5://127.0.0.1:8080

Then I run the program in Python console but fails:

~$ python
>>> import myscript
>>> id = '1213'
>>> token = 'jd87jd9'
>>> connect(id,token)

File "/home/username/.virtualenvs/venv/local/lib/python2.7/site-packages/requests/sessions.py", line 518, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/home/username/.virtualenvs/venv/local/lib/python2.7/site-packages/requests/sessions.py", line 475, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/username/.virtualenvs/venv/local/lib/python2.7/site-packages/requests/sessions.py", line 585, in send
    r = adapter.send(request, **kwargs)
  File "/home/username/.virtualenvs/venv/local/lib/python2.7/site-packages/requests/adapters.py", line 370, in send
    conn = self.get_connection(request.url, proxies)
  File "/home/username/.virtualenvs/venv/local/lib/python2.7/site-packages/requests/adapters.py", line 273, in get_connection
    proxy_manager = self.proxy_manager_for(proxy)
  File "/home/username/.virtualenvs/venv/local/lib/python2.7/site-packages/requests/adapters.py", line 169, in proxy_manager_for
    **proxy_kwargs
  File "/home/username/.virtualenvs/venv/local/lib/python2.7/site-packages/requests/adapters.py", line 43, in SOCKSProxyManager
    raise InvalidSchema("Missing dependencies for SOCKS support.")
requests.exceptions.InvalidSchema: Missing dependencies for SOCKS support.

This excerpt is from the adapters.py requests module:

> try:
>     from .packages.urllib3.contrib.socks import SOCKSProxyManager except ImportError:
>     def SOCKSProxyManager(*args, **kwargs):
>         raise InvalidSchema("Missing dependencies for SOCKS support.")

Now problem seems to be originated in urllib3's SOCKSProxyManager.

So I read you can use SOCKSProxyManager with SOCKS5 if you have install PySocks or you do a pip install urllib3[socks]

Alas, I tried both PySocks and urllib3 with Socks without any success.

Any idea of another workaround?

EDIT:

I also tried pip install requests[socks] (that's requests 2.10.0 with Socks support) and I am getting this:

  File "/home/username/.virtualenvs/venv/local/lib/python2.7/site-packages/requests/adapters.py", line 467, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: SOCKSHTTPSConnectionPool(host='api-server.com', port=443): Max retries exceeded with url: /auth (Caused by NewConnectionError('<requests.packages.urllib3.contrib.socks.SOCKSHTTPSConnection object at 0x95c7ccc>: Failed to establish a new connection: SOCKS5 proxy server sent invalid data',))

Rag answered 5/8, 2016 at 16:35 Comment(0)

I had the same issue with conda and requests 2.11 (I work in a Ubuntu VM behind a corporate proxy).

This issue helped me. I changed my environment variable all_proxy (which was originally set to a SOCK proxy socks://....) to the https version in my .bashrc file :

export all_proxy="https://<proxy>:<port>/"

and now it works.

Ordure answered 10/10, 2016 at 13:23 Comment(5)

Thanks! I totally ditched working from the library so I can't test. But I trust you! – Rag 11/10, 2016 at 20:8

but I AM using socks proxy – Autorotation 27/7, 2018 at 5:48

Hint for noobs: echo $all_proxy to see what your machine is currently set to. It's probably socks://<proxy>:<port>/, so all you have to do is replace the prefix. – Wacke 9/1, 2019 at 17:53

either this, or pip install pysocks – Sigler 24/6, 2019 at 9:17

this fix it for me. thanks. even with all the deps installed, this is the error you will get if the socks:// prefix is used – Sagittarius 31/8, 2019 at 10:51

167

This means that requests is using socks as a proxy and that socks is not installed.

Just run pip install pysocks

Or install requests via pip install "requests[socks]"

Search answered 4/6, 2017 at 11:32 Comment(8)

This works for me, however, we need to close the proxy first to install this module. – Emmy 23/1, 2018 at 14:45

That's the problem, if pip install doesn't go through the proxy how am I supposed to install pysocks – Yukyukaghir 27/11, 2019 at 3:3

Wow, for the second time I face that problem, googling, and reaching your answer which is the solution I always forget :) If only stackoverflow allows upvoting multi times, I'd upvote this thousand times :) – Proletariat 31/12, 2019 at 2:53

TypeError("__init__() got an unexpected keyword argument 'cert_file'") – Hagai 18/9, 2021 at 16:6

In 2022 This is still the correct answer to the question. – Metathesize 7/3, 2022 at 14:11

you can install pysocks using tarball first – Yolondayon 15/3, 2022 at 9:11

@Yukyukaghir you can first unset the proxy by unset all_proxy && unset ALL_PROXY, then install the pysocks and again set the proxy if needed export all_proxy="socks5://127.0.0.1:1080" – Airdrie 5/2, 2023 at 18:40

@BenyaminJafari if I unset the proxy I can't connect to the internet – Yukyukaghir 4/3, 2023 at 13:32

I had the same issue with conda and requests 2.11 (I work in a Ubuntu VM behind a corporate proxy).

This issue helped me. I changed my environment variable all_proxy (which was originally set to a SOCK proxy socks://....) to the https version in my .bashrc file :

export all_proxy="https://<proxy>:<port>/"

and now it works.

Ordure answered 10/10, 2016 at 13:23 Comment(5)

Thanks! I totally ditched working from the library so I can't test. But I trust you! – Rag 11/10, 2016 at 20:8

but I AM using socks proxy – Autorotation 27/7, 2018 at 5:48

Hint for noobs: echo $all_proxy to see what your machine is currently set to. It's probably socks://<proxy>:<port>/, so all you have to do is replace the prefix. – Wacke 9/1, 2019 at 17:53

either this, or pip install pysocks – Sigler 24/6, 2019 at 9:17

this fix it for me. thanks. even with all the deps installed, this is the error you will get if the socks:// prefix is used – Sagittarius 31/8, 2019 at 10:51

In Ubuntu you can run :
unset all_proxy && unset ALL_PROXY

Oatcake answered 17/4, 2018 at 12:6 Comment(1)

Well, this disables the proxy, which is not what OP asked for. – Endostosis 24/9, 2021 at 18:10

I added the requests[socks]>=2.10.0 to my requirements.txt, updated my https_proxy env variable, and came across the above error. I then tried a regular pip install requests[socks] after resetting the https_proxy env variable and PySocks was installed. I'm not sure why the pip install -Ur requirements.txt failed to install PySocks the first time.

After that, I was able to make a request in python using the socks proxy.

It looks like your socks server is not behaving. I would see if you, or your admin, could watch the logs and see what the machine is complaining about.

Gereld answered 8/8, 2016 at 18:27 Comment(2)

I'll test it your suggestion but as soon as I have the chance. For the moment I decided to work from elsewhere (not from the University). And I'll let you know. For the moment I'll upvote your answer. Thanks, Kirk! – Rag 9/8, 2016 at 10:18

Yes, this was a non-obvious issue. For some reason, this module was not correctly installed via requirements.txt – Viperous 2/8, 2017 at 20:18

In debian 10, Python 3:

pip3 install pysocks

Audible answered 7/4, 2020 at 21:7 Comment(0)

How about using apt-get instead of pip?
When I use apt-get, I can ignore proxy settings, so the following works good for me.

Using

apt-get update && apt-get install  python-socks

Instead of

pip install pysocks

Addend answered 15/12, 2021 at 3:1 Comment(2)

This method works well, when I build docker image under setting vpn proxy. – Addend 15/12, 2021 at 3:8

This is definitely not recommended. This way you are altering the operating system. You either want to use pip to install it for the local user only (in their home directory) or in a virtual environment. Polluting the OS will bite you in the long term. – Kamerad 7/1 at 21:28

I also stumbled upon this issue while doing a simple pip install -U pip, but information I found from your question helped me resolved my issue. I am on Mac OS X.

As you have pointed out, adapters.py from the requests package was trying to do this:

try:
    from .packages.urllib3.contrib.socks import SOCKSProxyManager
except ImportError:
    def SOCKSProxyManager(*args, **kwargs):
        raise InvalidSchema("Missing dependencies for SOCKS support.")

So it seems sensible to look for the place of definition of SOCKSProxyManager. It seems to be in a "contrib" module in urllib3 and not installed alongside urllib3 by default. The docstring of that module says:

This module contains provisional support for SOCKS proxies from within
urllib3. This module supports SOCKS4 (specifically the SOCKS4A variant) and
SOCKS5. To enable its functionality, either install PySocks or install this
module with the ``socks`` extra.

The instructions of the pip docs says this about setuptools extras:

6. Install a package with setuptools extras.

$ pip install SomePackage[PDF]
$ pip install git+https://git.repo/some_pkg.git#egg=SomePackage[PDF]
$ pip install SomePackage[PDF]==3.0
$ pip install -e .[PDF]==3.0  # editable project in current directory

So I followed the instructions and did:

$ pip install 'urllib3[socks]'

I then continued with pip install -U pip, which was what I was supposed to be doing, and now it works.

I wonder how many people got tricked-up by the square brackets, as Bash and other shells often treat it as a special character, which needs to be escaped for it to reach the invoked program (in this case, pip).

Cons answered 15/3, 2017 at 12:58 Comment(1)

Good to know my issue helped you resolve yours. And thankfully yours will solve someone else's issue too! – Rag 17/3, 2017 at 17:44

I got the same error just a few mins ago.Then re-installed request[socks] via pip. It seems that there was a missing part of socks which is win-inet_pton. Pip installed it and the problem is solved.

Pelton answered 5/6, 2017 at 2:56 Comment(3)

pip install request[socks] did NOT work, but pip install -U requests[socks] worked – Geronto 14/11, 2017 at 13:39

-U --upgrade option alone will not work if you had some previous package installed. pip install -U requests[socks] --ignore-installed will work. – Subalpine 11/8, 2018 at 22:47

@erm3nda --force-reinstall is somewhat cleaner than --ignore-installed, as it will clean up a (potentially different) previously installed package first. I use --ignore-installed only when I want to locally override a package that is installed system-wide (which can happen inside a virtual env set up with --system-site-packages). – Hydroxy 2/10, 2018 at 13:2

You must add socks support with python requests library:

pip install requests[socks]

Tremendous answered 18/8, 2020 at 11:26 Comment(0)

In ubuntu, I do the following commands:

# Unset socks proxy
unset all_proxy    
unset ALL_PROXY
# Install missing dependencies:
pip install pysocks
# Reset proxy
source ~/.bashrc

Kylix answered 20/9, 2019 at 11:25 Comment(0)

I faced the same issue where got the error Missing dependencies for SOCKS support. with Anaconda Python 3.7

As conda command was working fine, I installed pysocks with the command conda install pysocks

This resolved the issue.

Heigl answered 17/4, 2020 at 13:48 Comment(0)

Just unset your all_proxy environment variable, and this should work. Also you can refer to this issue in github.

On Ubuntu you can use the following command unset all_proxy and restart terminal

Reece answered 9/1, 2017 at 10:23 Comment(1)

Unfortunately I resolved the issue by not going to the library. So I can't give you feedback. But thanks anyway! :) – Rag 9/1, 2017 at 16:58

My environment is Ubuntu 16.4 LTS and Python3.5.2 I use pip3 to install libs got the same issue. so I use the command unset ALL_PROXY to solve this problem and it works.

PS: use printenv | grep -i proxy to show the proxy info.

Lin answered 1/9, 2017 at 17:16 Comment(1)

Thank you, your answer led me to fix my issue :) – Hatchment 7/6, 2018 at 14:27

I used all the above methods, but only this worked for me:

set | grep -i all_proxy

it returned:

ALL_PROXY=socks://127.0.0.1:1080/
all_proxy=socks://127.0.0.1:1080/

and then, I did this:

export all_proxy=""

Finally, it didn't appear mistakes again.

Dinnie answered 18/11, 2019 at 8:4 Comment(0)

Got the same error after a reported issue that pysocks would not be pulled by the requests[socks]>=0.23.0 requirement.

I could (esoterically) fix it by removing the requests>=0.23.0 requirement i also had in the same requirements.txt.

Running Python3.7 on Debian, and the user who reported the bug is running python3.6 on Ubuntu.

Before the patch

$ cat requirements.txt 
requests>=2.23.0
requests[socks]>=2.23.0
$ python3 -m venv venv
$ . venv/bin/activate
(venv) $ pip --no-cache-dir install -r requirements.txt
Collecting requests>=2.23.0 (from -r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/45/1e/0c169c6a5381e241ba7404532c16a21d86ab872c9bed8bdcd4c423954103/requests-2.24.0-py2.py3-none-any.whl (61kB)
    100% |████████████████████████████████| 71kB 202kB/s 
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests>=2.23.0->-r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/9f/f0/a391d1463ebb1b233795cabfc0ef38d3db4442339de68f847026199e69d7/urllib3-1.25.10-py2.py3-none-any.whl (127kB)
    100% |████████████████████████████████| 133kB 617kB/s 
Collecting idna<3,>=2.5 (from requests>=2.23.0->-r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/a2/38/928ddce2273eaa564f6f50de919327bf3a00f091b5baba8dfa9460f3a8a8/idna-2.10-py2.py3-none-any.whl (58kB)
    100% |████████████████████████████████| 61kB 2.8MB/s 
Collecting chardet<4,>=3.0.2 (from requests>=2.23.0->-r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)
    100% |████████████████████████████████| 143kB 10.4MB/s 
Collecting certifi>=2017.4.17 (from requests>=2.23.0->-r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/5e/c4/6c4fe722df5343c33226f0b4e0bb042e4dc13483228b4718baf286f86d87/certifi-2020.6.20-py2.py3-none-any.whl (156kB)
    100% |████████████████████████████████| 163kB 4.7MB/s 
Installing collected packages: urllib3, idna, chardet, certifi, requests
Successfully installed certifi-2020.6.20 chardet-3.0.4 idna-2.10 requests-2.24.0 urllib3-1.25.10

After the patch

Now applying the following patch:

diff --git a/sauron/requirements.txt b/sauron/requirements.txt
index a0c901b..f38d18f 100644
--- a/sauron/requirements.txt
+++ b/sauron/requirements.txt
@@ -1,2 +1 @@
-requests>=2.23.0
 requests[socks]>=2.23.0

$ cat requirements.txt 
requests[socks]>=2.23.0
$ cat requirements.txt 
requests[socks]>=2.23.0
(venv) $ pip --no-cache-dir install -r requirements.txt 
Collecting requests[socks]>=2.23.0 (from -r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/45/1e/0c169c6a5381e241ba7404532c16a21d86ab872c9bed8bdcd4c423954103/requests-2.24.0-py2.py3-none-any.whl (61kB)
    100% |████████████████████████████████| 71kB 234kB/s 
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests[socks]>=2.23.0->-r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/9f/f0/a391d1463ebb1b233795cabfc0ef38d3db4442339de68f847026199e69d7/urllib3-1.25.10-py2.py3-none-any.whl (127kB)
    100% |████████████████████████████████| 133kB 678kB/s 
Collecting chardet<4,>=3.0.2 (from requests[socks]>=2.23.0->-r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)
    100% |████████████████████████████████| 143kB 2.3MB/s 
Collecting idna<3,>=2.5 (from requests[socks]>=2.23.0->-r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/a2/38/928ddce2273eaa564f6f50de919327bf3a00f091b5baba8dfa9460f3a8a8/idna-2.10-py2.py3-none-any.whl (58kB)
    100% |████████████████████████████████| 61kB 6.2MB/s 
Collecting certifi>=2017.4.17 (from requests[socks]>=2.23.0->-r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/5e/c4/6c4fe722df5343c33226f0b4e0bb042e4dc13483228b4718baf286f86d87/certifi-2020.6.20-py2.py3-none-any.whl (156kB)
    100% |████████████████████████████████| 163kB 4.8MB/s 
Collecting PySocks!=1.5.7,>=1.5.6; extra == "socks" (from requests[socks]>=2.23.0->-r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/8d/59/b4572118e098ac8e46e399a1dd0f2d85403ce8bbaad9ec79373ed6badaf9/PySocks-1.7.1-py3-none-any.whl
Installing collected packages: urllib3, chardet, idna, certifi, PySocks, requests
Successfully installed PySocks-1.7.1 certifi-2020.6.20 chardet-3.0.4 idna-2.10 requests-2.24.0 urllib3-1.25.10

Pysocks was effectively pulled.

Pritchett answered 20/8, 2020 at 13:18 Comment(0)

My fix is:After I unproxy my terminal,the installation is running well.

Gibbeon answered 12/12, 2021 at 0:30 Comment(1)

As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center. – Stellite 12/12, 2021 at 1:0

This page shows up for a similar issue under AlmaLinux 8, a pip3 install failing, it was resolved by installing:

sudo dnf install 3proxy (Tried the Python-based cases above, no success)

Worldweary answered 15/7, 2023 at 19:38 Comment(0)

Hot tags

Godot Unity Godot Help Programming Godot 4.X GUI GDScript 3D 2D Physics CSharp Godot 3.X VR XR Projects C++

Before the patch

After the patch

Recommended topics

Hot tags