Using pip behind a proxy with CNTLM

Question

I am trying to use pip behind a proxy at work.

One of the answers from this post suggested using CNTLM. I installed and configured it per this other post, but running cntlm.exe -c cntlm.ini -I -M http://google.com gave the error Connection to proxy failed, bailing out.

I also tried pip install -–proxy=user:pass@localhost:3128 (the default CNTLM port) but that raised Cannot fetch index base URL http://pypi.python.org/simple/. Clearly something's up with the proxy.

Does anyone know how to check more definitively whether CNTLM is set up right, or if there's another way around this altogether? I know you can also set the http_proxy environment variable as described here but I'm not sure what credentials to put in. The ones from cntlm.ini?

Anderson

You need to know the IP address of your proxy and port number and set it accordingly in your cntlm.ini (Also your username and password for the proxy if you have one) – Outpatient

I wrote a blog on how to setup and use CNTLM. The post is for Ubuntu and for installing Rails, but I think the process might be the same. Edit: Read this article on how to setup CNTLM for Windows, basically same with my Ubuntu version. HTH. – Wonky

Answer 1

To setup CNTLM for windows, follow this article. For Ubuntu, read my blog post.

Edit:

Basically, to use CNTLM in any platform, you need to setup your username and hashed password, before using http://127.0.0.1:3128 as a proxy to your parent proxy.

1. Edit the config and add important information like domain, username, password and parent proxy.

2. Generate hashed password.

   Windows cntlm –c cntlm.ini –H

   Ubuntu/Linux cntlm -v -H -c /etc/cntlm.conf

3. Remove plain text password from the config and replace them with the generated passwords.

To check if working:

Windows cntlm –M http://www.google.com

Ubuntu/Linux sudo cntlm -M http://www.google.com/

For more detailed instructions, see links above.

Update:

Just for completeness sake, I was able to configure and use CNTLM in Windows recently. I encountered a problem during the syncing process of Kindle for PC because of our proxy and installing and configuring CNTLM for Windows fixed that issue for me. Refer to my article for more details.

Wonky

If this guide does not solve your issue, try running pip with commandline option --trusted-host pypi.python.org which did the trick for me. – Benjie

I'd just like to add to this, that I couldn't actually get cntlm working on Windows or Mac within our corporate environment, but installing Fiddler on a Windows machine and using it as the proxy solved the issue. – Looseleaf

Answer 2

With Ubuntu I could not get the proxy option to work as advertised – so the following command did not work:

sudo pip --proxy http://web-proxy.mydomain.com install somepackage

But exporting the https_proxy environment variable (note it's https_proxy not http_proxy) did the trick:

export https_proxy=http://web-proxy.mydomain.com

Then:

sudo -E pip install somepackage

Whittling

setting https_proxy does the job in Mac OS X too. – Montage

same here, --proxy did not work, I had to specify BOTH http_proxy AND https_proxy (Debian behind a corp firewall) – Ragouzis

Thank you for including -E in the sudo command, that constantly throws me when I wonder why programs aren't seeing my ENV. – Castled

Works under windows too! – Plath

-E added to sudo did the trick for me. It preserves the environment variable "http_proxy" that I set up earlier. Thanks! – Utham

I am behind a proxy and the following worked to get me updated sudo pip --proxy http://my.proxy:port install -U pip – Silencer

It only worked for me after updating to pip==6.0.8 on Ubuntu 14.04, with the pip suited within python-pip apt package it does NOT work nor with environment variables nor with --proxy when using -r option – Southerner

pip --proxy= someproxy-url:port install some-package worked for me in Win8.1 with pip 6.1.1 – Batter

I think this is the best answer. I've been setting up a bunch of things all day at work, and for the most part, setting http_proxy, https_proxy, HTTP_PROXY, and HTTPS_PROXY covered all sins. For some reason, I just couldn't get get_pip.py to work behind a proxy though. – Vacuva

Depending on your distribution and proxy settings, the above will not work. The general case will be: export http_proxy="http://proxy-server:port" (no authentication required) export http_proxy="http://user:password@proxy-server:port" (authentication required). Similar for https protocol. See shellhacks.com/en/HowTo-Use-a-Proxy-on-the-Linux-Command-Line – Stereogram

this should be the right answer because it solves without need to install additional software/packages. – Proa

sudo pip --proxy web-proxy.mydomain.com install somepackage worked for windows 7 – Mayamayakovski

Adding -H for sudo may also be needed, so as not to inherit HOME directory of originating user but that of superuser. – Biased

It works well on Mac pro. sudo pip --proxy web-proxy.mydomain.com install hvac. – Bluey

Answer 3

Under Windows dont forget to set

SET HTTPS_PROXY=<proxyHost>:<proxyPort>

what I needed to set for

pip install pep8

Torbernite

Works on fedora also with 'export http_proxy=<proxy_host>:<port>' – Dank

This worked for me, but only in the form SET HTTPS_PROXY=ht tps://user:pass@addr:port [minus the space I had to add to keep stackoverflow from garbling it] and I had to replace special characters in my password with hex in the form %nn – Montpellier

this worked for me. before, I was doing: SET HTTPS_PROXY=username:password@<proxyHost>:<proxyPort> and it was not working. – Beak

Does somebody have an explanation why the environment variable is needed and how its used by pip? For conda its enough to configure the proxy in the .condarc file. – Trochophore

An example call would look like: SET HTTPS_PROXY = https://512893:Pass%[email protected]:6050 for username 512893 & password pass#h98. Remember to url encode special characters in password or username (# in this case). Maybe also try setting the HTTP_PROXY flag as well. – Resource

this worked for me. nothing worked with the "--proxy" parameter from pip command itself. – Salesclerk

Answer 4

It was not working for me. I had to use https at work:

pip install --proxy=https://user@mydomain:port somepackage

In order to update, add -U.

Proto

This seems like the simplest solution for Windows, requiring no additional packages. – Convertiplane

If you need to use a password: pip install somepackage --proxy https://user:password@mydomain:port – Courtney

And if you don't know what user to put, maybe that's because there is none to put: pip install --proxy=https://mydomain:port somepackage – Evangelista

Answer 5

You can continue to use pip over HTTPS by adding your corporation's root certificate to the cacert.pem file in your site-packages/pip folder. Then configure pip to use your proxy by adding the following lines to ~/pip/pip.conf (or ~\pip\pip.ini if you're on Windows):

[global]
proxy = [user:passwd@]proxy.server:port

That's it. No need to use third party packages or give up HTTPS (of course, your network admin can still see what you're doing).

Cavity

This worked for me, but only after manually installing pip, which means you can't use get-pip.py initially – Nullipore

For the location of pip.ini see pip.pypa.io/en/latest/user_guide.html#configuration In windows I found cacert in "C:\Python34\Lib\site-packages\pip_vendor\requests\cacert.pem" – Bordie

This worked for me but I didnt have a pip.config file so I had to create one in ~/.pip/pip.conf then export PIP_CONFIG_FILE=/root/.pip/pip.conf and my installation worked! Thanks! – Surbeck

On windows7 this did not work. After installing python35 on windows into c:\python35-32 I find pip in the c:\python35-32\Scripts folder. No pip.ini. Manually creating one in %APPDATA%\pip as suggested by @Bordie did not do anything. – Kella

On Windows 7, creating pip.ini in the ~\pip directory worked. – Neolamarckism

are those square brackets around user required? why is symbol @ used after password? – Unfair

Thanks, you set me on the right path! FYI, for Python 3.7 on Windows 7 pip's certificates file is apparently located at [Python37]\Lib\site-packages\pip\_vendor\certifi\cacert.pem. Export your corp's SSL interception certificate in Base64 .cer format, paste the public key into that file, and voilà, full https support behind the corporate proxy. – Handspike

Your pip ini location on windows may be AppData/Roaming/pip/pip.ini .... – Spillman

pip config set global.proxy [user:passwd@]proxy.server:port command auto create file – Artful

I had to put it here: C:\Users\MYUSERNAME\AppData\Roaming\pip\pip.ini This one alone didn't work: C:\Users\MYUSERNAME\AppData\Local\pip\pip.ini – Clarey

Answer 6

for windows; set your proxy in command prompt as
set HTTP_PROXY=domain\username:password@myproxy:myproxyport

example:
set http_proxy=IND\namit.kewat:[email protected]:8880

Weasel

What if the password has special chars such as ! and @. – Hampden

escape special chars with the bash escape char: \ – Truthfunction

Escaping did not do the trick... I've got the characters ! and @ in my proxy username and password... Neither using the environment variables nor the --proxy parameter did the trick (both escaped and unescaped) – Ramify

You can't escape '@' in your password with escape character (), you have to url encode it to %40 in the proxy string. – Masturbation

I needed set HTTP_PROXY=username:password@myproxy:myproxyport set HTTPS_PROXY=username:password@myproxy:myproxyport – Montpellier

If you have pac file, you can download it, and look for the proxy server to setup the windows environment. – Arouse

Setting this environment variable seems to have no effect on pip on my machine – Funke

Answer 7

This worked for me (on Windows via CMD):

pip install --proxy proxyserver:port requests

Eliezer

Answer 8

$ pip --proxy http://proxy-host:proxy-port install packagename

This is what worked for me on

Dogooder

This is really useful when only temporarily behind a proxy. I also had to add my username and password to the syntax: pip --proxy http://username:password@proxy-host:proxy-port install <package> – Blackjack

Answer 9

Under our security policy I may not use https with pypi, SSL-inspection rewrites certificates, it breaks the built-in security of pip for www.python.org. The man in the middle is the network-admin.

So I need to use plain http. To do so I need to override the system proxy as well as the default pypi:

bin/pip install --proxy=squidproxy:3128 -i http://www.python.org/pypi --upgrade "SQLAlchemy>=0.7.10"

Interrupt

This doesn't work anymore. PyPi now automatically forwards you to https. – Medusa

SSL inspection does not seem to work; could it be because key pinning, or forward secrecy? – Interrupt

@SamuelHarmer . Security has four syllables. Too many for Trump. – Anamariaanamnesis

I think @Interrupt may be saying that his pip broke (in 2013) with HTTPS because his corporate proxy rewrites all certs to use their own root cert (so they can decrypt all SSL traffic) and pip fails the cert verification. Of course in 2017, pip automatically avoids certificate checking when --proxy is used. Many corporate proxies already effectively change all of your traffic to non-SSL anyway because even when navigating to SSL sites, you are proxying via non-SSL HTTP proxy. – Hordein

Answer 10

Phone as mobile hotspot/USB tethering

If I have much trouble finding a way through the corporate proxy, I connect to the web through my phone (wireless hotspot if I have wifi, USB tether if not) and do a quick pip install.

Might not work for all setups, but should get most people by in a pinch.

Glennglenna

Answer 11

Open the Windows command prompt.

Set proxy environment variables.

set http_proxy=http://user:password@proxy_ip:port
set https_proxy=https://user:password@proxy_ip:port

Install Python packages using proxy in the same Windows command prompt.

pip install --proxy="user:password@proxy_ip:port" package_name

Enid

Answer 12

In Windows 7:

pip install --proxy DOMAIN\user:password@proxyaddress:port package

i.e.:

pip install --proxy BR\neo:[email protected]:8080 virtualenv

Lazos

Answer 13

In Ubuntu 14.04 LTS

   sudo pip --proxy http://PROXYDOM:PROXYPORT install package

Cheers

Nipper

Answer 14

I had the same issue : behind a corporate proxy with auth at work, I couldn't have pip work, as well as Sublime Text 2 (well, it worked with custom setup of my proxy settings). For pip (and I'll try that on git), I solved it installing cntlm proxy. It was very simple to configure :

1. Edit cntlm.ini
2. Edit "Username", "Domain", "Password" fields
3. Add a "Proxy" line, with your proxy settings : server:port
4. Make sure the line "NoProxy" integrates "localhost" (like that by default)
5. Note the default port : 3128
6. Save and that's it.

To test that works, just launch a new command line tool, and try :

pip install django --proxy=localhost:3128

That worked for me. Hope this will help you.

Pigheaded

Answer 15

Set up invironment variable in Advanced System Settings. In Command prompt it should behave like this :

C:\Windows\system32>echo %http_proxy%

http://username:passowrd@proxy:port

C:\Windows\system32>echo %https_proxy%

http://username:password@proxy:port

Later , Simply pip install whatEver should work.

Wayside

Answer 16

I could achieve this by running:

pip install --proxy=http://user:[email protected]:3128 package==version

I'm using Python 3.7.3 inside a corporative proxy.

Cloyd

Answer 17

if you want to upgrade pip by proxy, can use (for example in Windows):

python -m pip --proxy http://proxy_user:proxy_password@proxy_hostname:proxy_port insta
ll --upgrade pip

Choreographer

Answer 18

For windows users: if you want to install Flask-MongoAlchemy then use the following code

pip install Flask-MongoAlchemy --proxy="http://example.com:port"**

Steal

Answer 19

Using pip behind a work proxy with authentification, note that quotation is required for some OSes when specifing the proxy url with user and password:

pip install <module> --proxy 'http://<proxy_user>:<proxy_password>@<proxy_ip>:<proxy_port>'

Documentation: https://pip.pypa.io/en/stable/user_guide/#using-a-proxy-server

Example:

pip3 install -r requirements.txt --proxy 'http://user:[email protected]:1234'

Example:

pip install flask --proxy 'http://user:[email protected]:1234'

Proxy can also be configured manually in pip.ini. Example:

[global]
proxy = http://user:[email protected]:1234 

Documentation: https://pip.pypa.io/en/stable/user_guide/#config-file

Bootie

Hello Mihai, you should be providing an answer, but instead you're asking a question. You may instead want to review the previous answers and add a comment to the relevant one(s). I'd suggest looking at whether the environment variable is properly exported (export http_proxy). Consider solving the problem and updating your answer so that it provides a solution. – Indecorum

Thank you for reviewing this answer, great job! – Indecorum

Answer 20

If you are connecting to the internet behind a proxy, there might be problem in running the some commands.

Set the environment variables for proxy configuration in the command prompt as follows:

set http_proxy=http://username:password@proxyserver:proxyport
set https_proxy=https://username:password@proxyserver:proxyport

Jedlicka

Answer 21

How about just doing it locally? Most likely you are able to download from https source through your browser

1. Download your module file (mysql-connector-python-2.0.3.zip /gz... etc).

2. Extract it and go the extracted dir where setup.py is located and call:

   C:\mysql-connector-python-2.0.3>python.exe setup.py install
   

Faulkner

But then you have to go and get all of the dependencies (and their dependencies, and ...) yourself – Christenachristendom

Answer 22

At CentOS (actually I think all linux distros are similar) run

env|grep http_proxy

and

env|grep https_proxy

check what is the output of those commands (they should contain your proxy addresses).

If the outputs are empty or have incorrect values, modify them, for ex:

export http_proxy=http://10.1.1.1:8080
export https_proxy=http://10.1.1.1:8080

Now try to fetch and install some packages by using pip:

pip --proxy http://10.1.1.1:8080 install robotframework

and actually I have never met the case when it didn't work. For some systems you need to be a root (sudo is not enough).

Ascender

Answer 23

Warning, there is something very bad with the "pip search" command. The search command do not use the proxy setting regardless of the way it's being passed.

I was trying to figure out the problem only trying the "search" command, and found this post with detailed explanation about that bug: https://github.com/pypa/pip/issues/1104

I can confirm the bug remains with pip 1.5.6 on Debian 8 with python 2.7.9. The "pip install" command works like a charm.

Maltese

Answer 24

I got the error:

chris@green:~$ sudo http_proxy=http://localhost:3128 pip install django==1.8.8 
Downloading/unpacking django==1.8.8
  Cannot fetch index base URL http://pypi.python.org/simple/
  Could not find any downloads that satisfy the requirement django==1.8.8
No distributions at all found for django==1.8.8
Storing complete log in /home/chris/.pip/pip.log

(The proxy server's port is ssh port forwarded to localhost:3128).

I had to set both http and https proxies to make it work:

chris@green:~$ sudo http_proxy=http://localhost:3128 https_proxy=http://localhost:3128 pip install django==1.8.8
Downloading/unpacking django==1.8.8
  Downloading Django-1.8.8.tar.gz (7.3Mb): 7.3Mb downloaded
  Running setup.py egg_info for package django

    warning: no previously-included files matching '__pycache__' found under directory '*'
    warning: no previously-included files matching '*.py[co]' found under directory '*'
Installing collected packages: django
  Running setup.py install for django

    warning: no previously-included files matching '__pycache__' found under directory '*'
    warning: no previously-included files matching '*.py[co]' found under directory '*'
    changing mode of build/scripts-2.7/django-admin.py from 644 to 755
    changing mode of /usr/local/bin/django-admin.py to 755
    Installing django-admin script to /usr/local/bin
Successfully installed django
Cleaning up...

as http://pypi.python.org/simple/ redirects to https://pypi.python.org/simple but pip's error does not tell you.

Otranto

Answer 25

I am also no expert in this but I made it work by setting the all_proxy variable in the ~/.bashrc file. To open ~/.bashrc file and edit it from a terminal run following commands,

gedit ~/.bashrc &

Add following at the end of file,

export all_proxy="http://x.y.z.w:port"

Then either open a new terminal or run following in the same terminal,

source ~/.bashrc

Just setting http_proxy and https_proxy variables aren't enough for simple usage pip install somepackage. Though somehow sudo -E pip install somepackage works, but this have given me some problem in case I am using a local installation of Anaconda in my users' folder.

P.S. - I am using Ubuntu 16.04.

Puerperal

Answer 26

2022 for windows:

I know there are many answers and nearly every other question with pip and behind a proxy is refering to this question:

So in my opinion it is on the one hand the proxy thing, which is answered in the questions below.

pip install --proxy=https://<windowsuser>:<pw>@<proxy>:port package

After that you have to deal with the SSL certificates. You have to add the trusted sources. Usually they are standing in the Error message. For example: ERROR: .... host='files.pythonhosted.org' And here is my solution for installing for example Django:

pip install Django --proxy http://windowsuser:password@proxy:port --trusted-host pypi.python.org --trusted-host pypi.org --trusted-host files.pythonhosted.org

Blennioid

Answer 27

This is what works for me:

pip --proxy proxy url:port command package

Balcer

Answer 28

Set the following environment variable: export PIP_PROXY=http://web-proxy.mydomain.com

Duel

Answer 29

If you are using Linux, as root:

env https_proxy=http://$web_proxy_ip:$web_proxy_port pip install something

When you use env it exports the variable https_proxy for the current execution of the command pip install.

$web_proxy_ip is the hostname or IP of your Proxy $web_proxy_port is the Port

Mastic

Answer 30

I solved the problem with PIP in Windows using "Fiddler" (https://www.telerik.com/download/fiddler). After downloading and installing, do the following:

"Rules" => click "Automatically Authenticate"

Example: pip install virtualenv -proxy 127.0.0.1:8888

Just open your prompt and use.

https://github.com/pypa/pip/issues/1182 Search for "voltagex" (commented on 22 May 2015)

Obtrude

Answer 31

for windows go to C:/ProgramData/pip/pip.ini, and set

[global]

proxy = http://YouKnowTheRest

with your proxy details. This permanently configures the proxy for pip.

Westsouthwest

Answer 32

A simpler approach might be:

1. Create a folder named "pip" in your $HOME directory.
2. Create a file named "pip.ini" (Windows) or "pip.conf" (Linux) in the directory created in step 1

3. Copy & paste the following lines under the pip.ini/pip.conf:

   [global]
   trusted-host = pypi.python.org
                  pypi.org
                  files.pythonhosted.org 
   

Vacua

Answer 33

1. For PIP Installation:

ex:PORT = 9090
ex:PROXY_SERVER = stackoverflow
USERNAME:your user id
PASSWORD: your password

• sudo pip2 install PACKAGENAME --proxy https://USERNAME:PASSWORD@PROXY_SERVER:PORT/ for Python2.7

• sudo pip3 install PACKAGENAME --proxy https://USERNAME:PASSWORD@PROXY_SERVER:PORT/ for Python3.5

Example:

sudo pip2 install pandas --proxy https://USERNAME:PASSWORD@PROXY_SERVER:PORT/

2. For apt-get Installation

sudo http_proxy=http://USERNAME:PASSWORD@PROXY_SERVER:PORT/ apt-get install PACKAGENAME

Example:

sudo http_proxy=http://USERNAME:YOURPASSWORD@PROXY_SERVER:PORT/ apt-get install tensorrt

sudo http_proxy=http://USERNAME:YOURPASSWORD@PROXY_SERVER:PORT/ apt-get update

Charlenecharleroi