Jupyter notebook, wrong sys.path and sys.executable

Question

I'm trying to run the anaconda distribution of python libraries in a Jupyter Notebook, but when I run the notebook I keep getting ImportErrors because the python path is set to the default distribution from Mac OS X 10.11

When I print out the sys.path and sys.executable, they differ when running python vs running jupyter notebook. For example,

from pprint import pprint as p
import sys

p(sys.path)

After doing this in python I get the correct output:

['',
 '/Users/glennraskovich/anaconda2/lib/python27.zip',
 '/Users/glennraskovich/anaconda2/lib/python2.7',
 '/Users/glennraskovich/anaconda2/lib/python2.7/plat-darwin',
 '/Users/glennraskovich/anaconda2/lib/python2.7/plat-mac',
 '/Users/glennraskovich/anaconda2/lib/python2.7/plat-mac/lib-scriptpackages',
 '/Users/glennraskovich/anaconda2/lib/python2.7/lib-tk',
 '/Users/glennraskovich/anaconda2/lib/python2.7/lib-old',
 '/Users/glennraskovich/anaconda2/lib/python2.7/lib-dynload',
 '/Users/glennraskovich/anaconda2/lib/python2.7/site-packages',
 '/Users/glennraskovich/anaconda2/lib/python2.7/site-packages/aeosa']

But when running this in jupyter notebook I get:

['',
 '/usr/local/lib/python2.7/site-packages/dask-0.11.0-py2.7.egg',
 '/usr/local/lib/python2.7/site-packages/networkx-1.11-py2.7.egg',
 '/usr/local/lib/python2.7/site-packages/six-1.10.0-py2.7.egg',
 '/usr/local/lib/python2.7/site-packages/Pillow-3.3.1-py2.7-macosx-10.11-x86_64.egg',
 '/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python27.zip',
 '/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7',
 '/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin',
 '/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac',
 '/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages',
 '/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk',
 '/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old',
 '/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload',
 '/usr/local/lib/python2.7/site-packages',
 '/Library/Python/2.7/site-packages',
 '/usr/local/lib/python2.7/site-packages/IPython/extensions',
 '/Users/glennraskovich/.ipython']

For the sys.executable,

p(sys.executable)

In python, correct output:

/Users/glennraskovich/anaconda2/bin/python

But in jupyter notebook, sys.executable is not set to the anaconda version

/usr/local/opt/python/bin/python2.7

I've tried setting PATH in my .bashrc and .bash_profile, and using the commands which python, which jupyter and such show anaconda paths but jupyter notebook is not using the anaconda paths. What could be the issue here?

Apices

You are confusing sys.path for the notebook server and sys.path for the kernel you are running. – Bush

So is it the sys.path for the kernel I need to change somehow? How can I fix this? – Apices

Answer 1

I figured out the solution, since the kernel was set to use the default mac os x's python I fixed it by using the commands

python2 -m pip install ipykernel

python2 -m ipykernel install --user

Apices

Note: the --user in the second step is important. I tried the second without --user and with sudo instead, and it didn't work because it put the kernelspec into a different directory (on Windows 10, Ubuntu over WSL). – Five

I can verify this fixes the issue on virtualenv as well. Used it with the activated venv "python". The ONLY solution that worked for me. Thanks @Glenn Raskovich – Womanlike

Answer 2

For me, I installed Jupyter after creating an environment, but then was trying to run a module installed from the base env. I found by "jupyter kernelspec list" (https://github.com/jupyter/notebook/issues/2563), my kernel.json at C:\Users\username\Anaconda37\share\jupyter\kernels\python3\kernel.json was pointing to the python.exe in my working env. Changed the path and solved it.

This was an exhaustive description of python path setting.

Anticoagulant

Answer 3

My experience:

I had a conda virtual environment in Windows, and I had linked it as a kernel in Jupyter Notebook. However, the sys.executable was pointing at the global Python installation executable.

Performing the steps in @Glenn's answer didn't help. To solve it:

1. Find the executable python.exe for your environment. Mine was inside C:\Users\youruser\miniconda3\envs\my_conda_environment\python.exe.
2. Find the kernel.json configuration file for your environment inside C:\Users\youruser\AppData\Roaming\jupyter\kernels\yourkernel\kernel.json
3. Modifiy the kernel.json configuration path, replacing the existing executable path by the one from step 1.

Restart jupyter and it's done!

Sylvanite

Answer 4

I had this problem when I used Anaconda Navigator and the command line. I typed 'source activate ' into the console and then used Anaconda Navigator to open Jupyter. In Anaconda Navigator, however, I wasn't in the right environment which caused the problem. This is because Anaconda Navigator comes with its own activation for virtual environments (when you click on them). So you either need to activate the virtual environment from the console and then start Jupyter from the console or you need to activate the virtual environment in Anaconda Navigator and start Jupyter from the Navigator. Both ways work but not in combination. :-)

Earplug