How to install snappy C libraries on Windows 10 for use with python-snappy in Anaconda?

Question

I want to install parquet for python using pip within an Anaconda 2 installation on Windows 10.

While installing I ran into the error that is described here, the installer can't find snappy-c.h.

There is no mention on how to install this on Windows in the answers.

I downloaded the Snappy library from http://google.github.io/snappy/ and now I'm stuck.

From my error message I would have assumed that the header files need to be in C:\Users\...\AppData\Local\Continuum\Anaconda2\include, but in the downlaoded archive header and library files are just all in the same folder.

How do I install these properly in the Anaconda folder?

Full error message:

Building wheels for collected packages: python-snappy
  Running setup.py bdist_wheel for python-snappy ... error
  Complete output from command C:\Users\...\AppData\Local\Continuum\Anaconda2\python.exe -u -c "import setup
tools, tokenize;__file__='c:\\users\\...\\appdata\\local\\temp\\pip-build-kl4zef\\python-snappy\\setup.py';f=ge
tattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec
'))" bdist_wheel -d c:\users\...\appdata\local\temp\tmpt8fz9bpip-wheel- --python-tag cp27:
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build\lib.win-amd64-2.7
  copying snappy.py -> build\lib.win-amd64-2.7
  running build_ext
  building '_snappy' extension
  creating build\temp.win-amd64-2.7
  creating build\temp.win-amd64-2.7\Release
  C:\Users\...\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Bin\amd64\cl.exe /c /nol
ogo /Ox /MD /W3 /GS- /DNDEBUG -IC:\Users\...\AppData\Local\Continuum\Anaconda2\include -IC:\Users\...
\AppData\Local\Continuum\Anaconda2\PC /Tpsnappymodule.cc /Fobuild\temp.win-amd64-2.7\Release\snappymodule.obj
  snappymodule.cc
  snappymodule.cc(31) : fatal error C1083: Cannot open include file: 'snappy-c.h': No such file or directory
  error: command 'C:\\Users\\...\\AppData\\Local\\Programs\\Common\\Microsoft\\Visual C++ for Python\\9.0\\V
C\\Bin\\amd64\\cl.exe' failed with exit status 2

Phalan

Answer 1

After a really long and frustrating time trying different methods to get the C files working, I found this site:

http://www.lfd.uci.edu/~gohlke/pythonlibs/

Where C. Gohlke has kindly compiled and uploaded the files for us windows users.

Just download the version you need based on your system and python version. I am using 64bits and python 3.6, so I used the following command from my download folder:

pip install python_snappy-0.5-cp36-cp36m-win_amd64.whl

It worked like a charm :)

Johnsson

I'm awarding you the best answer after I ran into big problems both upgrading my snappy installation and doing a new installation on Python 3. After some trouble I just did it your way and it went without effort. – Phalan

Thanks! Glad it was of some help for you. It surely was for me! – Johnsson

This also works with pipenv. Pycharm was running into errors and now it´s solved. – Heterochromous

Hi I've managed to install it Installing collected packages: python-snappy Successfully installed python-snappy-0.5.4 but when I execute the script in PyCharm, seems like it's still giving the same error RuntimeError: Decompression 'SNAPPY' not available. Options: ['GZIP', 'UNCOMPRESSED'] Am I missing anything? – Hermaherman

(In PyCharm, it still indicates python-snappy is not installed) Am I missing anything? – Hermaherman

@Johnsson I've raised my quetion separately, could you take a look please: #65109944 Many thanks. – Hermaherman

The link is dead as of today, does someone know we can find it? – Baptiste

lfd.uci.edu/~gohlke/pythonlibs working accessed today – Johnsson

Answer 2

It seems there are now Conda packages that should simplify things (in Anaconda). I was able to do (on Python 3.6, Windows 10):

conda install -c conda-forge snappy
conda install -c conda-forge python-snappy

Dotterel

Or do conda install -c conda-forge snappy and then pip install python-snappy. Making sure you conda activate your-env-name first. – Internationale

Answer 3

EDIT: Refer to the other answers in this thread, I only keep this up for reference.

Here's the steps it took me to install Snappy and Python-Snappy on Windows 10:

1. Install cygwin.
2. Download snappy from http://google.github.io/snappy/ and unpack it somewhere into the cygwin directory.
3. Write AM_PROG_AR into configure.ac above LT_INIT and run autogen.sh from the cygwin terminal, install missing cygwin packages if needed.
4. Copy snappy-c.h into the Anaconda2/include folder and libsnappy.a into the Anaconda2/Lib folder (in ~/AppData/Local/Continuum/).
5. stdint.h was missing for Visual C++ Compiler for Python 2.7., I found it here and put it into ~\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\include.
6. Installing python-snappy should now work without throwing any errors.
7. If you get problems and weird imports check if the SnapPy library is also installed and deinstall it because it causes a conflict.

(Everything works fine now.)

EDIT: No it does not. This method is not reliable, once you try to update you run into new problems. And installing on Python 3 is another monster that isn't tackled by this solution.

Phalan

Answer 4

I have same problem with python-snappy on Puthon 3.11/3.12. I just deleted 3.11/3.12 versions and installed 3.6.8 version and then python-snappy installed on my PC

Python 3.6.8: https://www.python.org/downloads/release/python-368/ Delede actual version of Python Installing python-snappy: pip install python_snappy

Waine

Hi. We may assume OP knew how to update Python if needed, so an answer should rather focus on how to instal python-snappy and from what source? – Dustcloth