Cannot install kenlm package in anaconda environment

Question

When trying to install the python wrapper of kenlm from pip within an anaconda environment, I get the error:

(lm_1b) adamg:lm_1b adamg$ pip install https://github.com/kpu/kenlm/archive/master.zip
Collecting https://github.com/kpu/kenlm/archive/master.zip
  Cache entry deserialization failed, entry ignored
  Downloading https://github.com/kpu/kenlm/archive/master.zip
     - 4.4MB 51.1MB/s
Installing collected packages: kenlm
  Running setup.py install for kenlm ... error
    Complete output from command /Users/adamg/anaconda2/envs/lm_1b/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/2l/hd8b8vx566ld71lfd8hjglbc0000gn/T/pip-9yzty3hd-build/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/2l/hd8b8vx566ld71lfd8hjglbc0000gn/T/pip-aff_d2b8-record/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_ext
    building 'kenlm' extension
    creating build
    creating build/temp.macosx-10.9-x86_64-3.6
    creating build/temp.macosx-10.9-x86_64-3.6/util
    creating build/temp.macosx-10.9-x86_64-3.6/lm
    creating build/temp.macosx-10.9-x86_64-3.6/util/double-conversion
    creating build/temp.macosx-10.9-x86_64-3.6/python
    clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/adamg/anaconda2/envs/lm_1b/include -arch x86_64 -I/Users/adamg/anaconda2/envs/lm_1b/include -arch x86_64 -I. -I/Users/adamg/anaconda2/envs/lm_1b/include/python3.6m -c util/pool.cc -o build/temp.macosx-10.9-x86_64-3.6/util/pool.o -O3 -DNDEBUG -DKENLM_MAX_ORDER=6 -std=c++11
    In file included from util/pool.cc:1:
    In file included from ./util/pool.hh:4:
    /Users/adamg/anaconda2/envs/lm_1b/bin/../include/c++/v1/cassert:21:10: fatal error: 'assert.h' file not found
    #include <assert.h>
             ^~~~~~~~~~
    1 error generated.
    error: command 'clang' failed with exit status 1

    ----------------------------------------
Command "/Users/adamg/anaconda2/envs/lm_1b/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/2l/hd8b8vx566ld71lfd8hjglbc0000gn/T/pip-9yzty3hd-build/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/2l/hd8b8vx566ld71lfd8hjglbc0000gn/T/pip-aff_d2b8-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/2l/hd8b8vx566ld71lfd8hjglbc0000gn/T/pip-9yzty3hd-build/

The pip command works outside a conda environment, but then kenlm is not active within the environment. I was also able to run this from an AWS ec2 instance running linux, so maybe it's a Mac OSX issue. Any idea how can this be solved?

Selaginella

Try running the pip install from inside Anaconda prompt – Groupie

That's what I've tried, and what leads to this error – Selaginella

As I mentioned in the answer, it works fine outside of a conda environment. – Selaginella

Are you sure kenlm works with python2? – Groupie

Installing from Anaconda Prompt worked for me on anaconda3 and windows – Groupie

Perhaps the error relates to availability of c compoler on a system? – Ostiary

Can you elaborate on that, and how I can check and correct it? – Selaginella

conda create -n condapy27 python=2.7 anaconda, this will install more library for the conda env, it works for both python2 and 3 – Pimento

@Pimento Doesn't that just create a new environment? – Selaginella

Answer 1

On Mac usually there is a problem on clang with different libraries that are missing. In most of this cases install Xcode will solve the problem. Also, in case this does not, you can also run in terminal xcode-select --install
Hope this helps!

Sauncho

Yup, that did it! Thanks – Selaginella