M2Crypto doesn't install in venv, or swig doesn't define __x86_64__ which breaks compiling against OpenSSL

Question

I'm trying to install the Python M2Crypto package into a virtualenv on an x86_64 RHEL 6.1 machine. This process invokes swig, which fails with the following error:

$ virtualenv -q --no-site-packages venv
$ pip install -E venv M2Crypto==0.20.2
Downloading/unpacking M2Crypto==0.20.2
  Downloading M2Crypto-0.20.2.tar.gz (412Kb): 412Kb  downloaded
  Running setup.py egg_info for package M2Crypto
Installing collected packages: M2Crypto
  Running setup.py install for M2Crypto
    building 'M2Crypto.__m2crypto' extension
    swigging SWIG/_m2crypto.i to SWIG/_m2crypto_wrap.c
    swig -python -I/usr/include/python2.6 -I/usr/include -includeall -o SWIG/_m2crypto_wrap.c SWIG/_m2crypto.i
    /usr/include/openssl/opensslconf.h:31: Error: CPP #error ""This openssl-devel package does not work your architecture?"". Use the -cpperraswarn option to continue swig processing.
    error: command 'swig' failed with exit status 1
    Complete output from command /home/lorin/venv/bin/python -c "import setuptools;__file__='/home/lorin/venv/build/M2Crypto/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally-managed --record /tmp/pip-BFiNtU-record/install-record.txt --install-headers /home/lorin/venv/include/site/python2.6:

I've got OpenSSL 1.0.0 installed via RPM packages from RedHat.

The part of /usr/include/openssl/opensslconf.h that causes the error looks like this:

#if defined(__i386__)
#include "opensslconf-i386.h"
#elif defined(__ia64__)
#include "opensslconf-ia64.h"
#elif defined(__powerpc64__)
#include "opensslconf-ppc64.h"
#elif defined(__powerpc__)
#include "opensslconf-ppc.h"
#elif defined(__s390x__)
#include "opensslconf-s390x.h"
#elif defined(__s390__)
#include "opensslconf-s390.h"
#elif defined(__sparc__) && defined(__arch64__)
#include "opensslconf-sparc64.h"
#elif defined(__sparc__)
#include "opensslconf-sparc.h"
#elif defined(__x86_64__)
#include "opensslconf-x86_64.h"
#else
#error "This openssl-devel package does not work your architecture?"
#endif

gcc has the right variable defined:

$ echo | gcc -E -dM - | grep x86_64
#define __x86_64 1
#define __x86_64__ 1

But apparenty swig doesn't, since this is the line that's failing:

swig -python -I/usr/include/python2.6 -I/usr/include -includeall -o \
  SWIG/_m2crypto_wrap.c SWIG/_m2crypto.i

Is there a way to fix this by changing something in my system configuration? M2Crypto gets installed in a virtualenv as part of a larger script I don't control, so avoiding mucking around with the M2Crypto files would be a good thing.

Austerlitz

There is fedora_setup.sh but it requires mucking around with the M2Crypto files. – Pittance

@J.F.Sebastian: Not familiar with that, can you provide more details in a complete answer? – Austerlitz

Answer 1

M2Crypto supplies a fedora_setup.sh script to handle the problems with Fedora/RL/CentOs releases, but pip, of course, doesn't know anything about it.

After the pip install fails, it leaves the downloaded stuff in the venv/build/M2Crypto directory. do this:

cd <path-to-your-venv>/venv/build/M2Crypto
chmod u+x fedora_setup.sh
./fedora_setup.sh build
./fedora_setup.sh install

This has worked in my install process

Crassus

As of today (version 1.5.6) pip cleans up the build directory, so there's nothing to do therein anymore. – Linalool

Answer 2

You just don't have swig installed.

Try:

sudo yum install swig

And then:

sudo easy_install M2crypto

Outpour

On Mac OS X 10.9.4 I installed swig using MacPorts (sudo port install swig). I also had to install Python's markupsafe globally (sudo pip install markupsafe), because swig depends on it (.../markupsafe/_speedups.so: Error opening or reading file). – Linalool

Looks like with MacPorts we also need to install swig-python (sudo port install swig-python). – Linalool

This worked for me when I had a similar problem on Ubuntu (though of course using apt-get instead of yum). – Merca

Answer 3

I did this and it works very well :

env SWIG_FEATURES="-cpperraswarn -includeall -I/usr/include/openssl" pip install M2Crypto

Of course you have to install swigg with sudo yum install swig before

Skywriting

Answer 4

If you are seeing this and are on Ubuntu, use apt-get instead of pip to avoid this issue.
apt-get install python-m2crypto

Jipijapa

Answer 5

I had a similar issue where /usr/include/openssl was missing opensslconf.h (source https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=733644#10)

sudo ln -s /usr/include/x86_64-linux-gnu/openssl/opensslconf.h /usr/include/openssl

Blackburn

Answer 6

There's a repository where "pip install" works:

https://github.com/martinpaljak/M2Crypto

Leodora

You could try this: pip install git+github.com/martinpaljak/M2Crypto.git As I'm planning to wrap up some changes to M2Crypto that are floating around into new release(s) that will end up in pypi, you can safely use the same git tree at the moment. – Leodora

"pip install git+github.com/martinpaljak/M2Crypto.git" fails with "ValueError: ('Expected version spec in', 'git+github.com/martinpaljak/M2Crypto.git', 'at', '+github.com/martinpaljak/M2Crypto.git')" on my Ubuntu 10.04 system. – Sunward

The formatting. Do add https and colons and slashes in front of github.com – Leodora

Thanks for this, this is supremely helpful for keeping our build process sane. – Diwan

This doesn't work for a virtual environment on Ubuntu. – Poise

Which ubuntu? Will try in a sec. – Leodora

Try now. github.com/martinpaljak/M2Crypto/commit/… should fix it. – Leodora

Answer 7

sudo yum install m2crypto

worked for me to get around this problem.

Meaganmeager

Answer 8

I found a new way to fix this problem in centos5.8, try it.

vim setup.py

def finalize_options(self):
  ...
  self.swig_opts.append('-includeall') # after this line
  self.swig_opts.append('-I/usr/include/openssl') # add here

then python setup.py install will work.

Wolver

Answer 9

On FreeBSD I had to install Swig (the obvious part) as well (by sudo pkg install swig), but Swig 2.0 executable was named swig2.0 and handle swig resulted in command not found. Solution: symlink Swig 2.0 to handle swig:

ln -s /usr/local/bin/swig2.0 /usr/local/bin/swig

Granniah

Answer 10

It seems like not having swig is the problem, as @LeoC said.

For those on MacOS, I'd recommend downloading swig via a package manager like homebrew because it's cleaner.

I.e. you'd run

brew install swig

Extrados