python c extension, problems with dlopen on mac os

Question

I've taken a library that is distributed as a binary lib (.a) and header, written some c++ code against it, and want to wrap the results up in a python module.

I've done this here.

The problem is that when importing this module on Mac OSX (I've tried 10.5 and 10.6), I get the following error:

dlopen(/Library/Python/2.5/site-packages/dirac.so, 2): Symbol not found: _DisposePtr
  Referenced from: /Library/Python/2.5/site-packages/dirac.so
  Expected in: dynamic lookup

This looks like symbols defined in the Carbon framework aren't being properly resolved, but I'm not sure what to do about that. I am supplying -framework Carbon to distutil.core.Extension's extra_link_args parameter, so I'm not sure what else I should do.

Any help would be much appreciated.

Update:

The compile line generated by setup.py looks like this:

gcc -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -Os -Wall -Wstrict-prototypes -DMACOSX -I/usr/include/ffi -DENABLE_DTRACE -arch i386 -arch ppc -pipe -Isource -I/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/numpy/core/include -I/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/numpy/numarray -I/usr/lib/python/2.5/site-packages/numpy/numarray/numpy -I/usr/lib/python/2.5/site-packages/numpy/numarray -I/usr/lib/python/2.5/site-packages/numpy/core/include -I/System/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 -c source/Dirac_LE.cpp -o build/temp.macosx-10.5-i386-2.5/source/Dirac_LE.o

The linker line looks like this:

g++ -Wl,-F. -bundle -undefined dynamic_lookup -arch i386 -arch ppc build/temp.macosx-10.5-i386-2.5/diracmodule.o build/temp.macosx-10.5-i386-2.5/source/Dirac_LE.o -Llibs/MacOSX -lDiracLE -o build/lib.macosx-10.5-i386-2.5/dirac.so -framework Carbon

otool reports:

dirac.so:
 /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.4.0)
 /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.1.5)

Update 2: On MacOS 10.5, modifying the dlopen flags from the default of RTLD_NOW to RTLD_LAZY solves the problem. However, this does not work on Mac OS 10.6.

On 10.6, the following sequence allows the library to run properly, although I'm not sure why:

1. python setup.py build -v
2. run the linker line (printed to console by setup.py) again, manually.
3. python setup.py install

I'm still looking for a good answer as to how to get this to work properly. Thanks!

Scission

What is the actual compiler commandline that setup.py executes? Delete the build directory and run setup.py build -v to see. Also, what does otool -L say about the dirac.so file? – Radically

@Thomas, I've updated the question with that information, thanks. – Scission

That looks alright then; the -framework argument is in the proscribed place. The only things I can imagine is that you need a different framework, or that the framework is supposed to introduce a shlib dependency and somehow isn't (I don't know if the Carbon framework is supposed to do that or not.) – Radically

@Thomas how do I check for a shlib dependency? – Scission

That's what otool -L does. I don't know how to check the framework configuration, sorry. – Radically

Can you create a dynamic library using Xcode's template for that (the file ends with .dylib)? I can offer advice on how to wrap that with a Python class... – Goethe

Isn't the python interpreter compiled and linked with the C compiler? According to docs.python.org/extending/… you have to declare your extension's callable functions with extern "C" if they are written in C++. – Transplant

Answer 1

You're going to kick yourself when you see the answer to this! Try changing this:

link_args = ['-framework Carbon'] if platform == 'Darwin' else []

to this:

link_args = ['-framework', 'Carbon'] if platform == 'Darwin' else []

Once I made this change I was able to do a clean build and import the module straight away :)

Cagliari