Problem using py2app with the lxml package

Question

I am trying to use 'py2app' to generate a standalone application from some Python scripts. The Python uses the 'lxml' package, and I've found that I have to specify this explicitly in the setup.py file that 'py2app' uses. However, the resulting application program still won't run on machines that haven't had 'lxml' installed.

My Setup.py looks like this:

from setuptools import setup

OPTIONS = {'argv_emulation': True, 'packages' : ['lxml']}

setup(app=[MyApp.py],
      data_files=[],
      options={'py2app' : OPTIONS},
      setup_requires=['py2app'])

Running the application produces the following output:

MyApp Error
An unexpected error has occurred during execution of the main script
ImportError: dlopen(/Users/ake/XXXX/XXXX/MyApp.app/Contents/Resources/lib/python2.5/lxml/etree.so, 2): Symbol not found: _xmlSchematronParse
Referenced from: /Users/ake/XXXX/XXXX/MyApp.app/Contents/Resources/lib/python2.5/lxml/etree.so
Expected in: dynamic lookup

The symbol '_xmlSchematronParse' is from a library called 'libxml2' that 'lxml' depends on. The version that comes preinstalled with Mac OS X isn't up to date enough for 'lxml', so I had to install version 2.7.2 (in /usr/local). py2app, for some reason, is linking in the version in /Developer/SDKs/MacOSX10.3.9.sdk/usr/lib. When I run my application as a Python script though, the correct version is found. (I checked this just now by hiding the 2.7.2 version.)

So my question now is, how can I tell py2app where to look for libraries?

Ablaut

Do you find libxml2.2.dylib in the Contents/Frameworks subdir of your app ? – Riba

Answer 1

Found it. py2app has a 'frameworks' option to let you specify frameworks, and also dylibs. My setup.py file now looks like this:

from setuptools import setup

DATA_FILES = []
OPTIONS = {'argv_emulation': True,
           'packages' : ['lxml'],
           'frameworks' : ['/usr/local/libxml2-2.7.2/lib/libxml2.2.7.2.dylib']
          }

setup(app=MyApp.py,
      data_files=DATA_FILES,
      options={'py2app' : OPTIONS},
      setup_requires=['py2app'])

and that's fixed it.

Thanks for the suggestions that led me here.

Ablaut

On Mac OS X, libxml is located in /usr/lib/libxml2.2.dylib incase anybody's wondering – Bespangle

@Bespangle that is system, if you create py2app against another distribution (e.g. Homebrew), it is safer to package from that distribution. – Groundless

Thank you so much for answering your own question <3 – Leitman

Answer 2

------------- Edit--------------
libxml2 is standard in the python.org version of Python. It is not standard in Apple's version of Python. Make sure py2app is using the right version of Python, or install libxml2 and libxslt on your Mac.

Acetone

libxml2 is not a python package but a shared library. – Riba

libxml2 is not included in Apple's Python distro. – Acetone

Answer 3

I have no experience with the combination of lxml and py2app specifically, but I've had issues with py2app not picking up modules that were not explicitly imported. For example, I had to explicitly include modules that are imported via __import__(), like this:

OPTIONS['includes'] = [filename[:-3].replace('/', '.') for filename \
    in glob.glob('path/to/*.py')]

Maybe it also helps in your case to insert an explicit from lxml import etree somewhere in your code?

Kevel

Answer 4

I just tried my app (uses py2app and lxml, with a similar setup) on another Mac without development libraries installed, and it works, so there must be something wrong in your system. My guess is that py2app picks the wrong version of libxml2 (I see it comes bundled with the iPhone SDK for example, which is probably not the version you want).

Mine, as the whole python toolchain, comes from MacPorts, md5 sum of the latest libxml2.2.dylib (the one that ends up in my .app) is 863c7208b6c34f116a2925444933c22a on my system.

Riba