SSLError in Requests when packaging as OS X .app

Question

I'm developing an application for OS X. The application involves communicating with a server through python-requests, using a secure connection.

I am able to run the python file I intend to package, and it succeeds with the SSL connection. However, when I package the file with py2app and try to run it, I get the following error:

Traceback (most recent call last):
File "/Users/yossi/Documents/repos/drunken-octo-nemesis/dist/drunken-octo.app/Contents/Resources/__boot__.py", line 338, in <module>
    _run()
File "/Users/yossi/Documents/repos/drunken-octo-nemesis/dist/drunken-octo.app/Contents/Resources/__boot__.py", line 333, in _run
    exec(compile(source, path, 'exec'), globals(), globals())
File "/Users/yossi/Documents/repos/drunken-octo-nemesis/dist/drunken-octo.app/Contents/Resources/media_test.py", line 16, in <module>
    cmpbl.syncWithCloud()
File "src/compare_book_lists.pyc", line 172, in syncWithCloud
File "src/compare_book_lists.pyc", line 64, in checkMediaOnCloud
File "src/get_cloud_book_list.pyc", line 26, in getCloudFulfilledBookList
File "requests/api.pyc", line 55, in get
File "requests/api.pyc", line 44, in request
File "requests/sessions.pyc", line 354, in request
File "requests/sessions.pyc", line 460, in send
File "requests/adapters.pyc", line 250, in send
requests.exceptions.SSLError: [Errno 185090050] _ssl.c:340: error:0B084002:x509 certificate routines:X509_load_cert_crl_file:system lib
2013-06-12 11:39:49.119 drunken-octo[1656:707] drunken-octo Error

I was able to package part of my application successfully. The problem begins when the target file depends, somewhere in the chain, on Requests.

I am using zc.buildout to organize my imports. Therefore, I am running in a local python interpreter created by the buildout, so any fixes, unfortunately, will be easier to implement if they don't involve modifying the system Python. However, all suggestions are welcome, and I'll do my best to modify them for my specifics.

This only happens when I run the packaged app. Any ideas?

Mano

Answer 1

The easiests workaround is to add an option for py2app to your setup.py file:

setup(
   ...
   options={
      'py2app':{
          'packages': [ 'requests' ]
       }
   }
)

This includes the entire package into the application bundle, including the certificate bundle.

I've filed an issue for this in my py2app tracker, a future version of py2app will include logic to detect the use of the request library and will copy the certificate bundle automaticly.

Sweated

Hi Ronald, this workaround no longer seems to work. Is there any news on the update to py2app? Thanks! – Misdoubt

Answer 2

Requests uses a bundle of ceritificates to verify a servers identity. This bundle is kept (it has to be) in an independent file. Normally requests ships with its own bundle, but if packaged into a single file the bundle is lost. You can ship a new bundle alongside your app or let requests use the systemwide certificate. (I don't know, where OS X keeps this file, but on my linux box its /etc/ssl/certs/ca-certificates.crt)

To see, where requests expects the file you can do this:

import requests
print(requests.certs.where())

To change the location, where requests looks for the bundle you can pass the verify-parameter with a string value:

import requests
requests.get("https://httpbin.org/", verify="path/to/your/bundle")

If you don't want to pass the parameter every time, create a session and configure it to use your bundle:

import requests
s = requests.Session()
s.verify = "path/to/your/bundle"
s.get("https://httpbin.org")

Kalynkam

Answer 3

The previous accepted answer did not work for me - maybe the way requests works has changed.

To resolve this I changed my setup.py options to include the certifi package where the certificate pem file lives:

OPTIONS = {'argv_emulation': True,'packages': ['certifi']}

Then added this to the python requests calls:

is_py2app = hasattr(sys, "frozen")
pem_path = "lib/python2.7/certifi/cacert.pem" if is_py2app else None 

...

requests.get(..., verify=pem_path)

This may be different on other Python versions.

Shala

Yes, this was an important step for my package too. Thank eAi – Ludwig

FYI by default my requests package was using the certifi package for the cacert.pem rather than its own, but certifi was in site-package.zip rather than in an open folder. An alternative might be pem_path = "lib/python2.7/requests/cacert.pem" if is_py2app else None to use the one packaged with requests (not needing certifi to be added). But it makes little difference I imagine. – Ludwig

Hi @Shala I am sorry if this is a silly question. But what is sys in the line is_py2app = hasattr(sys, "frozen")? – Misdoubt

Answer 4

I ran into the same problem and had to distribute my application to users who may not have Python or the certifi package installed on their Mac. Drawing on inspirations from the answers here, I came up with the following solution.

Step 1: Download a OpenSSL package from https://www.openssl.org/source/. Find /openssl-1.0.2n/certs/demo/ca-cert.pem and place it under the same directory as your Python program (for example main.py).

Step 2: Create a setup.py as usual, but include ca-cert.pem in the DATA_FILES list. So your setup.py should look something like this:

from setuptools import setup

APP = ['main.py']
DATA_FILES = ['ca-cert.pem']
OPTIONS = {'argv_emulation': False}

setup(
    app=APP,
    data_files=DATA_FILES,
    options={'py2app': OPTIONS},
    setup_requires=['py2app'],
)

Step 3: Use the verify parameter so requests will use the certificate file you provide.

import requests
requests.get("https://httpbin.org/", verify="ca-cert.pem")

Alternatively, you can also create a session so that you don't have to specify verify every time.

import requests
s = requests.Session()
s.verify = "ca-cert.pem"
s.get("https://httpbin.org")

Step 4: Package the application using py2app as usual. The resulting app should be able to run normally.

python setup.py py2app

Misdoubt