How to install cffi package on AWS Beanstalk

Question

This question looks the same as this post, but since there was no answer, I am re-asking here.

I have a Django project to be deployed on AWS Beanstalk, which is using a package cffi. When I run eb deploy, the error logs looks like:

  ......
  Installing collected packages: cffi, characteristic, idna, pyasn1, enum34, ipaddress, cryptography, cssselect, Django, django-bower, django-celery, sqlparse, django-debug-toolbar, django-devserver, django-extensions, django-filter, django-model-utils, futures, django-pipeline, djangorestframework, html5lib, ipython, ipdb, librabbitmq, lxml, Markdown, openxmllib, Pillow, psycopg2, py, pyasn1-modules, PyDocX, pyOpenSSL, pytest, python-docx, python-magic, queuelib, w3lib, zope.interface, Twisted, Scrapy, service-identity, South, uWSGI, Werkzeug
Running setup.py install for cffi
  Complete output from command /opt/python/run/venv/bin/python2.7 -c "import setuptools, tokenize;__file__='/tmp/pip-build-WWakYN/cffi/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-PNAk7u-record/install-record.txt --single-version-externally-managed --compile --install-headers /opt/python/run/venv/include/site/python2.7/cffi:
  Package libffi was not found in the pkg-config search path.
  Perhaps you should add the directory containing `libffi.pc'
  to the PKG_CONFIG_PATH environment variable
  No package 'libffi' found
  Package libffi was not found in the pkg-config search path.
  Perhaps you should add the directory containing `libffi.pc'
  to the PKG_CONFIG_PATH environment variable
  No package 'libffi' found
  Package libffi was not found in the pkg-config search path.
  Perhaps you should add the directory containing `libffi.pc'
  to the PKG_CONFIG_PATH environment variable
  No package 'libffi' found
  Package libffi was not found in the pkg-config search path.
  Perhaps you should add the directory containing `libffi.pc'
  to the PKG_CONFIG_PATH environment variable
  No package 'libffi' found
  Package libffi was not found in the pkg-config search path.
  Perhaps you should add the directory containing `libffi.pc'
  to the PKG_CONFIG_PATH environment variable
  No package 'libffi' found
  running install
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-2.7
  creating build/lib.linux-x86_64-2.7/cffi
  copying cffi/model.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/__init__.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/ffiplatform.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/cffi_opcode.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/vengine_cpy.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/cparser.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/api.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/setuptools_ext.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/commontypes.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/backend_ctypes.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/verifier.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/lock.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/recompiler.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/vengine_gen.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/gc_weakref.py -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/_cffi_include.h -> build/lib.linux-x86_64-2.7/cffi
  copying cffi/parse_c_type.h -> build/lib.linux-x86_64-2.7/cffi
  running build_ext
  building '_cffi_backend' extension
  creating build/temp.linux-x86_64-2.7
  creating build/temp.linux-x86_64-2.7/c
  gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DUSE__THREAD -I/usr/include/ffi -I/usr/include/libffi -I/usr/include/python2.7 -c c/_cffi_backend.c -o build/temp.linux-x86_64-2.7/c/_cffi_backend.o
  c/_cffi_backend.c:13:17: error: ffi.h: No such file or directory
  In file included from c/_cffi_backend.c:61:
  c/malloc_closure.h:81: error: expected specifier-qualifier-list before ‘ffi_closure’
  c/malloc_closure.h: In function ‘more_core’:
  c/malloc_closure.h:117: warning: division by zero
  c/malloc_closure.h:149: error: ‘union mmaped_block’ has no member named ‘next’
  c/malloc_closure.h: At top level:
  c/malloc_closure.h:158: error: expected ‘)’ before ‘*’ token
  c/malloc_closure.h:166: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘*’ token
  c/_cffi_backend.c:238: error: expected specifier-qualifier-list before ‘ffi_cif’
  c/_cffi_backend.c: In function ‘cdataowninggc_dealloc’:
  c/_cffi_backend.c:1573: error: ‘ffi_closure’ undeclared (first use in this function)
  c/_cffi_backend.c:1573: error: (Each undeclared identifier is reported only once
  c/_cffi_backend.c:1573: error: for each function it appears in.)
  c/_cffi_backend.c:1573: error: ‘closure’ undeclared (first use in this function)
  c/_cffi_backend.c:1573: error: expected expression before ‘)’ token
  c/_cffi_backend.c:1576: warning: implicit declaration of function ‘cffi_closure_free’
  c/_cffi_backend.c: In function ‘cdataowninggc_traverse’:
  c/_cffi_backend.c:1593: error: ‘ffi_closure’ undeclared (first use in this function)
  c/_cffi_backend.c:1593: error: ‘closure’ undeclared (first use in this function)
  c/_cffi_backend.c:1593: error: expected expression before ‘)’ token
  c/_cffi_backend.c: In function ‘cdataowninggc_clear’:
  c/_cffi_backend.c:1613: error: ‘ffi_closure’ undeclared (first use in this function)
  c/_cffi_backend.c:1613: error: ‘closure’ undeclared (first use in this function)
  c/_cffi_backend.c:1613: error: expected expression before ‘)’ token
  c/_cffi_backend.c: In function ‘cdataowninggc_repr’:
  c/_cffi_backend.c:1742: error: ‘ffi_closure’ undeclared (first use in this function)
  c/_cffi_backend.c:1742: error: expected expression before ‘)’ token
  c/_cffi_backend.c: At top level:
  c/_cffi_backend.c:2308: error: expected declaration specifiers or ‘...’ before ‘ffi_abi’
  c/_cffi_backend.c: In function ‘cdata_call’:
  c/_cffi_backend.c:2437: error: ‘ffi_abi’ undeclared (first use in this function)
  c/_cffi_backend.c:2437: error: expected ‘;’ before ‘fabi’
  c/_cffi_backend.c:2479: error: ‘fabi’ undeclared (first use in this function)
  c/_cffi_backend.c:2483: error: too many arguments to function ‘fb_prepare_cif’
  c/_cffi_backend.c:2488: error: ‘cif_description_t’ has no member named ‘exchange_size’
  c/_cffi_backend.c:2498: error: ‘cif_description_t’ has no member named ‘exchange_offset_arg’
  c/_cffi_backend.c:2528: error: ‘cif_description_t’ has no member named ‘exchange_offset_arg’
  c/_cffi_backend.c:2533: warning: implicit declaration of function ‘ffi_call’
  c/_cffi_backend.c:2533: error: ‘cif_description_t’ has no member named ‘cif’
  c/_cffi_backend.c: In function ‘new_primitive_type’:
  c/_cffi_backend.c:3524: error: ‘ffi_type’ undeclared (first use in this function)
  c/_cffi_backend.c:3524: error: ‘ffitype’ undeclared (first use in this function)
  c/_cffi_backend.c:3542: error: ‘ffi_type_sint8’ undeclared (first use in this function)
  c/_cffi_backend.c:3543: error: ‘ffi_type_sint16’ undeclared (first use in this function)
  c/_cffi_backend.c:3544: error: ‘ffi_type_sint32’ undeclared (first use in this function)
  c/_cffi_backend.c:3545: error: ‘ffi_type_sint64’ undeclared (first use in this function)
  c/_cffi_backend.c:3551: error: ‘ffi_type_float’ undeclared (first use in this function)
  c/_cffi_backend.c:3553: error: ‘ffi_type_double’ undeclared (first use in this function)
  c/_cffi_backend.c:3555: error: ‘ffi_type_longdouble’ undeclared (first use in this function)
  c/_cffi_backend.c:3561: error: ‘ffi_type_uint8’ undeclared (first use in this function)
  c/_cffi_backend.c:3562: error: ‘ffi_type_uint16’ undeclared (first use in this function)
  c/_cffi_backend.c:3563: error: ‘ffi_type_uint32’ undeclared (first use in this function)
  c/_cffi_backend.c:3564: error: ‘ffi_type_uint64’ undeclared (first use in this function)
  c/_cffi_backend.c: At top level:
  c/_cffi_backend.c:4200: error: expected specifier-qualifier-list before ‘ffi_type’
  c/_cffi_backend.c:4219: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘*’ token
  c/_cffi_backend.c: In function ‘fb_build’:
  c/_cffi_backend.c:4359: error: ‘struct funcbuilder_s’ has no member named ‘atypes’
  c/_cffi_backend.c:4359: error: ‘ffi_type’ undeclared (first use in this function)
  c/_cffi_backend.c:4359: error: expected expression before ‘)’ token
  c/_cffi_backend.c:4360: error: ‘struct funcbuilder_s’ has no member named ‘nargs’
  c/_cffi_backend.c:4363: error: ‘struct funcbuilder_s’ has no member named ‘rtype’
  c/_cffi_backend.c:4363: warning: implicit declaration of function ‘fb_fill_type’
  c/_cffi_backend.c:4371: error: ‘cif_description_t’ has no member named ‘exchange_offset_arg’
  c/_cffi_backend.c:4374: error: ‘struct funcbuilder_s’ has no member named ‘rtype’
  c/_cffi_backend.c:4375: error: ‘ffi_arg’ undeclared (first use in this function)
  c/_cffi_backend.c:4385: error: ‘atype’ undeclared (first use in this function)
  c/_cffi_backend.c:4398: error: ‘struct funcbuilder_s’ has no member named ‘atypes’
  c/_cffi_backend.c:4399: error: ‘struct funcbuilder_s’ has no member named ‘atypes’
  c/_cffi_backend.c:4402: error: ‘cif_description_t’ has no member named ‘exchange_offset_arg’
  c/_cffi_backend.c:4409: error: ‘cif_description_t’ has no member named ‘exchange_size’
  c/_cffi_backend.c: In function ‘fb_build_name’:
  c/_cffi_backend.c:4431: error: ‘struct funcbuilder_s’ has no member named ‘nargs’
  c/_cffi_backend.c:4440: error: ‘struct funcbuilder_s’ has no member named ‘fct’
  c/_cffi_backend.c:4442: error: ‘struct funcbuilder_s’ has no member named ‘fct’
  c/_cffi_backend.c: In function ‘fb_prepare_ctype’:
  c/_cffi_backend.c:4483: error: ‘struct funcbuilder_s’ has no member named ‘fct’
  c/_cffi_backend.c:4493: error: ‘struct funcbuilder_s’ has no member named ‘fct’
  c/_cffi_backend.c: At top level:
  c/_cffi_backend.c:4513: error: expected declaration specifiers or ‘...’ before ‘ffi_abi’
  c/_cffi_backend.c: In function ‘fb_prepare_cif’:
  c/_cffi_backend.c:4540: warning: implicit declaration of function ‘ffi_prep_cif’
  c/_cffi_backend.c:4540: error: ‘cif_description_t’ has no member named ‘cif’
  c/_cffi_backend.c:4540: error: ‘fabi’ undeclared (first use in this function)
  c/_cffi_backend.c:4540: error: ‘struct funcbuilder_s’ has no member named ‘nargs’
  c/_cffi_backend.c:4541: error: ‘struct funcbuilder_s’ has no member named ‘rtype’
  c/_cffi_backend.c:4541: error: ‘struct funcbuilder_s’ has no member named ‘atypes’
  c/_cffi_backend.c:4541: error: ‘FFI_OK’ undeclared (first use in this function)
  c/_cffi_backend.c: In function ‘new_function_type’:
  c/_cffi_backend.c:4585: error: too many arguments to function ‘fb_prepare_cif’
  c/_cffi_backend.c:4599: error: ‘struct funcbuilder_s’ has no member named ‘nargs’
  c/_cffi_backend.c:4609: error: ‘struct funcbuilder_s’ has no member named ‘nargs’
  c/_cffi_backend.c:4619: error: ‘struct funcbuilder_s’ has no member named ‘nargs’
  c/_cffi_backend.c:4622: error: ‘struct funcbuilder_s’ has no member named ‘nargs’
  c/_cffi_backend.c:4623: error: ‘struct funcbuilder_s’ has no member named ‘nargs’
  c/_cffi_backend.c:4625: error: ‘struct funcbuilder_s’ has no member named ‘nargs’
  c/_cffi_backend.c: In function ‘b_new_function_type’:
  c/_cffi_backend.c:4636: error: ‘FFI_DEFAULT_ABI’ undeclared (first use in this function)
  c/_cffi_backend.c: In function ‘convert_from_object_fficallback’:
  c/_cffi_backend.c:4655: error: ‘ffi_arg’ undeclared (first use in this function)
  c/_cffi_backend.c: At top level:
  c/_cffi_backend.c:4727: error: expected ‘)’ before ‘*’ token
  c/_cffi_backend.c: In function ‘b_callback’:
  c/_cffi_backend.c:4798: error: ‘ffi_closure’ undeclared (first use in this function)
  c/_cffi_backend.c:4798: error: ‘closure’ undeclared (first use in this function)
  c/_cffi_backend.c:4819: error: ‘ffi_arg’ undeclared (first use in this function)
  c/_cffi_backend.c:4837: warning: implicit declaration of function ‘cffi_closure_alloc’
  c/_cffi_backend.c:4855: warning: implicit declaration of function ‘ffi_prep_closure’
  c/_cffi_backend.c:4855: error: ‘cif_description_t’ has no member named ‘cif’
  c/_cffi_backend.c:4856: error: ‘invoke_callback’ undeclared (first use in this function)
  c/_cffi_backend.c:4856: error: ‘FFI_OK’ undeclared (first use in this function)
  In file included from c/cffi1_module.c:3,
                   from c/_cffi_backend.c:5973:
  c/realize_c_type.c: In function ‘realize_c_type_or_func’:
  c/realize_c_type.c:562: error: ‘FFI_DEFAULT_ABI’ undeclared (first use in this function)
  c/_cffi_backend.c: In function ‘init_cffi_backend’:
  c/_cffi_backend.c:6057: error: ‘FFI_DEFAULT_ABI’ undeclared (first use in this function)
  error: command 'gcc' failed with exit status 1

I can tell that AWS looks for libffi without success. However, how do I install it? I tried the workaround in the link above but not working either.

Matteo

sudo apt-get install libffi-dev on standard Linux. In amazon-web-services, you need to figure out how to do the same---which is not a Python question at all. – Feast

@ArminRigo True, and this is what I was asking. – Matteo

You may have more success if your question was formulated directly as "how do I install package X in amazon-web-services?". I'm pretty sure this question was already asked and answered here, too... – Feast

Answer 1

To install the cffi python package, you first need to install libffi-devel using yum:

sudo yum install libffi-devel

Which you can automate by adding it to your config file in the .ebextensions folder like this:

packages:
    yum:
        libffi-devel: []

This was mentioned in the other post, but since the search engine brought me here first and the other is marked as duplicate... I thought I should better reply here.

Hyperion

Same here. This still isn't documented on EB anywhere and you can't search the amazon packages without running an instance. – Elva

I tried putting this is a file called 01run.config in the .ebextensions folder and it did not fix the problem. – Guilty

added this to django.config - worked perfectly! Made my day! – Grenadines