How do I import a Python lambda layer?

Question

I have a file with this as the contents.

def print_hello_world():
    print ('Hello World')

It's zipped up in a folder with a __init__.py file.

I add this as the layer and set the correct runtime of python3.6.

How do import it into my lambda function via lambda code?

Edit: After researching I found that lambda mounts the layers at /opt and if you add /opt to your syspath via sys.path.insert(0, '/opt') then you can import your layers.

You can see your layers at: print(os.listdir("/opt"))

There's gotta be a more elegant way to do this!

Collier

Check here and here. – Illailladvised

Saw both of those. The AWS docs are 100% cli based. The article was just wrong(to put it bluntly). – Collier

I'm using cloudformation. I found that adding sys.path.insert(0, '/opt') allows me to import my layers. After identifying the location of my layer with print(os.listdir("/opt")). – Collier

There's gotta be a better way though right? – Collier

Answer 1

SOLUTION

You must to allocate all the packages you install with pip install ... into a zip file that ends with the folllowing file structure:

[your zip file].zip
|_python/
  |_all.../
  |_your.../
  |_packages, modules.../

Another boring way is:

[your zip file].zip
|_python/
  |_lib/
    |_python3.x/
      |_site-packages/
        |_all.../
        |_your.../
        |_packages, modules.../

with this structure you lambda code get the modules you have.

I hope this make done your issue.

Blackguardly

Answer 2

So I've recently ran into this issue, and I believe I found a cleaner way to import your layers.

First for the structure of the zip file which you upload:

• You do not need an __init__.py file
• Put all the scripts which you want to import into a folder name python
• Zip up that python folder (choose any name you want) and upload it to your layer
• Once uploaded, and the layer has been configured in your lambda function, you can simply use it with import {filename}

So if your script in the python folder was called something like custom_helper.py, import it in your lambda with import custom_helper.

I am not sure if this is the clean way to do it, but it seems simple enough to start.

Beaulieu

Answer 3

Your zip file should have the following structure:

python/lib/python3.7/site-packages

That is, it needs a folder named Python, and within that a folder named lib, and within that a folder named python3.7, and within that a folder named site-packages. Anything inside that folder will be available for import.

(If you're using another version of Python, that version should be in the path instead of 3.7)

Cloutier

Answer 4

You'll have to:

1. Build you lambda layer using Docker: sudo docker run -v "$PWD":/var/task "lambci/lambda:build-${penv}" /bin/sh -c "pip install -r requirements.txt -t python/lib/${penv}/site-packages/; exit"
2. Upload layer to AWS
3. Add layer to your lambda
4. Modity sys.path in your Lambda before import

    import sys; 
    sys.path.append('/opt/python/lib/python3.7/site-packages');
    import apsw
    print(apsw.__file__)

/opt/python/lib/python3.7/site-packages/apsw.cpython-37m-x86_64-linux-gnu.so

Update: It looks like you need to place build under /opt/python so you can use it without sys.path.append('/opt/python/lib/python3.7/site-packages');

New build steps:

mkdir -vp my-layer/python && cd my-layer/python
python3 -m pip install click -t ./ 
cd ..
zip -r9 my-layer python

now add layer to lambda function and you can import:

import click

Bledsoe