AWS Lambda not importing Asyncio

Question

So I'm working on making an application for making Reddit API calls and it seems like Lambda isn't liking how I import asyncio. I installed asyncio in a package folder using "pip install asyncio -t ." and then zipped that folder up with my project file. And I'm importing asyncio in the project file (import asyncio). However, every time I try to test my Alexa application in the Alexa Developer Console, the application won't run until I've gotten rid of the import statement.

Here's the message I get when I try to test:

{ "errorMessage": "Syntax error in module 'reddit_alexa_py': invalid syntax (base_events.py, line 296)", "errorType": "Runtime.UserCodeSyntaxError", "stackTrace": [ " File \"/var/task/asyncio/base_events.py\" Line 296\n future = tasks.async(future, loop=self)\n" ] }

and here's the log output:

START RequestId: ee952162-1d06-4c04-9a0d-cfd4f0fce80f Version: $LATEST [ERROR] Runtime.UserCodeSyntaxError: Syntax error in module 'reddit_alexa_py': invalid syntax (base_events.py, line 296) Traceback (most recent call last):   File "/var/task/asyncio/base_events.py" Line 296             future = tasks.async(future, loop=self)END RequestId: ee952162-1d06-4c04-9a0d-cfd4f0fce80f REPORT RequestId: ee952162-1d06-4c04-9a0d-cfd4f0fce80f Duration: 19.71 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 57 MB

This is happening with other things I try to import, too. I think I'll need to import these packages in order to finish this project, so any help would be greatly appreciated. Thanks!

EDIT: here's a link to the Python file https://drive.google.com/file/d/1_rbNLlwRBmt_6J0YMMa4idPSfmOKZb_j/view?usp=sharing

Ieper

Could you publish your code? Also, I don't think Python will import modules from zip files... why do you need to zip the module? Is it possible to run pip install asyncio in the Alexa Developer Console? – Exhibitioner

@EdWard I watched a tutorial online on how to add deployment packages to an AWS Lambda function's code. Here's a link to the video I used: youtube.com/watch?v=rDbxCeTzw_k. I also added a link to my Python file in the description. Every time I try to import asyncio, the Alexa test automatically breaks. – Ieper

Answer 1

I'm on python 3.8 and had my program running great locally, but when packaged it up and added it to a lambda function I was getting the exact same error. Turns out, asyncio is part of python, but if you don't know that and do a pip to install it like I did, you'll get a second copy. I used this second copy in a layer and kept getting the error. After trying a LOT of different variants/versions/etc, I realized that the version included with python is completely different from the one pip installed. The solution is simple, make sure you do NOT try to pip asyncio, just let lambda pick it up from the python version you are using and make sure that you don't have a second (different) version somewhere in your search path...

Samira

Thanks, I had to manually remove asyncio after build and it worked. – Lindbom

Answer 2

DO NOT include asyncio in your requirements because it is already part of stdlib.

Stylolite

removing asyncio solved the issue for me (on Azure functions). – Judge

Golden general answer also useful in other contexts (pycharm/dataspell) - thank you! Pip should prevent this? – Schrick

Answer 3

I used a similar process. I had to copy my imported module into a subdirectory of my working directory before zipping everything up. After loading the zip file into the Lambda function, I was able to verify that the code I wanted to import was loaded by looking at the function in the Lambda console.

The key for me was that, on my local computer, I had to find the psycopg2 directory and copy that whole directory into my working directory where I was writing postgresql_test.py before zipping up my working directory.

Perron

If you don't mind me asking, where did you find the psycopg2 folder? – Ieper

I don't remember for certain, but in my project notes I pointed to linkedin.com/pulse/… so I think I followed the git clone and cp instructions on that page. – Perron

Answer 4

I had the same problem. Uploading packaged to Lambda Layer will upload it to /opt/.

When Packaged is compressed into the python folder, it is uploaded to the /opt/python folder, and the default path of the module imported from the layer set in lambda is /opt/python.

However, the asyncio module should not be uploaded to the layer by putting it in the python folder and compressing it according to the aws guide. I don't know exactly why.

To avoid this error, install the pip modules in a different folder (not the python folder) and compress them. (Example: ./python-modules)

After uploading(python-modules.zip) to layer and writing as below in lambda, no problem occurs.

import sys
sys.path.append("/opt/python-modules")
import asyncio

Not enough, but thanks for reading.

Carousel

Answer 5

I'm not sure about asyncio, but for python packages in general you need to ensure that your packages are built for Amazon Linux 1 (for python 3.7 runtimes) and Amazon Linux 2 (for python 3.8 runtimes). Under the hood, Lambda uses containers running Amazon Linux, and you need to ensure that the packages you upload are compatible for that os. Pure python packages are ok, but anything with compiled stuff needs to be OS specific.

Also, you'll need to make sure you zip up the entire package together with your code.

An easier approach would be to use the Serverless Framework and their serverless-python-requirements plugin. That would help automate this, so you're not manually patching this stuff together all the time.

Also checkout https://github.com/keithrozario/Klayers, which has a whole bunch of python packages ready for quick consumption -- unfortunately no asyncio.

Chivers