AWS lambda throwing import error because of URLLIB

Question

Im running a python script on aws lambda and its throwing the following error.

 {
   "errorMessage": "Unable to import module 'app': urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with OpenSSL 1.0.2k-fips  26 Jan 2017. See: https://github.com/urllib3/urllib3/issues/2168",
   "errorType": "Runtime.ImportModuleError",
   "stackTrace": [] }

It was running perfectly an hour ago , and even after I have made no deployments , it seems to be failing.

my python version is 3.7. and Im only using urllib to parse and unquote urls . namely

from urllib.parse import urlparse

and

from urllib.parse import unquote

like its mentioned in the GitHub url I can upgrade my python version, but doing so would break other things. Are there any alternative librries I can use to get the same result?

from the GitHub link , it shows urllib no longer supports OpenSSL<1.1.1 but somehow some of our higher environments the same scripts is running without issues.

Caco

"even after I have made no deployments , it seems to be failing" if that's true that's very strange. Are you packaging ssl module yourself? "my python version is 3.7 ... would break other things" well, 3.7 is going to be EOL very soon, so I'm guessing that's part of your issue. – Lowson

Nope, not packaging the ssl module myself. – Caco

Answer 1

specifying my urllib3 to 1.26.15 in the requirement.txt file based on this https://github.com/urllib3/urllib3/issues/2168#issuecomment-1535838106 thread fixed this error.

Caco

The suggested solution in this link here fixed it for me. i.e pinning an older version of requests to requests==2.29.0 in requirements.txt. I am using python 3.8 – Streetlight

Answer 2

Upgrade you AWS Lambda function runtime to Python 3.10 and it should work.

Categorize

It worked, tried lot of answer, this one was simple worked quickly – Ferdinand

Answer 3

You can resolve it by specifying the lower version of urllib in your requirements.txt file.

requests
urllib3<2

Filler

Answer 4

I encountered this because of the following pip requrement in my Pipenv file:

requests = ">= 2.28.2, < 3"

What went wrong:

• requests depends on urllib3
• this dependency changed in requests 2.30.
  • requests/setup.py v2.29.0: "urllib3>=1.21.1,<1.27"
  • requests/setup.py v2.30.0: "urllib3>=1.21.1,<3"

Effectively the minor version upgrade of reguests 2.29 to 2.30 caused a major version upgrade of urllib3 version 1.x to 2.x. Which is backward incompatile and required a newer OpenSSL library on the system.

My simple fix was to change my requests dependency to

requests = ">= 2.28.2, < 2.29.0"

There will be similar errors with other pip dependencies which depend on urllib3. For example The bigtable package and many more.

Astrahan

Answer 5

I was able to resolve this issue using an older version of the urllib library when I was installing all the other libraries required by my code:

python -m pip install urllib3==1.26.16 --no-deps -t 

Indigene

Answer 6

If you look at the GH issue in the error message, you'll see that the problem only occurs to Python >=3.7 but <3.10

upgrade to python 3.10 and problem solved

Typical

Answer 7

I hacked at this quite a bit and ended up doing a few things.

I'm using python 3.8.16

1. I installed the openssl11

   sudo yum install openssl11
   

2. found the existing runtime.

whereis openssl
openssl: /usr/bin/openssl /usr/lib64/openssl /usr/include/openssl /usr/share/man/man1/openssl.1ssl.gz

3. renamed the old openssl

   sudo mv openssl openssl102
   

4. symlinked the new version

   ln -s openssl11 openssl
   

5. in the appropriate build directory, i then rebuilt the target python modules under the supported version.

   python3 -m pip install --target . urllib3==1.26.15
   

Oneida

Answer 8

I specified the version of urllib3 in requirements.txt file. urllib3==1.26.15 as suggested by @devarshi-goswami in this answer.

Then I got another error ImportError: Cannot find module named WebOb which was resolved by specifying WebOb==1.8.7 in the requirements.txt file.

Conclusion: My requirements.txt file contains

urllib3==1.26.15
WebOb==1.8.7

Tupiguarani

Answer 9

I'm unsure what caused this error, but I could fix it by importing boto3 into my function's requirements.

Northeastwards

Answer 10

I have been facing this issue, what worked for me is that I have installed urllib version 1.26.16 locally and made zip out of it, then added this zip in layers. I have used this in combination with openai and it worked since we are adding dependencies in layers.

pip install openai urllib3==1.26.16

Somber