Python requests.exceptions.SSLError: EOF occurred in violation of protocol

Question

I would retrieve some information from an ABB G13 gateway that offer a RESTful JSON API. API is hosted by the gateway via https endpoint. Basic authentication mechanism is used for authentication. However all traffic goes through SSL layers.

On linux with command:

curl -s -k -X GET -u user:password https://host/meters/a_serial/power

All goes well!

I'm trying to write a script for windows in Python 2.7.10 with Requests 2.8.1 and with this code:

import requests
requests.get('https://host/meters/a_serial/power', auth=('user', 'password'))

I have this error:

Traceback (most recent call last):
  File "C:/Users/mzilio/PycharmProjects/pwrgtw/test.py", line 20, in <module>
    requests.get('https://host/meters/a_serial/power', auth=('user', 'password'))
  File "C:\Python27\lib\site-packages\requests\api.py", line 69, in get
    return request('get', url, params=params, **kwargs)
  File "C:\Python27\lib\site-packages\requests\api.py", line 50, in request
    response = session.request(method=method, url=url, **kwargs)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "C:\Python27\lib\site-packages\requests\adapters.py", line 433, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:590)

I've searched for a solution and I've tried to fix with this code:

import requests
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.poolmanager import PoolManager
import ssl

class MyAdapter(HTTPAdapter):
    def init_poolmanager(self, connections, maxsize, block=False):
        self.poolmanager = PoolManager(num_pools=connections,
                                       maxsize=maxsize,
                                       block=block,
                                       ssl_version=ssl.PROTOCOL_TLSv1)

s = requests.Session()
s.mount('https://', MyAdapter())
s.get('https://host/meters/a_serial/power')

But it doesn't work for me cause I get this error:

Traceback (most recent call last):
  File "C:/Users/mzilio/PycharmProjects/pwrgtw/test.py", line 16, in <module>
    s.get('https://host/meters/a_serial/power')
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 480, in get
    return self.request('GET', url, **kwargs)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "C:\Python27\lib\site-packages\requests\adapters.py", line 433, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:590)

I'm stuck on this problem. Could someone help me? Thanks!

Haggadah

Refer the stackoverflow question #18578939 – Widener

With the fix refered, a new error: requests.exceptions.SSLError: ("bad handshake: SysCallError(-1, 'Unexpected EOF')",) – Haggadah

Did you ever fix this? I am getting the same with an unrelated issue – Changeless

Can't this randomly occur if internet connection shuts down in the middle of the script ? – Chancelor

Haunted me for a week, very sporadic. I had all the most up to date software and ssl stack. Problem was I had two conflicting DNS entries for the same domain, one without an A record. Deleted that one and had to wait TTL 48h. – Lifelong

Answer 1

This thing worked for me, just make sure whether these modules are installed or not, if not then install them, following are:

pip install ndg-httpsclient

pip install pyopenssl

pip install pyasn1

It removed my SSLError: EOF occurred in violation of protocol (_ssl.c:590) error.

Hope it helps.

Meander

This did nothing for me. – Marius

I downvoted because it did nothing for me. I guess these libraries download stats went up for no reason o.o – Twirp

I get this error trying to run pip! – Feltner

It didn't help any how, explain how to use these modules after installing, will it automatic remove aur the request have to send through these modules. – Tatianatatianas

ndg-httpsclient solved the issue for me while trying to run "pip install -r requirements.txt". Thank you enigma – Psychographer

Answer 2

I found it was going through a proxy when it should have connected to the server directly.

I fixed this by doing

unset https_proxy

Changeless

Where did you add this? – Gryphon

just in my linux terminal - it will unset the https_proxy environment variable. – Changeless

saved the day. I was on a shared system. – Euton

Answer 3

Step 1: Check that Python supports TLS 1.1

You may have a Python setup that only supports TLS 1.0 – not TLS 1.1 or above.

You can check it like this:

Python 3

from urllib.request import urlopen
urlopen('https://www.howsmyssl.com/a/check').read()

Python 2

from urllib2 import urlopen
urlopen('https://www.howsmyssl.com/a/check').read()

(If you get urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)> you may have to disable certificate verification. NOTE: doing this will disable SSL protections against evildoers who would impersonate or intercept traffic to that website - see https://en.wikipedia.org/wiki/Man-in-the-middle_attack )

import ssl
urlopen('https://www.howsmyssl.com/a/check', context=ssl._create_unverified_context()).read()

Check the output for the key tls_version. If it says TLS 1.0 and not TLS 1.1 or TLS 1.2 that could be the problem.

If you're using a virtualenv, be sure to run the command inside.

Step 2: Install Python with a newer version of OpenSSL

In order support TLS 1.1 or above, you may need to install a newer version of OpenSSL, and the install Python again afterwards. This should give you a Python that supports TLS 1.1.

The process depends on your operating system – here's a guide for OS X.

virtualenv users
For me, the Python outside of my virtualenv had TLS 1.2 support, so just I removed my old virtualenv, and created a new one with the same packages and then it worked. Easy peasy!

Prince

Tried above and urlopen showed tls_version as 1.2 but still I get the same error : ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:579). Any other ideas on how to fix this issue? – Epi

@Epi In that case you might be having a different issue. Like the missing ndg-httpsclient package as mentioned here https://mcmap.net/q/361681/-python-requests-exceptions-sslerror-eof-occurred-in-violation-of-protocol. I only know that the above answer (my answer) worked for me – Prince

one-line check from the command line: python -c "from urllib.request import urlopen ; print(urlopen('https://www.howsmyssl.com/a/check').read())" – Kutenai

what version of openssl is required? – Banwell

@HarryMoreno Check this SO answer. TLS 1.1 is supported in Openssl 1.0.1 or later. – Prince

What could be the cause of the above code giving me urllib.error.URLError: <urlopen error no host given> ? I pass the exact string given above to urlopen(), and the error is the same for every well-formed URL. – Unspotted

Answer 4

I had exactly the same error, turns out that I didn't have ndg-httpsclient installed, see my original issue link1 link2 raised in github.

pip install ndg-httpsclient

Councilor

this way helps me – Quilt

link deleted :/ – Jocelyn

Here's the updated link: github.com/psf/requests/issues/3605 – Maggiore

Answer 5

If you are getting this error for intermediate requests, you can refer to the solution mentioned in https://github.com/requests/requests/issues/3391.

Basically, if you are making a lot of requests to a server and facing this issue with some of those requests you can use Session to just retry the requests.

Let me know if this works.

Gliadin

My service is doing 100+ requests to same server (different parameters) but after 70-80 request was getting this error. Changed the implementation to use Session and so far no error. – Irrevocable

Answer 6

The only time I have seen errors of this nature have been times that I was using requests to screen scrape data and I was using a multiprocessing library. I would either try your code without the pool manager or split your app into two apps, one that doles out urls and another that consumes them.

The client-server pair here should give you some ideas. I was able to scale my client out horizontally when I used the hacky server code to load all urls into a Queue in memory just before app.run. Hope that helps.

Bushtit

Answer 7

Close you proxy and try it again.

Catamaran

Answer 8

ENV: Python 3.10, www.howsmyssl.com returns tls_version: TLS 1.3:

For poor guys like me who MUST make query through proxy, this cloud be blame on your incorrect HTTPS proxy setting (perhaps you aren't set it, but python somehow believes you've set it, don't know why exactly, maybe because you've set the http proxy), you need to set it "properly".

I'm using windows10, haven't set it before, after set it, python works normally, give it a try.

Pale

Answer 9

try to run: pip install urllib3==1.25.11

It works for me anyway.

Law

Answer 10

I was having this problem when trying to do requests to YouTube API with python-youtube in a virtualenv. This worked for me:

from pyyoutube import Api

yt_api = Api(api_key="abc")
yt_api.session.verify = False

Anzovin

Answer 11

I've encountered this problem when I was uploading large files. (The request message is too big. The server does not allow messages larger than 262144000 bytes.) In order to debug this you should enable logging in your python script:

import logging

# Configure logging
logging.basicConfig(level=logging.DEBUG, 
                    format='%(asctime)s - %(levelname)s - %(message)s')

To tackle this I used:

#upload file to sharepoint
    local_path = 'out.csv'  # Local file path
    siteurl = sharepoint_baseurl + sharepoint_basesite
    test_user_credentials = UserCredential(sharepoint_username, sharepoint_password)
    ctx = ClientContext(siteurl).with_credentials(test_user_credentials)

    target_folder = ctx.web.get_folder_by_server_relative_url(sharepoint_upload_directory)
    size_chunk = 1000000
    with open(local_path, "rb") as f:
        uploaded_file = target_folder.files.create_upload_session(
            local_path, size_chunk, print
        ).execute_query()

    print("File {0} has been uploaded successfully".format(uploaded_file.serverRelativeUrl))

Drabble