Python Requests requests.exceptions.SSLError: [Errno 8] _ssl.c:504: EOF occurred in violation of protocol

Asked 31/12, 2012 at 13:51 Answered 18/4, 2024 at 1:1

Solved python openssl httprequest python-requests ubuntu-12.10

I'm on Ubuntu 12.10 with OpenSSL 1.0.1c, python 2.7.3, Requests 1.0.3 and 1.0.4 (tried both), and when attempting to connect to the website in the url variable with the following code.

def SendInitialRequest(xmlmessage, redirecturl):
    url = 'https://centineltest.cardinalcommerce.com/maps/txns.asp'

    payload = 'cmpi_msg=' + ET.tostring(xmlmessage)
    headers = {
        'Content-Type': 'application/x-www-form-urlencoded',
    }
    r = requests.post(url, data=payload, headers=headers, verify=None)
    print r.text

It throws the following error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "clams/libs/centinel/thinclient.py", line 134, in SendInitialRequest
    r = requests.post(url, data=payload, headers=headers, verify=None)
  File "/home/jasonamyers/.virtualenv/clams/lib/python2.7/site-packages/requests/api.py", line 87, in post
    return request('post', url, data=data, **kwargs)
  File "/home/jasonamyers/.virtualenv/clams/lib/python2.7/site-packages/requests/api.py", line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/jasonamyers/.virtualenv/clams/lib/python2.7/site-packages/requests/sessions.py", line 269, in request
    resp = self.send(prep, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies)
  File "/home/jasonamyers/.virtualenv/clams/lib/python2.7/site-packages/requests/sessions.py", line 364, in send
    r = adapter.send(request, **kwargs)
  File "/home/jasonamyers/.virtualenv/clams/lib/python2.7/site-packages/requests/adapters.py", line 163, in send
    raise SSLError(e)
requests.exceptions.SSLError: [Errno 8] _ssl.c:504: EOF occurred in violation of protocol

Attempting the connection with openssl returns the following:

$ openssl s_client -connect centineltest.cardinalcommerce.com:443
CONNECTED(00000003)
140019346777760:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:177:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 226 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
---

If I force it to use tls1 it works (output truncated):

$ openssl s_client -tls1 -connect centineltest.cardinalcommerce.com:443
CONNECTED(00000003)
depth=2 C = US, O = "thawte, Inc.", OU = Certification Services Division, OU
verify error:num=20:unable to get local issuer certificate
verify return:0
---

I've seen numerous bug reports for this; however, I've not found a way to get around it using the python requests library. Any assistance would be greatly appreciated.

Substratosphere answered 31/12, 2012 at 13:51 Comment(4)

Did you install openssl through the package manager? If yes, did you check for updates? I can connect to this site using requests. – Sympathize 31/12, 2012 at 14:3

I did install openssl via the package manager, there are no updates. Can you share the versions of everything you are using? Also are you using the built in python or did you use pythonbrews etc to build your own? – Substratosphere 31/12, 2012 at 14:5

In my case the answer was this: https://mcmap.net/q/49585/-javax-net-ssl-sslhandshakeexception-remote-host-closed-connection-during-handshake-during-web-service-communicaiton – Alacrity 20/2, 2016 at 0:48

I had this error while trying to connect to an HTTP endpoint using HTTPS. I just changed the url and it worked fine. – Jeaniejeanine 24/7, 2017 at 7:55

Reposting this here for others from the requests issue page:

Requests' does not support doing this before version 1. Subsequent to version 1, you are expected to subclass the HTTPAdapter, like so:

from requests.adapters import HTTPAdapter
from requests.packages.urllib3.poolmanager import PoolManager
import ssl

class MyAdapter(HTTPAdapter):
    def init_poolmanager(self, connections, maxsize, block=False):
        self.poolmanager = PoolManager(num_pools=connections,
                                       maxsize=maxsize,
                                       block=block,
                                       ssl_version=ssl.PROTOCOL_TLSv1)

When you've done that, you can do this:

import requests
s = requests.Session()
s.mount('https://', MyAdapter())

Any request through that session object will then use TLSv1.

Substratosphere answered 3/1, 2013 at 19:45 Comment(2)

For the sake of keeping this answer up to date, I did end up writing this up here. The subclass API has changed a little bit since this answer was posted, so you should use the code in that blog post, not the code in this answer. =) – Drus 3/9, 2013 at 9:5

@Lukasa's link appears to be offline, here is an archive'ed version – Cruciferous 28/9, 2016 at 13:15

Setting verify=False only skips verifying the server certificate, but will not help to resolve SSL protocol errors.

This issue is likely due to SSLv2 being disabled on the web server, but Python 2.x tries to establish a connection with PROTOCOL_SSLv23 by default. This happens at https://github.com/python/cpython/blob/360aa60b2a36f5f6e9e20325efd8d472f7559b1e/Lib/ssl.py#L1057

You can monkey-patch ssl.wrap_socket() in the ssl module by overriding the ssl_version keyword parameter. The following code can be used as-is. Put this at the start of your program before making any requests.

import ssl
from functools import wraps
def sslwrap(func):
    @wraps(func)
    def bar(*args, **kw):
        kw['ssl_version'] = ssl.PROTOCOL_TLSv1
        return func(*args, **kw)
    return bar

ssl.wrap_socket = sslwrap(ssl.wrap_socket)

Tybalt answered 11/6, 2014 at 15:10 Comment(13)

Thank you so much for providing this monkey patch - had been tearing my hair out trying to find a way to set ssl_version while using Requests library :) – Aphis 24/11, 2014 at 15:14

Works like a charm!! But what is the scope of this change? Only my program is set to use TLS_1 or is it global? – Mydriasis 18/3, 2015 at 13:46

As long as the code has been run in any Python invocation, the change will be effective. If you run another Python script/program that doesn't invoke this code, then there is no change. You could put the code into it's own module (.py file) and import this file in any script that needs this override. – Tybalt 18/3, 2015 at 16:28

@MA1, please provide more details on the problem you are facing. – Tybalt 26/7, 2015 at 17:3

@Tybalt I tried this but still getting issue. The weird thing is i get the issue sometime not always. I have a python3 script that makes lots of connections(not in parallel) to a site. So the workaround i have is retry to connect if SSLError exception comes and this is working. – Closelipped 26/7, 2015 at 22:14

I suspect then the root cause of your problem may be with something else. The problem in this thread is about SSL version mismatch between server and client. – Tybalt 27/7, 2015 at 2:51

@Tybalt I have applied this patch but now get the error: urlopen error [Errno 1] _ssl.c:504: error:1409442E:SSL routines:SSL3_READ_BYTES:tlsv1 alert protocol version what could this mean? – Quotation 16/12, 2015 at 14:35

@Quotation what's the server URL? Looks like a protocol version mismatch between server and client. Can you share your relevant code as well? – Tybalt 17/12, 2015 at 3:26

@Tybalt I have an app that scrapes council websites using python and Mechazines. I get this error on a few URLs I try and scrape, one being: planning.middevon.gov.uk/online-applications. Mechazine trys to open the url using urllib urlopen(url). Sorry if thats not enough, any help would be great, thanks. – Quotation 17/12, 2015 at 10:53

@AfromanJ, I got a certificate verification error, which means that the protocol negotiation succeeded. It's probably because our environments are different. Having said that, this does not seem like the right place to troubleshoot this. – Tybalt 18/12, 2015 at 13:42

@chnrx python does not try to use SSLv2 exclusively in this case, it just tries to send an SSLv2-compatible ClientHello message, which may end up using SSLv2, but could use higher version as well. A server can have SSLv2 support disabled while still allowing SSLv2-compatible ClientHello messages to start the SSL negotation. But what is happening more and more is that servers are now disabling the acceptance of SSLv2-compatible ClientHello messages as well. – Ucayali 1/1, 2016 at 21:22

I don't know why it WORKED and how it WORKED it just WORKED – Cordite 7/1, 2016 at 17:0

I used this fix and now I get ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:590) – Novercal 1/2, 2016 at 19:34

Installing the "security" package extras for requests solved for me:

sudo apt-get install libffi-dev
sudo pip install -U requests[security]

Bluhm answered 15/2, 2016 at 21:35 Comment(4)

This fixed a shitty SSL protocol error problem in OSX with stripe SDK, I don't know how I arrived here, but I'm adding some extra google keywords for the future me. SSLError: EOF occurred in violation of protocol (_ssl.c:590)) – Series 1/4, 2017 at 20:49

The method fixed my issue, on my Ubuntu(WSL) with Windows 10. – Attis 8/5, 2017 at 5:53

On a Beaglebone I had to first sudo apt-get install python3-dev libssl-dev for the requests[security] install to finish. It still hung for a good couple of minutes, but eventually succeeded. Alas, it made no difference to the requests SSL error. – Found 21/1, 2018 at 2:9

just "pip install -U requests[security]" fixed my issue. – Sinus 27/11, 2018 at 6:58

This is a known bug, you can work it around with a hack:

Open up site-packages/requests/packages/urllib3/connectionpool.py (or otherwise just make a local copy of requests inside your own project), and change the block that says:

def connect(self):
    # Add certificate verification
    sock = socket.create_connection((self.host, self.port), self.timeout)

    # Wrap socket using verification with the root certs in
    # trusted_root_certs
    self.sock = ssl_wrap_socket(sock, self.key_file, self.cert_file,
                                cert_reqs=self.cert_reqs,
                                ca_certs=self.ca_certs,
                                server_hostname=self.host,
                                ssl_version=self.ssl_version)

to:

def connect(self):
    # Add certificate verification
    sock = socket.create_connection((self.host, self.port), self.timeout)

    # Wrap socket using verification with the root certs in
    # trusted_root_certs
    self.sock = ssl_wrap_socket(sock, self.key_file, self.cert_file,
                                cert_reqs=self.cert_reqs,
                                ca_certs=self.ca_certs,
                                server_hostname=self.host,
                                ssl_version=ssl.PROTOCOL_TLSv1)

Otherwise, I suppose there's an override somewhere which is less hacky, but I couldn't find one with a few glances.

NOTE: On a sidenote, requests from PIP (1.0.4) on a MacOS just works with the URL you provided.

Hematuria answered 31/12, 2012 at 14:7 Comment(10)

I also had it work perfectly on MacOS, just worried about production, and that when Apple updates OpenSSL that might not always be the case. Thanks so much for the response. Gonna see if I'm actually allow to get a custom package like that pushed up to heroku :) – Substratosphere 31/12, 2012 at 14:14

@jasonamyers: well, you can just make it a part of your project, just copy requests folder to your own project and import requests will prefer your patched version to the stock one. – Hematuria 31/12, 2012 at 14:22

really trying to avoid doing that for one connection out of many others :( I'll keep digging, thanks for the response. – Substratosphere 31/12, 2012 at 14:38

I had another deeper look, there's no way ssl_version can be overriden from the top of requests module, without modifications to the module itself :( – Hematuria 31/12, 2012 at 14:50

This isn't an issue on Mac because they use OpenSSL 0.9.8r 8 Feb 2011, which hasn't implemented these changes yet. My guess is it's just a matter of time :( – Substratosphere 31/12, 2012 at 15:4

You could also downgrade SSL ;) – Hematuria 31/12, 2012 at 15:7

It is worth noting that this same behavior occurs on a mac if you use openssl 1.0.1c :( – Substratosphere 3/1, 2013 at 14:9

This worked for me, but now I'm getting another error: requests.exceptions.SSLError: [Errno 1] _ssl.c:504: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number – Brillatsavarin 6/5, 2013 at 23:10

The code you referenced is is connection.py not connectionpool.py, at least in my version of requests (2.5.3). – Lethbridge 28/12, 2015 at 19:36

@ChrisJohnson might have changed over time :) it's been 3 years :) – Hematuria 28/12, 2015 at 19:37

I had the same issue:

raise SSLError(e)
requests.exceptions.SSLError: [Errno 8] _ssl.c:504: EOF occurred in violation of protocol

I had fiddler running, I stopped fiddler capture and did not see this error. Could be because of fiddler.

Romanfleuve answered 28/8, 2013 at 0:34 Comment(0)

I encountered this error, and the fix appears to be turning off SNI, which Python 2.7 does not support:

http://bugs.python.org/issue5639

urllib3 on python 2.7 SNI error on Google App Engine

Vareck answered 7/12, 2013 at 4:34 Comment(0)

To people that can't get above fixes working.

Had to change file ssl.py to fix it. Look for function create_default_context and change line:

context = SSLContext(PROTOCOL_SSLv23)

context = SSLContext(PROTOCOL_TLSv1)

Maybe someone can create easier solution without editing ssl.py?

Rempe answered 31/8, 2016 at 13:39 Comment(0)

I was having similar issue and I think if we simply ignore the ssl verification will work like charm as it worked for me. So connecting to server with https scheme but directing them not to verify the certificate.

Using requests. Just mention verify=False instead of None

    requests.post(url, data=payload, headers=headers, verify=False)

Hoping this will work for those who needs :).

Mcdougall answered 27/6, 2017 at 7:36 Comment(0)

Unfortunately the accepted answer did not work for me. As a temporary workaround you could also use verify=False when connecting to the secure website.

From Python Requests throwing up SSLError

requests.get('https://example.com', verify=True)

Galiot answered 6/3, 2013 at 11:0 Comment(1)

Its a bad idea to set verify=False. You need to figure out what's really broken and fix it. – Gimbals 19/4, 2014 at 18:17

I have had this error when connecting to a RabbitMQ MQTT server via TLS. I'm pretty sure the server is broken but anyway it worked with OpenSSL 1.0.1, but not OpenSSL 1.0.2.

You can check your version in Python using this:

import ssl
ssl.OPENSSL_VERSION

I'm not sure how to downgrade OpenSSL within Python (it seems to be statically linked on Windows at least), other than using an older version of Python.

Feudal answered 13/4, 2016 at 15:27 Comment(0)

So, I tried a bunch of these solutions, and none of them worked for me. And, this is going to sound stupid, but...

I had 'https' in the url when it was supposed to be 'http'. All I had to do was delete the 's', and the request worked seamlessly.

Hope this helps someone else.

Dosh answered 18/4, 2024 at 1:1 Comment(0)

Hot tags

Godot Unity Godot Help Programming Godot 4.X GUI GDScript 3D 2D Physics CSharp Godot 3.X VR XR Projects C++

Recommended topics

Hot tags