What does the Python InsecureRequestWarning really mean?
Asked Answered
B

1

14

I'm getting the warning:

/.../local/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
  InsecureRequestWarning)

I'm reading the doc.

I'm seeing lots of posts on how to disable it if I know what I'm doing, like this one.

But I'm still having trouble figuring out what the error means. I gather that it means that I'm missing a certificate (because it only happens on my VPS, not on my Mac running the same version of a script), but I don't understand why I need a certificate to make a secure request to a third-party API.

A helpful summary (or just a point in the right direction) would be much appreciated so I can decide whether or not to disable it. My gut is that I shouldn't disable it, so I'd like to figure out how to address the problem properly.

Bolanos answered 9/2, 2015 at 14:2 Comment(0)
S
23

I am glad that you did not simply disable the warning. Great question, actually! What's required here is basic understanding of how the "chain of trust" is working. That is not a shame, many do not have knowledge about this. However, as a developer one should know the basics! Go ahead, and maybe read about how the whole thing works.

In short, TLS is meant to guarantee secrecy, authenticity, and integrity. Common sense in the security community is (*): without certificate verification you get NONE of these three items, because you are vulnerable to man in the middle attacks. That is, verify the certificate, or you might just as well stop using HTTPS. That is what the warning is about.

A little more context: part of this security architecture is that the remote host claims to have a certificate signed by someone higher in the chain of trust, a so-called certificate authority (CA). The client needs to verify that this CA actually did sign that certificate in question. For this verification to work, the client needs a local database with the public keys of many CAs (think of these as "trust anchors", the collection of which can be called "certificate bundle").

I don't understand why I need a certificate to make a secure request to a third-party API

Please, read about the details elsewhere. But, for completeness of this answer, this is a high-level abstraction that should clarify why some external source of information is required:

  • Your client does not trust the remote end.
  • So, your client needs to involve a third party and ask about the credibility of the remote end.
  • That third party is your local database of public keys of many CAs, a so-called "certificate bundle".

You can use the requests library instead of urllib3, it performs certificate verification by default (and ships its own CA database).

(*) unverified HTTPS connections can be "better" than plain HTTP, but this needs to be evaluated on a case-to-case basis.

Spiritoso answered 9/2, 2015 at 14:43 Comment(7)
Helpful, thanks. Doesn't the requests library use urllib3 under the hood? Without looking further, the Warning traces to a module that appears to be within the requests package.Bolanos
I think it does. The high level HTTP method of requests have a verify parameter which is set to True by default. When using requests, an unverified connection should raise a requests.exceptions.SSLError, not just print a warning.Spiritoso
Yup, requests uses urllib3. In fact, it looks like op is already using requests. You're right, we could add more guidance on how to decide whether you should be concerned. I welcome you to open an issue or pull request with a proposal. Main thing is: Are you confident that your unverified connection can't be monitored or interfered with? Such as, if it's only performed within a local private network not exposed to the external internet.Randa
I'm making a request to a third-party API, so I assume that means it's not only performed within a local private network and that it is exposed to the external internet. Seems like I'd better further explore.Bolanos
@shazow: now, if he was using requests, wouldn't there an exception be raised instead of a warning printed?Spiritoso
@HaPsantran: Then I would suggest making sure the secure connection is properly validated. :) @Jan-Philip: See .../site-packages/requests/packages/urllib3..., it's coming from urllib3 vendored inside of requests. Maybe there's a bug in requests or @Bolanos is using urllib3 manually from within requests? Not sure.Randa
@Dr.Jan-PhilipGehrcke Used request for API, What happened security point of you, If we setup 'verify=False` with getting warning Unverified HTTP request.. ? another, If I set verify=True then request not work , getting error like request.exception.SSError..Max retries exceeded with url: ? solutionJehanna

© 2022 - 2024 — McMap. All rights reserved.