"Access denied | <url> used Cloudflare to restrict access" GET request Postman

Question

I am currently testing out GET requests with Postman. I just want to see the kind of information I can get back. Simply plugging in the url for a website works for the most part. However, there is a particular website that is giving me trouble. When I try to initiate the GET request, I am given the following error:

<title>Access denied | <url> used Cloudflare to restrict access</title>
<meta charset="UTF-8" />
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta http-equiv="X-UA-Compatible" content="IE=Edge,chrome=1" />
<meta name="robots" content="noindex, nofollow" />
<meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=1" />
<link rel="stylesheet" id="cf_styles-css" href="/cdn-cgi/styles/cf.errors.css" type="text/css"
    media="screen,projection" />
<!--[if lt IE 9]><link rel="stylesheet" id='cf_styles-ie-css' href="/cdn-cgi/styles/cf.errors.ie.css" type="text/css" media="screen,projection" /><![endif]-->

The website works fine when launched directly in Chrome. However, every time I try to sent the GET in Postman, I get the Response 403. I've tried a few solutions already:

• Copy and paste the User-Agent displayed in the Chrome Inspect tools (Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36).
• Set up Postman interceptor system with status "INTERCEPTOR CONNECTED". Also installed the plug-in in my Chrome extensions.
• Enabled the cookies for the website.

Anyway, if I'm missing some glaringly obvious problem or you require more information, please reach out! I'm trying not to share the direct link to the website but I will if there's no other way.

Invaluable

Answer 1

The website is using Cloudflare which is a security and content delivery product.

When Cloudflare is used, all traffic to the website goes through Cloudflare first, where various security checks are made. This is an industry leading system which has many sophisticated checks, most of which can be configured by the customer for their particular website. It helps to prevent things like malicious bots and attacks on the website.

A 403 error means you have been blocked from accessing the resource. This is probably because the firewall at Cloudflare has decided the request from Postman is malicious or unwanted, so it's blocking you. The website works fine when you visit in your browser, because this is considered a "normal" request.

Botanize

But is there a bypass for that. I am basically trying to scrape a wordpress page and am constantly denied access. I use Beautiful soup and adding headers only triggers the captcha. any workarounds? – Strang

@SuryaPalaniswamy If you're trying to scrape someone's site, that's probably precisely why they want to use Cloudflare to deny you. – Botanize

In my case, I was just trying to test some different scenarios with the API using Postman and now I'm blocked. How do I resolve this? Because I can't use a browser since I'm accessing a GraphQL API. – Salas

@Salas If it's not your API, then no. Again, that is the point of the API host using Cloudflare - they decide which requests to block, as part of their security infrastructure. If there was a way for you to bypass it, that would obviously make their security completely useless. – Botanize

You may take the User-Agent header entirely away from the request and it will work in Postman. – Scoter

Answer 2

Use this code to simulate a browser,

header = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36'}

Idaline

You should probably explain why would changing the user agent help, and what is the user agent you set. – Hideout

developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent The User-Agent request header is a characteristic string that lets servers and network peers identify the application, operating system, vendor, and/or version of the requesting user agent. – Cranio

This doesn't work, cloudflare protection is not so easy to bypass by just using a custom header – Solarium

Answer 3

You can use curl_cffi.requests to mimic a browser request. This works because even though requests can fake a User Agent, it cannot fake a Tls Fingerprint but curl_cffi can.

You can install it with:

• pip: pip install curl_cffi
• poetry: poetry add curl_cffi

And then just import it with:

from curl_cffi import requests

And just use requests as normal

Brokaw