Can cURL detect 307 response?

Question

For my research I need to cURL the fqdns and get their status codes. (For Http, Https services) But some http urls open as https although it returns 200 with cURL. (successful request, no redirect)

curl -I  http://example.example.com/
HTTP/1.1 200 OK
Server: nginx
Date: Mon, 22 Nov 2021 10:43:32 GMT
Content-Type: text/html; charset=UTF-8
Content-Length: 64991
Connection: keep-alive
Keep-Alive: timeout=20
Vary: Accept-Encoding
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Pragma: no-cache
Link: <https://example.example.com/>; rel=shortlink
X-Powered-By: WP Engine
X-Cacheable: SHORT
Vary: Accept-Encoding,Cookie
Cache-Control: max-age=600, must-revalidate
X-Cache: HIT: 10
X-Cache-Group: normal
Accept-Ranges: bytes

As seen above I get 200 response with curl request. But I can see the 307 code in my browser. (available in the picture below)

Request URL: http://example.example.com/
Request Method: GET
Status Code: 307 Internal Redirect
Referrer Policy: strict-origin-when-cross-origin

Can I detect 307 code with curl? (-L parameter doesn't work) Any suggestions?

Sandoval

are you looking for a terminal command, or does a script work? – Nonparticipation

thanks for the reply. only terminal command. But as a result, I will make it a python script. – Trust

-L option tells cURL to automatically follow redirects. So, remove this option and catch the response_code – Putrefaction

okay. I added the results as text. – Trust

One possible difference is the header 'Upgrade-Insecure-Requests' that most browsers send during requests. That may cause an internal redirect to https. – Lupus

Open dev tools in your browser. Click network tab. Go to url. Look at headers your browser sends. Repeat curl command adding all the headers. Is it the same now? – Evy

@BrianWhite I've tried. But again I got the same result. My curl command: curl -I -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9" -H "Upgrade-Insecure-Requests: 1" -H "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.55 Safari/537.36" http://example.example.com – Trust

Answer 1

curl -w '%{response_code}\n' -so /dev/null $URL

It can be tested out like this:

curl -w '%{response_code}\n' -so /dev/null httpbin.org/status/307 

so what is the 307 in the question?

As Stefan explains here in a separate answer: that's an internal message from Chrome that informs you that it uses HSTS. It is not an actual response code. Which is why curl can't show it. Chrome should make that clearer.

HSTS

HSTS is a way for a HTTPS server to ask clients to not contact them over clear text HTTP again. curl also supports HSTS but then you need to use --hsts - and curl will still not confusingly claim any 307 response codes.

Genitalia

request query: curl -w '%{response_code}\n' -so /dev/null http://example.example.com response data: 200 I don't want the parse. I want to see the 307 code I see in the browser in cURL. Your command shows the response code part of the curl data. – Trust

My command line shows the response code - only. If you use a URL that actually gives you a 307 you'll see that. – Genitalia

Your code is completely correct. The problem is that the URL returns 307 but cURL cannot detect it. – Trust

No. curl shows exactly what it gets. When curl doesn't show 307, it means it didn't get any 307. Possibly the server returns differently to different clients. – Genitalia

@Abdullah You see different response codes on curl and your browser because they don't send exactly same request, so response from server differs. You can convert browser request to curl command, it's documented here: everything.curl.dev/usingcurl/copyas – Doublefaced

agreed with @yozel, there are some headers browser sent, like user-agent, among others – Inveteracy

This was the answer I was looking for. "curl does not show 307". Thank you. – Trust

Curl do not show the 307 because the server didn't sent it that's why it's called Internal Redirect by the browser. – Zawde

Answer 2

The 307 http status isn't actually a response that is sent by a server. It's an internal redirect, something that your browser does for you before even sending the request to the server. That's why it won't show up in curl. It's a feature of your browser. cURL is much more reliable when it comes to sending unaltered requests.

A 307 (especially since you mention https redirects) internal redirect is usually encountered when dealing with the security feature of HSTS (HTTP strict-transport-security) where the whole purpose is to make sure that you never send unencrypted http requests to a server that wants to communicate via encrypted https.

See this.

Hayott

Hi! Welcome to Stack Overflow and thanks for posting your first answer. Unfortunately the answer you gave is incorrect. 307 is indeed a response code that is sent by HTTP servers. It's a slight variant of 302. See developer.mozilla.org/en-US/docs/Web/HTTP/Status/307 for details. – Pricillaprick

Stefan correctly explains why the asked about 307 isn't actually a response code over HTTP which is also why curl doesn't get it. The strange 307 message is just Chrome's (confusing) way to say it uses HSTS. – Genitalia

@Pricillaprick Wrong. 307 Internal Redirect from the question is Chrome telling the user it’s switching from HTTP to HTTPS after checking its local cache of HSTS data. – Byng

Ah, thank you for the corrections to my corrections. Re-reading the OP I see that's indeed what's happening. I read this answer as being about 307 response codes in general as opposed to the a specific instance here, and so didn't read the OP as thoroughly as I should have. Apologies to you, Stefan! – Pricillaprick