Getting only response header from HTTP POST using cURL
Asked Answered
U

11

741

One can request only the headers using HTTP HEAD, as option -I in curl(1).

$ curl -I /

Lengthy HTML response bodies are a pain to get in command-line, so I'd like to get only the header as feedback for my POST requests. However, HEAD and POST are two different methods.

How do I get cURL to display only response headers to a POST request?

Unprincipled answered 8/4, 2012 at 3:12 Comment(0)
P
938
-D, --dump-header <file>
       Write the protocol headers to the specified file.

       This  option  is handy to use when you want to store the headers
       that a HTTP site sends to you. Cookies from  the  headers  could
       then  be  read  in  a  second  curl  invocation by using the -b,
       --cookie option! The -c, --cookie-jar option is however a better
       way to store cookies.

and

-S, --show-error
       When used with -s, --silent, it makes curl show an error message if it fails.

from the man page. so

curl -sS -D - www.acooke.org -o /dev/null

follows redirects, dumps the headers to stdout and sends the data to /dev/null (that's a GET, not a POST, but you can do the same thing with a POST - just add whatever option you're already using for POSTing data)

note the - after the -D which indicates that the output "file" is stdout.

Pino answered 8/4, 2012 at 4:18 Comment(14)
above comment is valid if you're using powershell. for cmd.exe use curl -s -D - http://yahoo.com -o nulErigena
@Erigena for me $null worked on Win7. Is it due to cLink installed on windows.Mallemuck
The "-" in front of the URL may seem unimportant, but it's not.Legato
@WahidSadik Why's is that the case in particular? What's the function of the single dash?Olivette
@Olivette -D takes an argument that says where the output should go. the single dash means it should go to stdout.Pino
Man I'm never gonna remember all that. I guess I need to make an alias for this one since I frequently want to do this. :\Jurat
Dropping the -o /dev/null will get your entire response body PLUS header. Also useful IMO.Hinduism
Nice answer, I wrapped that in a bash function: github.com/tlehman/bin/blob/master/headersJoannejoannes
Or as helper function in bash function headers {curl -s -D - $1 -o /dev/null}.Cyprio
If you get empty output and you're targeting HTTPS, then you either need to provide a cert (--cacert), or use --insecure flagSheryllshetland
@SatyaPrakash For me, $null writes to a file when cLink is installed.Caballero
I'd add -L to follow redirectsHawser
If for some reason you literally just want the headers and not the HTTP status code line (HTTP/1.1 200 OK), you can pipe it to tail: ` | tail -n +2`Parker
This still prints response body with MINGWMusculature
C
279

The other answers require the response body to be downloaded. But there's a way to make a POST request that will only fetch the header:

curl -s -I -X POST http://www.google.com

An -I by itself performs a HEAD request which can be overridden by -X POST to perform a POST (or any other) request and still only get the header data.

Cephalometer answered 31/7, 2016 at 0:12 Comment(10)
This answer is actually correct because web servers can return different headers based on request method. If you want to check headers on GET, you have to use GET request.Fluorocarbon
This is the most correct answer, in my opinion. It is easy to remember, it actually sends GET request and doesn't download the whole response body (or at least doesn't output it). The -s flag is nor necessary.Bacitracin
@JeffPuckettII well kinda nitpicking I would say. You can replace GET with POST in above command and it will work as expected. or any other is key there.Fluorocarbon
This does not work when you actually want to POST some data. Curl says: Warning: You can only select one HTTP request method! You asked for both POST Warning: (-d, --data) and HEAD (-I, --head).Wording
The answer below is better, using -X HEAD, which won't download the whole file. This is useful if you want to see how large a zip is before fetching it from a server. curl -s -I -X HEAD https://path/to/some/large.zip for example.Justus
@Justus The point here is that a server might respond differently to a HEAD request than to a POST or GET request (and some servers actually do that), so -X HEAD is no reliable solution here.Cephalometer
Very useful tips.Sapless
I get Warning: You can only select one HTTP request method! You asked for both POST Warning: (-d, --data) and HEAD (-I, --head).Commines
@Commines read the other comments first please. By the by, -s does become useful once you pipe the output, for example to grep.Miamiami
@Miamiami I got the same error as @Wording above. Nothing to do with -sCommines
C
109

The Following command displays extra informations

curl -X POST http://httpbin.org/post -v > /dev/null

You can ask server to send just HEAD, instead of full response

curl -X HEAD -I http://httpbin.org/

Note: In some cases, server may send different headers for POST and HEAD. But in almost all cases headers are same.

Chape answered 9/5, 2016 at 3:9 Comment(11)
It's unfortunate that the other answer won, because this is the correct answer - it doesn't unnecessarily transfer a ton of data.Litho
@dmd If I understand the cURL manual for -X, --request correctly, -X HEAD still results in “a ton of data” but there is -I, --head which should results in what you are anticipating.Arrowroot
You do not understand it correctly. -X HEAD and -I are exactly equivalent.Litho
Problem with -X HEAD is that the server might respond differently, since it now receives a HEAD request instead of a GET (or whatever the previous request was)Collectanea
Warning: Setting custom HTTP method to HEAD with -X/--request may not work the Warning: way you want. Consider using -I/--head instead.Debris
Why does this have a ton of upvotes? The question is "how to only get the headers when making a POST request", not "how to make a HEAD request".Addiction
@Addiction The answer shows both POST and HEAD and since you can't get away with body from POST request it shows how to redirect it to NUL device. So author recommends to still use HEAD because the question was about how to avoid lengthy body.Jerold
@Chape -vvv - is that super-extra verbose? (can't find anything in docs)Monogenic
@FrankNocke yes its super extra verbose, although it can just work with -vChape
@Addiction a perfect example of a XY-ProblemJackie
@Grav, this is an interesting proviso because by the RFC, "HTTP headers in response to a HEAD request SHOULD be identical to the information sent in response to a GET request" [rfc-editor.org/rfc/rfc2616#section-9.4 ], although that may not be actually adhered to, making the proviso necessary, even though it on some level should not be.Diskson
S
60

For long response bodies (and various other similar situations), the solution I use is always to pipe to less, so

curl -i https://api.github.com/users | less

or

curl -s -D - https://api.github.com/users | less

will do the job.

Surpassing answered 5/11, 2014 at 13:44 Comment(2)
these are not equivalent. the first issues a HEAD request to which many servers respond differently. the second issues a GET request which is more like what we are looking for here.Azotobacter
This is useful, but not an answer to the question. Therefore, I am voting down.Barbbarba
K
33

Maybe it is little bit of an extreme, but I am using this super short version:

curl -svo. <URL>

Explanation:

-v print debug information (which does include headers)

-o. send web page data (which we want to ignore) to a certain file, . in this case, which is a directory and is an invalid destination and makes the output to be ignored.

-s no progress bar, no error information (otherwise you would see Warning: Failed to create the file .: Is a directory)

warning: result always fails (in terms of error code, if reachable or not). Do not use in, say, conditional statements in shell scripting...

Karakoram answered 14/3, 2019 at 14:10 Comment(7)
Why use -o. instead of -o /dev/null?Addiction
@Addiction -o. is used versus -o /dev/null for brevityKarakoram
it doesn’t have the same behavior, so it’s strange to use that only to save 8 chars.Addiction
@Addiction there are other answers that show how to do this the most correct way, this one is here to show the short alternative that does the same thing basically.Karakoram
You should clarify in your answer that this command always fails. curl -svo. <url> && echo foo won’t print foo because -o. make curl return a non-zero (= error) code: curl: (23) Failed writing body.Addiction
a "solution" that ends with returning an error is not a valid solution. it's a happy accident. if something goes wrong, you have no way of knowing because you've already swallowed the errorMirnamirror
@Mirnamirror it is perfectly fine when you want to quickly use it interactively. Though I do not see how -vo. is shorter than -I ^^Miamiami
V
30

Much easier – this also follows links.

curl -IL http://shortlinktrack.er/in-the-shadows
  • -I is an alias of --head, the man page states that it fetch the headers only
  • -L is an alias of --location, the man page states that curl will follow the location header if there is one
Visionary answered 10/3, 2016 at 21:18 Comment(0)
A
16

While the other answers have not worked for me in all situations, the best solution I could find (working with POST as well), taken from here:

curl -vs 'https://some-site.com' 1> /dev/null

Arrowroot answered 14/2, 2016 at 9:23 Comment(2)
I had to put the url between quotes to get this working.Kheda
Whether this is necessary or not might depend on url and used shell. I improved the answer accordingly. Thanks.Arrowroot
M
12

headcurl.cmd (windows version)

curl -sSkv -o NUL %* 2>&1

real-world example (on troubleshooting proxy issues):

C:\depot>headcurl google.ch | grep -i -e http -e cache
Hostname was NOT found in DNS cache
GET HTTP://google.ch/ HTTP/1.1
HTTP/1.1 301 Moved Permanently
Location: http://www.google.ch/
Cache-Control: public, max-age=2592000
X-Cache: HIT from company.somewhere.ch
X-Cache-Lookup: HIT from company.somewhere.ch:1234

Linux version

for your .bash_aliases / .bash_rc:

alias headcurl='curl -sSkv -o /dev/null $@  2>&1'
Monogenic answered 29/11, 2019 at 8:31 Comment(4)
This will download the body and consume bandwidth, time. @Cephalometer 's answer (https://mcmap.net/q/63325/-getting-only-response-header-from-http-post-using-curl) doesn't have this overhead.Libelous
If & when you want POST, add -X POST to the passthrough parameters, if you want GET, use GET (i.e. default), as responses may differ. - Unless you do heavy curling in production scripts (not for diagnosis anddevelopment) I don't care about a bit of bandwidth.Monogenic
I am planning it to see if files on server are updated or not using 'Last-Modified'. The files in themselves are large, some are in GBs, and I am usually on cellular internet. So, this large bandwidth is an issue for me.Libelous
That would be hacky. I don't need to do this as siracusa's answer performs the task accurately.Libelous
O
1

The -w, --write-out <format> option can be very helpful. You can get all http headers, or a single one:

$ curl -s -w '%{header_json}' https://httpbin.org/get -o /dev/null
{"date":["Sun, 18 Feb 2024 13:47:12 GMT"],
"content-type":["application/json"],
"content-length":["254"],
"server":["gunicorn/19.9.0"],
"access-control-allow-origin":["*"],
"access-control-allow-credentials":["true"]
}

$ curl -s -w '%header{content-type}' https://httpbin.org/get -o /dev/null
application/json

read more

Operant answered 18/2 at 13:52 Comment(0)
A
0

-D, --dump-header Write the protocol headers to the specified file.

   This  option  is handy to use when you want to store the headers
   that a HTTP site sends to you. Cookies from  the  headers  could
   then  be  read  in  a  second  curl  invocation by using the -b,
   --cookie option! The -c, --cookie-jar option is however a better
   way to store cookies.
Aport answered 18/8, 2022 at 9:59 Comment(0)
G
0

To get only the response header, use the silent output -s along side -i, then output only the first 10 lines using the head command.

curl -si 0:80 | head
Gerlachovka answered 8/4 at 8:38 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.