How to send line break with curl?

Question

I've tried the following to send a line break with curl, but \n is not interpreted by curl.

curl -X PUT -d "my message\n" http://localhost:8000/hello

How can I send a line break with curl?

Tissue

On what platform? May be relevant – Crucify

Answer 1

Sometimes you want to provide the data to be sent verbatim.

The --data-binary option does that.

Hedgepeth

This is the best way to do it. The alternative of using -d @message.txt as suggested in the other answer in particular can alter your line breaks. --data-binary on the other hand will not (which is important if you need to keep your CRLF linebreaks for multipart/form-data, see: #10765743) – Duleba

Because it took me a sec: if you're uploading a file you'll probably want to use a subshell for this curl -H "Content-Type:text/plain" --data-binary "$(<myfile)" http://localhost:8888 – Paraselene

Interesting, but what is the advantage? – Hedgepeth

+1, correct answer. curl --data-binary @/path/to/file.txt http://example.com/target – Cordero

I could not get --data-binary to work but was able to use the %0A character (see @malcolmocean) answer. When I use --data-binary "ip=33.44.55.*\n5.6.7.8" it does not treat it as newline, but --data ""ip=33.44.55.*%0A5.6.7.8" does send the newline through – Ramtil

--data-binary @/path/to/file.txt, as suggested by @FrankOlschewski, does not work for me with curl 7.47.0 on x86_64 Linux. Using a subshell via "(<myfile)" as suggested by @Paraselene to insert the multiline text into the command does work for me, however. – Unaccountedfor

This worked for me for plain/text files, no need to specify "Content-Type:text/plain" , ex. curl -X POST -H "AuthToken: CF179AE4-3C99-45F5-A7CC-3284AA91CF67" http://localhost:8080/collector --data-binary @large.log -s – Prink

Answer 2

Your shell is passing \ followed by n rather than a newline to curl rather than "my message\n". Bash has support for another string syntax that supports escape sequences like \n and \t. To use it, start the string with $' and end the string with ':

curl -X PUT -d $'my message\n' http://localhost:8000/hello

See ANSI-C Quoting in the Bash Reference Manual

Isthmus

This worked for me too. I'll have to play around with it, because it didn't work with double-quotes, which means that I can't use single-quotes within the string. – Councilor

I don't know where you got this idea that this is "JavaScript shell syntax". The shell passes my message\n verbatim, not with two escapes as you say. – Skiver

@ChrisDown, you misquoted me. I said "JavaScript string syntax", not "JavaScript shell syntax". I'm using JavaScript string syntax to be clear about what I mean with my string examples. I think what you're referring to as my message\n is the same as what I'm referring to as "my message\n". – Isthmus

@BenAtkin Sorry, freudian slip. However, my reading was still correct. \n has nothing to do with JavaScript. In fact nothing here has anything at all to do with JavaScript. – Skiver

I'm using it for the sake of explaining it to people. And it seems to have worked. Shell string syntax isn't widely understood. If it was, why would this question have been asked? What should I have used to explain it? – Isthmus

@BenAtkin The correct terminology is "backslash escape sequences". – Skiver

@ChrisDown you were right...changed my post to remove unnecessary reference to JavaScript – Isthmus

Answer 3

There's a much easier way!

curl -X PUT -d $'my message\n' http://localhost:8000/hello

This will use ANSI-C Quoting to insert the newline character.

No piping, no data files. See also Sending Newlines with cURL.

Sextant

That one should be an accepted answer despite of using Bash syntax – Proprietress

This is the only thing that worked for me out of all the answers – Seduce

Answer 4

The solution for someone who doesn't want to use files, and doesn't want to resort to shell escaping magic is:

curl -X POST --data-binary @- http://url.com <<EOF
line one
line two
EOF

But this is literal newlines in the post data payload, and not in form fields.

Ares

I'm having trouble understanding this. I get that @ is to indicate a file name but is there some special meaning when using @-? What is <<EOF doing? – Alongside

@- tells curl to consume input from standard in, and <<EOF is the end of stream indicator for bash. We then later use the magic word EOF in the data payload to tell bash that we are done writing to the stream. – Ares

Also, - is sort of the standard way in GNU/Linux to specify STDIN when a file name is expected. It's not universal, but it's pretty common. – Prepossessing

By consulting the manual we see that it should be just - and not @- – Zany

Answer 5

Had similar issue. While uploading a CSV file from Mac to cloud storage, new lines were being removed. After downloading it, the entire file looked like a single line. I tried adding different EOL characters \n \r \r\n with no success. Using --data-binary instead of -d solved the issue.

Btw this issue occurred only from Mac. -d worked just fine while making the call from CentOS machine. This very much looks like due to Mac's newline character. But don't feel like debugging any more.

Thanks a lot for your help.

curl -X PUT -d @filename.csv https://cloudstorage -H "content-type: text/csv"

vs

curl -X PUT --data-binary @filename.csv https://cloudstorage -H "content-type: text/csv"

Cellulose

Thanks a lot ! This is not related to your Mac : I was having the exact same issue on Linux, and using --data-binary @ has solved my issue (sending a multiline .ics file to a CalDAV server). – Inside

Answer 6

(I ended up here with a slightly different question, so I'm just going to post my answer because it might help future explorers)

My solution applies to people who are sending form-style data, i.e. key/value pairs in a query string. Use the encoded line break, which is %0A, just like how an encoded space is %20. You can use http://meyerweb.com/eric/tools/dencoder/ to convert other symbols.

So if you want to set the key message to the value:

line one
another

you would send

curl --data "message=line%20one%0Aanother" http://localhost:8000/hello

Horsetail

minor comment (maybe typo) for a line break/carraige return character it should be %0A rather than %A0 – Ramtil

Answer 7

A very easy way, just Shift-Enter in the console for the break. Very readable typing it in too.

curl -d "line1
line2" http-echo.com

Server gets this: line1\nline2

Do this to remove the line break:

curl -d "line1 \
line2" http-echo.com

Server gets this: line1 line2

Copt

Answer 8

Not an answer to your question, but I would work around it by creating a temporary file containing the message and line break, and give curl that file to work on:

curl -X PUT -d @message.txt http://localhost:8000/hello

From the manual:

If you start the data with the letter @, the rest should be a file name to read the data from, or - if you want curl to read the data from stdin. The contents of the file must already be URL-encoded. Multiple files can also be specified. Posting data from a file named 'foobar' would thus be done with --data @foobar.

Crucify

Using temporary files is a handy approach. As per Szocske's answer, --data-binary is a more faithful alternative to -d, as it will send the data verbatim. – Duleba

-1; Using a temporary file with -d @/path/to/temp/file.txt does NOT solve the line-break problem. --data-binary does, see above. – Cordero

If you're seeing this because you're wondering why your curl commands don't work after upgrading curl or upgrading to windows 10, make sure you add quotes around your file reference. For example: curl -X PUT -d "@message.txt" localhost:8000/hello My elasticsearch rebuild scripts had stopped working. – Hexone

Answer 9

I was using Sendgrid with this code (copied below) originally found here https://sendgrid.com/docs/API_Reference/Web_API_v3/index.html

\n\n worked in Gmail, but \n was ignored. I tried to double the escape and other suggestions. I also tried \r\n and that did not work in Gmail either. Note: I didn't bother to test other email clients, maybe it was a Gmail-specific problem.

    curl --request POST \
  --url https://api.sendgrid.com/v3/mail/send \
  --header 'Authorization: Bearer YOUR_API_KEY' \
  --header 'Content-Type: application/json' \
  --data '{"personalizations": [{"to": [{"email": "[email protected]"}]}],"from": {"email": "[email protected]"},"subject": "Hello, World!","content": [{"type": "text/plain", "value": "Heya!"}]}'

Eventually I gave up looking for a solution and switched the text/plain to text/html and just used <br /> tags.

Someone suggested that Sendgrid converts plaintext to HTML if you have a tracking pixel enabled, which makes sense. Maybe the newlines were destroyed in the plaintext-to-html conversion process. I assume the client wants a tracking pixel, so decided to switch to HTML.

Leflore