How can I write a heredoc to a file in Bash script?

Asked 1/6, 2010 at 20:28 Answered 23/8, 2022 at 14:32

Solved bash heredoc

990

How can I write a here document to a file in Bash script?

Pearcy answered 1/6, 2010 at 20:28 Comment(1)

See also #22698188 – Warfore 16/2, 2018 at 11:2

1489

Read the Advanced Bash-Scripting Guide Chapter 19. Here Documents.

Here's an example which will write the contents to a file at /tmp/yourfilehere

cat << EOF > /tmp/yourfilehere
These contents will be written to the file.
        This line is indented.
EOF

Note that the final 'EOF' (The LimitString) should not have any whitespace in front of the word, because it means that the LimitString will not be recognized.

In a shell script, you may want to use indentation to make the code readable, however this can have the undesirable effect of indenting the text within your here document. In this case, use <<- (followed by a dash) to disable leading tabs (Note that to test this you will need to replace the leading whitespace with a tab character, since I cannot print actual tab characters here.)

#!/usr/bin/env bash

if true ; then
    cat <<- EOF > /tmp/yourfilehere
    The leading tab is ignored.
    EOF
fi

If you don't want to interpret variables in the text, then use single quotes:

cat << 'EOF' > /tmp/yourfilehere
The variable $FOO will not be interpreted.
EOF

To pipe the heredoc through a command pipeline:

cat <<'EOF' |  sed 's/a/b/'
foo
bar
baz
EOF

Output:

foo
bbr
bbz

... or to write the the heredoc to a file using sudo:

cat <<'EOF' |  sed 's/a/b/' | sudo tee /etc/config_file.conf
foo
bar
baz
EOF

Mercaptide answered 2/6, 2010 at 3:40 Comment(16)

You don't even need Bash, this feature is in the Bourne/Korn/POSIX shells too. – Interclavicle 1/5, 2013 at 17:24

what about <<<, what are they called? – Pompei 28/11, 2013 at 8:14

@PineappleUndertheSea <<< are called 'Here Strings'. Code like tr a-z A-Z <<< 'one two three' will result in the string ONE TWO THREE. More information at en.wikipedia.org/wiki/Here_document#Here_strings – Mercaptide 9/12, 2013 at 18:3

incomplete example. What if output isn't a file but unnamed pipe to another program? – Gearing 26/5, 2014 at 10:1

Below I have attempted to combine and organize this answer with that of Serge Stroobandt. – Hettie 18/9, 2014 at 18:31

The final EOF should not have any whitespace after it either. At least on bash, this results in it being unrecognised as the delimiter – Schoolboy 12/7, 2015 at 13:43

@Schoolboy the final example works for me (Apple Yosemite and Bash 3.2). The final EOF is encapsulated in single quotes, and there is no whitespace after the EOF; even if there is whitespace (for readability) after the final single quote. – Mercaptide 13/7, 2015 at 22:25

@StefanLasiewski Sorry, I wasn't suggesting your examples didn't work. You said "the final 'EOF' should not have any whitespace in front of the word" -- What I found on OSX 10.9 and Bash is is that a trailing space after the EOF can also cause it to fail too. I was just leaving a comment to help other people as it took me a while to figure out :) – Schoolboy 14/7, 2015 at 23:33

If one doesn't want to be forced to use tab characters as indentation with cat <<- EOF, they could use cat <<- 'EOF' | awk 'NR==1 && match($0, /^ +/){n=RLENGTH} {print substr($0, n+1)}'. That removes the amount spaces preceding the first line in the here document from every consecutive line (see answer by anubhava). – Blame 21/11, 2015 at 9:56

Since this particular heredoc is intended to be literal content, rather than containing substitutions, it should be <<'EOF' rather than <<EOF. – Redness 27/5, 2016 at 16:13

You could add that EOF is not a special keyword, and any other limit word could be used instead. EOF is a meta-syntactic variable in your examples. – Gavrila 27/11, 2017 at 22:34

I'd like to add that one could set up the line nslookup<<EOF > output.txt equivalently to the example in the answer. The next line would be www.google.com followed by exit and then EOF. This was helpful to me because nslookup has an interactive mode and I wanted to store its output to a file. – Stenograph 19/6, 2018 at 19:39

what if you have docker-compose content in userdata pf cloudformation template? #55700162 – Willdon 16/4, 2019 at 15:13

Life saver, i had added a space in front of ending EOF and it wasted my half an hour – Visakhapatnam 6/7, 2022 at 13:32

Why do I need to use 'cat' rather than 'echo'? – Collator 29/7, 2023 at 5:28

Recommending the ABS casts a shadow over your entire answer. – Warfore 5/12, 2023 at 12:47

215

Instead of using cat and I/O redirection it might be useful to use tee instead:

tee newfile <<EOF
line 1
line 2
line 3
EOF

It's more concise, plus unlike the redirect operator it can be combined with sudo if you need to write to files with root permissions.

Vadose answered 13/6, 2013 at 17:35 Comment(6)

I'd suggest adding > /dev/null at the end of the first line to prevent the contents of the here file being displayed to stdout when it's created. – Divulgence 19/6, 2013 at 12:50

Nice, thank you! Though that negates some of the conciseness of using tee in the first place I guess. – Vadose 19/6, 2013 at 17:2

True, but your solution appealed to me because of its compatibility with sudo, rather than because of its brevity :-) – Divulgence 24/6, 2013 at 13:32

How would you use this method to append to an existing file? – Bountiful 21/7, 2013 at 4:9

@Bountiful Check out man tee. Use the -a flag to append instead of overwrite. – Vadose 23/7, 2013 at 17:47

For use in a config script that I sometimes need to oversee, I like this one more because it prints the contents. – Sculptor 29/7, 2013 at 18:12

Note:

the following condenses and organizes other answers in this thread, esp the excellent work of Stefan Lasiewski and Serge Stroobandt
Lasiewski and I recommend Ch 19 (Here Documents) in the Advanced Bash-Scripting Guide

The question (how to write a here document (aka heredoc) to a file in a bash script?) has (at least) 3 main independent dimensions or subquestions:

Do you want to overwrite an existing file, append to an existing file, or write to a new file?
Does your user or another user (e.g., root) own the file?
Do you want to write the contents of your heredoc literally, or to have bash interpret variable references inside your heredoc?

(There are other dimensions/subquestions which I don't consider important. Consider editing this answer to add them!) Here are some of the more important combinations of the dimensions of the question listed above, with various different delimiting identifiers--there's nothing sacred about EOF, just make sure that the string you use as your delimiting identifier does not occur inside your heredoc:

To overwrite an existing file (or write to a new file) that you own, substituting variable references inside the heredoc:

cat << EOF > /path/to/your/file
This line will write to the file.
${THIS} will also write to the file, with the variable contents substituted.
EOF

To append an existing file (or write to a new file) that you own, substituting variable references inside the heredoc:

cat << FOE >> /path/to/your/file
This line will write to the file.
${THIS} will also write to the file, with the variable contents substituted.
FOE

To overwrite an existing file (or write to a new file) that you own, with the literal contents of the heredoc:

cat << 'END_OF_FILE' > /path/to/your/file
This line will write to the file.
${THIS} will also write to the file, without the variable contents substituted.
END_OF_FILE

To append an existing file (or write to a new file) that you own, with the literal contents of the heredoc:

cat << 'eof' >> /path/to/your/file
This line will write to the file.
${THIS} will also write to the file, without the variable contents substituted.
eof

To overwrite an existing file (or write to a new file) owned by root, substituting variable references inside the heredoc:

cat << until_it_ends | sudo tee /path/to/your/file
This line will write to the file.
${THIS} will also write to the file, with the variable contents substituted.
until_it_ends

To append an existing file (or write to a new file) owned by user=foo, with the literal contents of the heredoc:

cat << 'Screw_you_Foo' | sudo -u foo tee -a /path/to/your/file
This line will write to the file.
${THIS} will also write to the file, without the variable contents substituted.
Screw_you_Foo

Hettie answered 18/9, 2014 at 3:11 Comment(5)

#6 is best. But how do you overwrite contents of existing file with #6? – Stele 21/4, 2015 at 13:47

@Aleksandr Makov: how do you overwrite contents of existing file with #6? Omit the -a == --append; i.e., tee -a -> tee. See info tee (I'd quote it here, but comment markup is too limited. – Hettie 22/4, 2015 at 19:3

Is there a benefit to #6 using cat and piping to tee instead of sudo tee /path/to/your/file << 'Screw_you_Foo'? – Mabel 10/3, 2017 at 1:27

Why FOE instead of EOF in the append example? – Antediluvian 6/8, 2018 at 8:47

@becko: just to illustrate that the label is just a label. Note that I used a different label in each example. – Hettie 1/9, 2018 at 22:1

To build on @Livven's answer, here are some useful combinations.

variable substitution, leading tab retained, overwrite file, echo to stdout
```
tee /path/to/file <<EOF
${variable}
EOF
```
no variable substitution, leading tab retained, overwrite file, echo to stdout
```
tee /path/to/file <<'EOF'
${variable}
EOF
```
variable substitution, leading tab removed, overwrite file, echo to stdout
```
tee /path/to/file <<-EOF
    ${variable}
EOF
```
variable substitution, leading tab retained, append to file, echo to stdout
```
tee -a /path/to/file <<EOF
${variable}
EOF
```
variable substitution, leading tab retained, overwrite file, no echo to stdout
```
tee /path/to/file <<EOF >/dev/null
${variable}
EOF
```

the above can be combined with sudo as well

sudo -u USER tee /path/to/file <<EOF
${variable}
EOF

Zigzagger answered 2/8, 2016 at 21:48 Comment(0)

When root permissions are required

When root permissions are required for the destination file, use |sudo tee instead of >:

cat << 'EOF' |sudo tee /tmp/yourprotectedfilehere
The variable $FOO will *not* be interpreted.
EOF

cat << "EOF" |sudo tee /tmp/yourprotectedfilehere
The variable $FOO *will* be interpreted.
EOF

Breban answered 13/2, 2014 at 20:55 Comment(4)

Is it possible to pass variables to here documents? How could you get it so $FOO was interpreted? – Hydromagnetics 24/7, 2014 at 17:3

Below I have attempted to combine and organize this answer with that of Stefan Lasiewski. – Hettie 18/9, 2014 at 18:32

@Hydromagnetics Just don't enclose EOF in single quotes. Then $FOO will be interpreted. – Thalamus 1/2, 2017 at 8:46

You can also use | sudo cat > instead of | sudo tee if you don't want the input to be printed back to the stdout again. Of course, now you're using cat twice and doubly invoking that "unnecessary use of cat" meme, probably. – Illbehaved 19/2, 2021 at 22:43

For future people who may have this issue the following format worked:

(cat <<- _EOF_
        LogFile /var/log/clamd.log
        LogTime yes
        DatabaseDirectory /var/lib/clamav
        LocalSocket /tmp/clamd.socket
        TCPAddr 127.0.0.1
        SelfCheck 1020
        ScanPDF yes
        _EOF_
) > /etc/clamd.conf

Pearcy answered 1/6, 2010 at 21:2 Comment(5)

Don't need the parentheses: cat << END > afile followed by the heredoc works perfectly well. – Bias 2/6, 2010 at 0:12

Thanks, this actually solved another issue I ran into. After a few here docs there was some issues. I think it had to do with the parens, as with the advice above it fixed it. – Pearcy 7/6, 2010 at 17:53

This won't work. The output redirection needs to be at the end of the line which starts with cat as shown in the accepted answer. – Rustic 1/5, 2013 at 20:7

@DennisWilliamson It works, that's what the parens are for. The whole cat runs inside a subshell, and all the output of the subshell is redirected to the file – Arrowroot 15/8, 2015 at 6:45

@Izkata: If you look at the edit history of this answer, the parentheses were removed before I made my comment and added back afterwards. glenn jackman's (and my) comment applies. – Rustic 15/8, 2015 at 10:21

For those looking for a pure bash solution (or a need for speed), here's a simple solution without cat:

# here-doc tab indented
{ read -r -d '' || printf >file '%s' "$REPLY"; } <<-EOF
        foo bar
EOF

or for an easy "mycat" function (and avoid leaving REPLY in environment):

mycat() {
  local REPLY
  read -r -d '' || printf '%s' "$REPLY"
}
mycat >file <<-EOF
        foo bar
EOF

Quick speed comparison of "mycat" vs OS cat (1000 loops >/dev/null on my OSX laptop):

mycat:
real    0m1.507s
user    0m0.108s
sys     0m0.488s

OS cat:
real    0m4.082s
user    0m0.716s
sys     0m1.808s

NOTE: mycat doesn't handle file arguments, it just handles the problem "write a heredoc to a file"

Impeccable answered 2/11, 2021 at 19:19 Comment(0)

As instance you could use it:

First(making ssh connection):

while read pass port user ip files directs; do
    sshpass -p$pass scp -o 'StrictHostKeyChecking no' -P $port $files $user@$ip:$directs
done <<____HERE
    PASS    PORT    USER    IP    FILES    DIRECTS
      .      .       .       .      .         .
      .      .       .       .      .         .
      .      .       .       .      .         .
    PASS    PORT    USER    IP    FILES    DIRECTS
____HERE

Second(executing commands):

while read pass port user ip; do
    sshpass -p$pass ssh -p $port $user@$ip <<ENDSSH1
    COMMAND 1
    .
    .
    .
    COMMAND n
ENDSSH1
done <<____HERE
    PASS    PORT    USER    IP
      .      .       .       .
      .      .       .       .
      .      .       .       .
    PASS    PORT    USER    IP    
____HERE

Third(executing commands):

Script=$'
#Your commands
'

while read pass port user ip; do
    sshpass -p$pass ssh -o 'StrictHostKeyChecking no' -p $port $user@$ip "$Script"

done <<___HERE
PASS    PORT    USER    IP
  .      .       .       .
  .      .       .       .
  .      .       .       .
PASS    PORT    USER    IP  
___HERE

Forth(using variables):

while read pass port user ip fileoutput; do
    sshpass -p$pass ssh -o 'StrictHostKeyChecking no' -p $port $user@$ip fileinput=$fileinput 'bash -s'<<ENDSSH1
    #Your command > $fileinput
    #Your command > $fileinput
ENDSSH1
done <<____HERE
    PASS    PORT    USER    IP      FILE-OUTPUT
      .      .       .       .          .
      .      .       .       .          .
      .      .       .       .          .
    PASS    PORT    USER    IP      FILE-OUTPUT
____HERE

Nevil answered 16/2, 2014 at 10:17 Comment(0)

I like the following method of basic redirection for its concision, readability and presentation in an indented script:

<<-End_of_file >file
→       foo bar
End_of_file

Where → is a tab character.

This is standard Bourne shell redirection without forking any cat or tee process. But it is not working with current bash even when called through /bin/sh. It is still working with /bin/zsh since more than 20 years.

Jamieson answered 15/11, 2018 at 19:1 Comment(2)

This does not write anything to the file – Balsamiferous 14/2, 2023 at 12:16

This is failing with current bash ( even /bin/sh ), but working with zsh. This is standard since the original Bourne shell. I was used to give this example in my course on shell to avoid stupid uses of cat or tee since the sh is doing it without creating a new process. – Jamieson 16/2, 2023 at 16:15

If you want to keep the heredoc indented for readability:

$ perl -pe 's/^\s*//' << EOF
     line 1
     line 2
EOF

The built-in method for supporting indented heredoc in Bash only supports leading tabs, not spaces.

Perl can be replaced with awk to save a few characters, but the Perl one is probably easier to remember if you know basic regular expressions.

Confederation answered 9/9, 2020 at 2:29 Comment(0)

In addition, if you're writing to a file, it can be a good idea to check whether or not your write succeeded for failed. For example:

if ! echo "contents" > ./file ; then
    echo "ERROR: failed to write to file" >& 2
    exit 1
fi

To do the same with heredoc, there are two possible approaches.

if ! cat > ./file << EOF
contents
EOF
then
    echo "ERROR: failed to write to file" >& 2
    exit 1
fi

if ! cat > ./file ; then
    echo "ERROR: failed to write to file" >& 2
    exit 1
fi << EOF
contents
EOF

You can test the error case in the above code by replacing the destination file ./file with /file (assuming you're not running as root).

Interfluent answered 23/8, 2022 at 14:32 Comment(0)

Hot tags

Godot Unity Godot Help Programming Godot 4.X GUI GDScript 3D 2D Physics CSharp Godot 3.X VR XR Projects C++

When root permissions are required

Recommended topics

Hot tags