What does IFS= do in this bash loop: `cat file | while IFS= read -r line; do ... done`

Question

I'm learning bash and I saw this construction:

cat file | while IFS= read -r line;
do
    ...
done

Can anyone explain what IFS= does? I know it's input field separator, but why is it being set to nothing?

Underwood

possible duplicate of Is there any difference in bash between "while read -r line; do ...; done < file` and `cat file | while IFS= read -r line; do ...; done`? – Nominal

@AvinashRaj I asked that question yesterday about difference between two loops. This is a different question and I'm asking only about what IFS= means. – Underwood

IFS= basically sets IFS to null. In your case it will act same as IFS=$'\n' – Eelgrass

It's a simple question, You could ask this to the previous question answerer (@Chepner). – Nominal

@Eelgrass Thanks. So if IFS is null, then it's the same as newline? – Underwood

Only in this case it is same. Sometimes input can be delimited by \0 using find -0 then only this IFS= helps. – Eelgrass

Related: When do I set IFS to a newline in Bash?. – Battle

Answer 1

IFS does many things but you are asking about that particular loop.

The effect in that loop is to preserve leading and trailing white space in line. To illustrate, first observe with IFS set to nothing:

$ echo " this   is a test " | while IFS= read -r line; do echo "=$line=" ; done
= this   is a test =

The line variable contains all the white space it received on its stdin. Now, consider the same statement with the default IFS:

$ echo " this   is a test " | while read -r line; do echo "=$line=" ; done
=this   is a test=

In this version, the white space internal to the line is still preserved. But, the leading and trailing white space have been removed.

What does -r do in read -r?

The -r option prevents read from treating backslash as a special character.

To illustrate, we use two echo commands that supply two lines to the while loop. Observe what happens with -r:

$ { echo 'this \\ line is \' ; echo 'continued'; } | while IFS= read -r line; do echo "=$line=" ; done
=this \\ line is \=
=continued=

Now, observe what happens without -r:

$ { echo 'this \\ line is \' ; echo 'continued'; } | while IFS= read line; do echo "=$line=" ; done
=this \ line is continued=

Without -r, two changes happened. First, the double-backslash was converted to a single backslash. Second, the backslash on the end of the first line was interpreted as a line-continuation character and the two lines were merged into one.

In sum, if you want backslashes in the input to have special meaning, don't use -r. If you want backslashes in the input to be taken as plain characters, then use -r.

Multiple lines of input

Since read takes input one line at a time, IFS behaves affects each line of multiple line input in the same way that it affects single line input. -r behaves similarly with the exception that, without -r, multiple lines can be combined into one line using the trailing backslash as shown above.

The behavior with multiple line input, however, can be changed drastically using read's -d flag. -d changes the delimiter character that read uses to mark the end of an input line. For example, we can terminate lines with a tab character:

$ echo $'line one \n line\t two \n line three\t ends here'
line one 
 line    two 
 line three      ends here
$ echo $'line one \n line\t two \n line three\t ends here' | while IFS= read -r -d$'\t' line; do echo "=$line=" ; done
=line one 
 line=
= two 
 line three=

Here, the $'...' construct was used to enter special characters like newline, \n and tab, \t. Observe that with -d$'\t', read divides its input into "lines" based on tab characters. Anything after the final tab is ignored.

How to handle the most difficult file names

The most important use of the features described above is to process difficult file names. Since the one character that cannot appear in path/filenames is the null character, the null character can be used to separate a list of file names. As an example:

while IFS= read -r -d $'\0' file
do
    # do something to each file
done < <(find ~/music -type f -print0)

Gt

This is useful. Any question about loop constructs in shell should [also] be referred to why-is-using-a-shell-loop-to-process-text-considered-bad-practice where all is made clear... – Ferrosilicon

Check your shell's man page, its read will probably accept -d '' (the empty string) to specify null-delimited data. – Deplorable