How to cope with spaces in file names when iterating results from git diff --name-only

V

5

13

A script I am working on needs to go through each file from a git diff. However, I don't know how to deal with spaces in the file name. Any files that have a space are split into "2 files". I know they need to be wrapped in " " but I don't know how to achieve that before it goes to the @ param.

When there are spaces in filenames, how should I iterate over the files from

git diff --name-only  $1

?

Here is a simple test that reproduces the error:

copyfiles()
{
    echo "Copying added files"
    for file in $@; do

        new_file=$(echo ${file##*/})

        directory=$(echo ${file%/*})
        echo "Full Path is is  $file"
        echo "File is  $new_file"
        echo "Directory is  $directory"
        cp $file $COPY_TO
    done    
}

COPY_TO="testDir"
DIFF_FILES=$( git diff --name-only  $1) 
copyfiles $DIFF_FILES

The script is currently run like:

test.sh <git commit id>

Villosity answered 23/1, 2015 at 12:3 Comment(0)

I

4

Use -z to get git-diff to use null terminators. For example:

export COPY_TO
git diff -z --name-only | xargs -0 sh -c 'for file; do
    new_file=$(echo ${file##*/})
    directory=$(echo ${file%/*})
    echo "Full Path is is  $file"
    echo "File is  $new_file"
    echo "Directory is  $directory"
    cp "$file" "$COPY_TO"
done' sh

Note that the more reasonable solution is to refuse pull requests from people who create files with whitespace in the name.

Implacental answered 23/1, 2015 at 12:23 Comment(9)

love the reasonable solution, I cant understand why source files end up with white space! – Villosity 23/1, 2015 at 13:34

can I ask why just adding the "-z" to my current script didnt work? – Villosity 23/1, 2015 at 13:34

I can't work out how to use this in a way that allows me to run a function on each file.. – Villosity 23/1, 2015 at 14:20

If you are using bash, you can export the function with export -f and then do xargs -0 -I {} bash -c 'function_name {}'. This invokes the function once for each file rather than passing multiple filenames. I highly advise against this though, because exporting functions is wanky. Put it in a shell script instead. – Implacental 23/1, 2015 at 18:9

Easier to just do xargs -0 bash -c 'function_name "$@"' bash, though. That will invoke the function with multiple arguments. The important thing is export -f function_name – Implacental 23/1, 2015 at 18:43

If i send it as multiple arguments would i not have the same problem with the spaces? When you say put it in a shell script do you mean a separate one that i invoke? At the moment i just copy and pasted the function contents into the ' ' and it does work, just feels a touch untidy :p – Villosity 25/1, 2015 at 11:31

Using multiple arguments is only a problem if your function uses for file in $@ instead of for file; do or for file in "$@";do – Implacental 26/1, 2015 at 2:11

Oh, how strange, is always best to use "do" then? – Villosity 26/1, 2015 at 6:49

The "do" is required syntax for the for loop. The important distinction is the double quotes around $@. I mention the "do" in the second example to clarify that for file; do is identical to for file in "$@"; do, which behaves very differently from for file in $@; do. – Implacental 26/1, 2015 at 9:52

S

7

git diff -z --name-only |
while read -d $'\0' file
do
    echo ${file}
done

Sylph answered 2/2, 2018 at 12:44 Comment(1)

You have to be careful: read -d only works in bash, not sh. – Hatchet 6/10, 2020 at 15:44

H

6

The output from --name-only is subject to a certain amount of escaping. Unfortunately it is awkward to work with.

git diff explains the escaping (and an alternative) under the -z option:

-z

When --raw, --numstat, --name-only or --name-status has been given, do not munge pathnames and use NULs as output field terminators.

Without this option, each pathname output will have TAB, LF, double quotes, and backslash characters replaced with \t, \n, \", and \, respectively, and the pathname will be enclosed in double quotes if any of those replacements occurred.

An example:

$ git init ugh
$ cd ugh
$ touch 'spa ce' $'new\nline' $'t\tab'
$ ls # Unhelpful really
new?line  spa ce  t?ab
$ ls --quote # Minorly helpful but wrong (for shell usage)
"new\nline"  "spa ce"  "t\tab"
$ git add -A
$ git diff --cached --name-only
"new\nline"
spa ce
"t\tab"
$ git diff --cached --name-only -z # Doesn't copy and paste well and is a bit confusing to read this way
new
line^@spa ce^@t ab^@
$ printf %q\\n "$(git diff --cached --name-only -z )"
$'new\nlinespa cet\tab'

Anyway, the point here is that the best way to do this is to use the -z output and read the list of files with read.

while IFS= read -r -d '' file; do
    printf 'file = %q\n' "$file"
done < <(git diff --cached --name-only -z)

You could also pipe the output from git diff to the while loop but if you need variables from inside the loop once the loop is done you need this Process Substitution method to avoid the subshell problems with the pipe methodD.

Homocentric answered 23/1, 2015 at 12:26 Comment(1)

This answer was super helpful. Thank you. It allowed me to pass the outpuf from git diff --name-only -z as the input into git diff/git difftool. I demo that here: https://mcmap.net/q/450141/-how-to-cope-with-spaces-in-file-names-when-iterating-results-from-git-diff-name-only – Birch 11/7, 2020 at 20:50

I

4

Use -z to get git-diff to use null terminators. For example:

export COPY_TO
git diff -z --name-only | xargs -0 sh -c 'for file; do
    new_file=$(echo ${file##*/})
    directory=$(echo ${file%/*})
    echo "Full Path is is  $file"
    echo "File is  $new_file"
    echo "Directory is  $directory"
    cp "$file" "$COPY_TO"
done' sh