Make xargs handle filenames that contain spaces

Question

$ ls *mp3 | xargs mplayer  

Playing Lemon.  
File not found: 'Lemon'  
Playing Tree.mp3.  
File not found: 'Tree.mp3'  

Exiting... (End of file)  

My command fails because the file "Lemon Tree.mp3" contains spaces and so xargs thinks it's two files. Can I make find + xargs work with filenames like this?

Art

Possible duplicate of How can I use xargs to copy files that have spaces and quotes in their names? – Jayejaylene

This question is also answered by https://mcmap.net/q/56981/-how-can-i-use-xargs-to-copy-files-that-have-spaces-and-quotes-in-their-names – Jayejaylene

Answer 1

The xargs command takes white space characters (tabs, spaces, new lines) as delimiters.

You can narrow it down only for the new line characters ('\n') with -d option like this:

ls *.mp3 | xargs -d '\n' mplayer

It works only with GNU xargs.

For MacOS:

ls *.mp3 | tr \\n \\0 | xargs -0 mplayer

The more simplistic and practically useful approach (when don't need to process the filenames further):

mplayer *.mp3

Erfert

Best answer for general use! This works even if your previous command is not "find" – Clayborn

On OS X, -E '\n' didn't have an effect for me, nor would I expect it to as it modified the eofstr and not the record separator. However, I was able to utilize the -0 flag as a solution, even if the previous command is not 'find', by simulating the effect of find's -print0 flag in my input, e.g.: ls *mp3 | tr '\n' '\0' | xargs -0 mplayer – Hughhughes

For OS X, you can "brew install findutils", which gives you the "gxargs" command that does have the -d switch. – Benitobenjamen

Can you confirm whether the second method works or not on OSX please? Thanks. – Erfert

Using ls in scipts is a bad idea. The proper workaround for this specific case is to simply mplayer *.mp3 instead of trying to use xargs. – Suntan

Working perfectly in CentOS. Thank you – Phony

ls *.mp3 | xargs -0 mplayer is invalid - ls does not output zero terminated output. Use printf "%s\0" *.mp3 | xargs -0 mplayer, but really, just mplayer *.mp3. – Khalil

for the printf-trick it might be didactically helpful to issue commands (set -x;echo *.mp3) to understand what is happening behind the scenes especially for filenames containing whitespaces. – Unvoice

Works like a charm when handling bulk file deletion inside a directory and sub directories in bash. – Linnlinnaeus

In Git Bash in Windows, I needed to use double quotes for the newline delimiter: | xargs -d "\n" – Pruter

I notice that adding -d '\n' increase the time of execution of xargs by x5 in my case. – Dashtikavir

How did I never know this? I know using find is quite sensible for doing stuff with lists of files, but sometimes your list is coming from a git command or whatever. This is super! – Capitulum

Answer 2

The xargs utility reads space, tab, newline and end-of-file delimited strings from the standard input and executes utility with the strings as arguments.

You want to avoid using space as a delimiter. This can be done by changing the delimiter for xargs. According to the manual:

 -0      Change xargs to expect NUL (``\0'') characters as separators,
         instead of spaces and newlines.  This is expected to be used in
         concert with the -print0 function in find(1).

Such as:

find . -name "*.mp3" -print0 | xargs -0 mplayer

To answer the question about playing the seventh mp3; it is simpler to run

mplayer "$(ls *.mp3 | sed -n 7p)"

Timoteo

This is using GNU find and GNU xargs; not all versions of those programs support those options (though there's a case to be made that they should). – Philippopolis

@JonathanLeffler s/GNU/FreeBSD/g; POSIX sadly is afraid of NUL characters in text files and hasn't had enough therapy yet :-) My advice in fact resorts to non-portable options. – Timoteo

And Mac OS X (a BSD derivative) has find with -print0 and xargs with -0. AFAIK, HP-UX, AIX and Solaris do not, however (but I stand to be corrected: HP-UX 11i didn't; Solaris 10 didn't; AIX 5.x didn't; but they're not current versions). It wouldn't be hard to change sed, for instance, to use 'lines' ending with '\0' instead of '\n', and the POSIX 2008 getdelim() would make it easy to manage. – Philippopolis

+1 + 1 trick for using with file paths containing list files : cat $file_paths_list_file | perl -ne 's|\n|\000|g;print'| xargs -0 zip $zip_package – Bereniceberenson

Good idea to replace the newlines with NUL - I had to do this on an embedded system that did not have GNU find nor GNU xargs nor perl - but the tr command can be leveraged to do the same: cat $file_paths_list_file | tr '\n' '\0' | xargs -0 du -hms – Manipulate

@YordanGeorgiev your comment (especially with @joensson's use of tr instead of perl) should be an answer! – Cully

@JonathanLeffler, although macos has -print0 and -0, it does not work when file name contains space. – Spermogonium

@user2807219: I'm curious to know how you demonstrate that. It works for me. If need be, send me email — see my profile — and include a link to this question and an MCVE that demonstrates how you run into problems. – Philippopolis

@JonathanLeffler, thanks for reply, i have tried again, the problem is not the space, find with xargs works on space filename even without -print0 and -0. My real problem is the folder names and file names are too long, then the execute result is "mv: rename {} to /pathTo/{}: No such file or directory". – Spermogonium

@user2807219: That's curious. It's also completely different. I did once have a problem with a software configuration test that failed on very, very deep directory hierarchies on a Mac — it was trying to test how deep they could go and got confused, assisted by a Mac OS X bug. But that would be unusual. Either create your own question outlining the problem or contact me offline with the details. There are probably ways to deal with it, but you'll need to know and explain why you have such long names, illustrate some examples (they won't fit in a comment), and explain what you're trying to do. – Philippopolis

Answer 3

Try

find . -name \*.mp3 -print0 | xargs -0 mplayer

instead of

ls | grep mp3 

Coward

Answer 4

xargs on MacOS doesn't have -d option, so this solution uses -0 instead.

Get ls to output one file per line, then translate newlines into nulls and tell xargs to use nulls as the delimiter:

ls -1 *mp3 | tr "\n" "\0" | xargs -0 mplayer

Duotone

Answer 5

Dick.Guertin's answer [1] suggested that one could escape the spaces in a filename is a valuable alternative to other solutions suggested here (such as using a null character as a separator rather than whitespace). But it could be simpler - you don't really need a unique character. You can just have sed add the escaped spaces directly:

ls | grep ' ' | sed 's| |\\ |g' | xargs ...

Furthermore, the grep is only necessary if you only want files with spaces in the names. More generically (e.g., when processing a batch of files some of which have spaces, some not), just skip the grep:

ls | sed 's| |\\ |g' | xargs ...

Then, of course, the filename may have other whitespace than blanks (e.g., a tab):

ls | sed -r 's|[[:blank:]]|\\\1|g' | xargs ...

That assumes you have a sed that supports -r (extended regex) such as GNU sed or recent versions of bsd sed (e.g., FreeBSD which originally spelled the option "-E" before FreeBSD 8 and supports both -r & -E for compatibility through FreeBSD 11 at least). Otherwise you can use a basic regex character class bracket expression and manually enter the space and tab characters in the [] delimiters.

[1] This is perhaps more appropriate as a comment or an edit to that answer, but at the moment I do not have enough reputation to comment and can only suggest edits. Since the latter forms above (without the grep) alters the behavior of Dick.Guertin's original answer, a direct edit is perhaps not appropriate anyway.

Blacktail

Answer 6

find . -name 'Lemon*.mp3' -print0 | xargs -­0 -i mplayer '{}' 

This helped in my case to delete different files with spaces. It should work too with mplayer. The necessary trick is the quotes. (Tested on Linux Xubuntu 14.04.)

Finisterre

Answer 7

I know that I'm not answering the xargs question directly but it's worth mentioning find's -exec option.

Given the following file system:

[root@localhost bokeh]# tree --charset assci bands
bands
|-- Dream\ Theater
|-- King's\ X
|-- Megadeth
`-- Rush

0 directories, 4 files

The find command can be made to handle the space in Dream Theater and King's X. So, to find the drummers of each band using grep:

[root@localhost]# find bands/ -type f -exec grep Drums {} +
bands/Dream Theater:Drums:Mike Mangini
bands/Rush:Drums: Neil Peart
bands/King's X:Drums:Jerry Gaskill
bands/Megadeth:Drums:Dirk Verbeuren

In the -exec option {} stands for the filename including path. Note that you don't have to escape it or put it in quotes.

The difference between -exec's terminators (+ and \;) is that + groups as many file names that it can onto one command line. Whereas \; will execute the command for each file name.

So, find bands/ -type f -exec grep Drums {} + will result in:

grep Drums "bands/Dream Theater" "bands/Rush" "bands/King's X" "bands/Megadeth"

and find bands/ -type f -exec grep Drums {} \; will result in:

grep Drums "bands/Dream Theater"
grep Drums "bands/Rush"
grep Drums "bands/King's X"
grep Drums "bands/Megadeth"

In the case of grep this has the side effect of either printing the filename or not.

[root@localhost bokeh]# find bands/ -type f -exec grep Drums {} \;
Drums:Mike Mangini
Drums: Neil Peart
Drums:Jerry Gaskill
Drums:Dirk Verbeuren

[root@localhost bokeh]# find bands/ -type f -exec grep Drums {} +
bands/Dream Theater:Drums:Mike Mangini
bands/Rush:Drums: Neil Peart
bands/King's X:Drums:Jerry Gaskill
bands/Megadeth:Drums:Dirk Verbeuren

Of course, grep's options -h and -H will control whether or not the filename is printed regardless of how grep is called.

---

xargs

xargs can also control how man files are on the command line.

xargs by default groups all the arguments onto one line. In order to do the same thing that -exec \; does use xargs -l. Note that the -t option tells xargs to print the command before executing it.

[root@localhost bokeh]# find ./bands -type f  | xargs -d '\n' -l -t grep Drums
grep Drums ./bands/Dream Theater 
Drums:Mike Mangini
grep Drums ./bands/Rush 
Drums: Neil Peart
grep Drums ./bands/King's X 
Drums:Jerry Gaskill
grep Drums ./bands/Megadeth 
Drums:Dirk Verbeuren

See that the -l option tells xargs to execute grep for every filename.

Versus the default (i.e. no -l option):

[root@localhost bokeh]# find ./bands -type f  | xargs -d '\n'  -t grep Drums
grep Drums ./bands/Dream Theater ./bands/Rush ./bands/King's X ./bands/Megadeth 
./bands/Dream Theater:Drums:Mike Mangini
./bands/Rush:Drums: Neil Peart
./bands/King's X:Drums:Jerry Gaskill
./bands/Megadeth:Drums:Dirk Verbeuren

xargs has better control on how many files can be on the command line. Give the -l option the max number of files per command.

[root@localhost bokeh]# find ./bands -type f  | xargs -d '\n'  -l2 -t grep Drums
grep Drums ./bands/Dream Theater ./bands/Rush 
./bands/Dream Theater:Drums:Mike Mangini
./bands/Rush:Drums: Neil Peart
grep Drums ./bands/King's X ./bands/Megadeth 
./bands/King's X:Drums:Jerry Gaskill
./bands/Megadeth:Drums:Dirk Verbeuren
[root@localhost bokeh]# 

See that grep was executed with two filenames because of -l2.

Shackle

Answer 8

On macOS 10.12.x (Sierra), if you have spaces in file names or subdirectories, you can use the following:

find . -name '*.swift' -exec echo '"{}"' \; |xargs wc -l

Disunion

Answer 9

ls | grep mp3 | sed -n "7p" | xargs -i mplayer {}

Note that in the command above, xargs will call mplayer anew for each file. This may be undesirable for mplayer, but may be okay for other targets.

Eldwun

A useful addition to the existing answers, but it would be worth noting that this will cause mplayer to be called anew for each file. It matters if you try e.g. ... | xargs -I{} mplayer -shuffle {}: this will play in a completely deterministic order, despite -shuffle. – Wetzell

It's probably usually not the intent. xargs is mostly used with commands that accept a list of file names (easy example: rm), and attempts to pass as many file names as it can fit into each invocation, only splitting into multiple invocations if needed. You can see the difference when you use a command where each invocation is visible, such as echo (the default): seq 0 100000 | xargs prints all numbers from 0 to 23695 (platform-specific, but that's what happens on my system) on the first line, to 45539 on line 2, etc. And you're right, for most commands, it won't matter. – Wetzell

Answer 10

I recently use the following solution:

$ find <some_path> | sed -e 's/^/"/' -e 's/$/"/' | xargs <command>

ex: $ ls *.mp3 | sed -e 's/^/"/' -e 's/$/"/' | xargs mplayer

this make all files accessible

Hilmahilt

Answer 11

It depends on (a) how attached you are to the number 7 as opposed to, say, Lemons, and (b) whether any of your file names contain newlines (and whether you're willing to rename them if they do).

There are many ways to deal with it, but some of them are:

mplayer Lemon*.mp3

find . -name 'Lemon*.mp3' -exec mplayer {} ';'

i=0
for mp3 in *.mp3
do
    i=$((i+1))
    [ $i = 7 ] && mplayer "$mp3"
done

for mp3 in *.mp3
do
    case "$mp3" in
    (Lemon*) mplayer "$mp3";;
    esac
done

i=0
find . -name *.mp3 |
while read mp3
do
    i=$((i+1))
    [ $i = 7 ] && mplayer "$mp3"
done

The read loop doesn't work if file names contain newlines; the others work correctly even with newlines in the names (let alone spaces). For my money, if you have file names containing a newline, you should rename the file without the newline. Using the double quotes around the file name is key to the loops working correctly.

If you have GNU find and GNU xargs (or FreeBSD (*BSD?), or Mac OS X), you can also use the -print0 and -0 options, as in:

find . -name 'Lemon*.mp3' -print0 | xargs -0 mplayer

This works regardless of the contents of the name (the only two characters that cannot appear in a file name are slash and NUL, and the slash causes no problems in a file path, so using NUL as the name delimiter covers everything). However, if you need to filter out the first 6 entries, you need a program that handles 'lines' ended by NUL instead of newline...and I'm not sure there are any.

The first is by far the simplest for the specific case on hand; however, it may not generalize to cover your other scenarios that you've not yet listed.

Philippopolis

Answer 12

Alternative solutions can be helpful...

You can also add a null character to the end of your lines using Perl, then use the -0 option in xargs. Unlike the xargs -d '\n' (in approved answer) - this works everywhere, including OS X.

For example, to recursively list (execute, move, etc.) MPEG3 files which may contain spaces or other funny characters - I'd use:

find . | grep \.mp3 | perl -ne 'chop; print "$_\0"' | xargs -0  ls

(Note: For filtering, I prefer the easier-to-remember "| grep" syntax to "find's" --name arguments.)

Chemaram

Answer 13

Given the specific title of this post, here's my suggestion:

ls | grep ' ' | tr ' ' '<' | sed 's|<|\\ |g'

The idea is to convert blanks to any unique character, like '<', and then change that into '\ ', a backslash followed by a blank. You can then pipe that into any command you like, such as:

ls | grep ' ' | tr ' ' '<' | sed 's|<|\\ |g' | xargs -L1 GetFileInfo

The key here lies in the 'tr' and 'sed' commands; and you can use any character besides '<', such as '?' or even a tab-character.

Tarr

What's the purpose of the detour via tr? Why not just ls *.mp3 | sed -n '7!b;s/\([[:space:]]\)/\\\1/g;p'? – Suntan

I've found that "tr ' ' '?'" eliminates the need for "sed". The single "?" character is non-blank, but matches ANY single character, in this case: blank. The odds of it being something else are quite small, and acceptable since you're trying to process ALL files ending in .mp3: "ls | grep ' ' | tr ' ' '?' | xargs -L1 GetFileInfo" – Valarievalda

You can also handle "tab" at the same time: tr ' \t' '??' handles both. – Valarievalda

Answer 14

On macOS (Monterey/12 forward and I'm not sure how far back past 10.15/Catalina), if you have spaces in file names or subdirectories, you can use the following:

mdfind  -0 -onlyin . -name .txt | xargs -0 grep stackoverflow |  wc -l

As Jen's answer notes:

The xargs utility reads space, tab, newline and end-of-file delimited strings from the standard input and executes utility with the strings as arguments.

You want to avoid using space as a delimiter. This can be done by changing the delimiter for xargs. According to the manual:

 -0      Change xargs to expect NUL (``\0'') characters as separators,
         instead of spaces and newlines.  This is expected to be used in
         concert with the -print0 function in find(1).

Tiflis

Answer 15

ls *mp3 | xargs -I{} mplayer {}

As long as you do not mind mplayer running once per line.

It runs once per newline-terminated line. Space is no longer additionally used as a delimiter. I cannot get the MacOS/BSD xargs to not use space as a delimiter in any other way (-L 1 does not work).

Eterne