How to join multiple lines of filenames into one with custom delimiter

Question

How do I join the result of ls -1 into a single line and delimit it with whatever I want?

Intimist

Answer 1

paste -s -d joins lines with a delimiter (e.g. ","), and does not leave a trailing delimiter:

ls -1 | paste -sd "," -

Blacktail

Just as a note, the version of paste I tried requires a "-" argument at the end to tell it to read from STDIN. e.g. ls -1 | paste -s -d ":" - Not sure if that's universal with all versions of paste – Gamali

this one is better because it allows empty delimiter :) – Franklinfranklinite

Note paste gets - (standard input) as default, at least on my paste (GNU coreutils) 8.22. – Obstacle

i just upvoted it, this is and now it has the same votes as the selected answer. THIS IS THE ANSWER. no trailing delimeter – Towery

The empty delimiter can be specified using "\0", so paste -sd "\0" - worked for me! – Counterfactual

@BradParks, what's wrong with just paste -sd "" -? – Byelostok

@Franklinfranklinite not if it's empty 😖 – Gonick

How can I do it with no delimiter? – Gonick

This way, the delimiter can only be a single character. – Spancake

Answer 2

EDIT: Simply "ls -m" If you want your delimiter to be a comma

Ah, the power and simplicity !

ls -1 | tr '\n' ','

Change the comma "," to whatever you want. Note that this includes a "trailing comma" (for lists that end with a newline)

Vivie

+1, but a more elaborate version should handle last \n differently – Cabinda

If the file name contains a \n in it, this will replace that too. – Breastpin

@unicornaddict you can use -b or -q for ls to handle strange cases like you mention. – Vivie

@Cabinda that would be a nice touch. – Vivie

@mouviciel: You can just append a "\n" at the end, with say, ls -1 | tr "\\n" ","; echo – Knowle

@ShreevatsaR: he means to not append a trailing "," I believe. like so ls -1 | tr "\\n" "," | sed 's/\(.*\),/\1/' – Comyns

@Chris: your sed could be a little more efficient with the end-marker character: ls -1 | tr "\\n" "," | sed 's/,$//'; echo '' – Freese

Or just ls -1 | tr '\n' , – Veiled

Can I use two commas as the delimiter. It seems it always show only one. – Immanent

@ElgsQianChen I suppose you can. Add another comma in the command. Have you tried it? – Vivie

ls | paste -sd, - is the best way to do it - no need to worry about the trailing comma. No need for -1 in ls because it is implied when the output is not going to the terminal. – Weatherford

Using sed after tr seems just to remove last symbol seems unreasonable. I go with ls -1 | tr '\n' ',' | head -c -1 – Plio

This almost works, but as many mentioned, leaves the trailing comma. What worked for me in keeping with the simplicity: ls -1 | xargs | tr " " "," – Thrips

Answer 3

This replaces the last comma with a newline:

ls -1 | tr '\n' ',' | sed 's/,$/\n/'

ls -m includes newlines at the screen-width character (80th for example).

Mostly Bash (only ls is external):

saveIFS=$IFS; IFS=$'\n'
files=($(ls -1))
IFS=,
list=${files[*]}
IFS=$saveIFS

Using readarray (aka mapfile) in Bash 4:

readarray -t files < <(ls -1)
saveIFS=$IFS
IFS=,
list=${files[*]}
IFS=$saveIFS

Thanks to gniourf_gniourf for the suggestions.

Putamen

This will not take care of files with white spaces in the name. Try this one: dir=/tmp/testdir; rm -rf $dir && mkdir $dir && cd /$dir && touch "this is a file" this_is_another_file && ls -1 && files=($(ls -1)) && list=${files[@]/%/,} && list=${list%*,} && echo $list – Premonition

@dimir: Many of the answers to this question suffer from this problem. I have edited my answer to allow for filenames with tabs or spaces, but not newlines. – Putamen

Your bash version suffers from pathname expansions too. To build an array from lines, please consider using mapfile (Bash ≥4) as: mapfile -t files < <(ls -1). No need to fiddle with IFS. And it's shorter too. – Cicerone

And when you have your array, you can use IFS to join the fields: saveIFS=$IFS; IFS=,; list=${files[*]}; IFS=$saveIFS. Or use another method if you want a separator with more that one character. – Cicerone

Also, your concatenation list=${files[@]/%/,} will not quite work as expected, as you'll also get the first character of IFS after the commas (very likely a space). Sorry for the many comments, but there are a few things to fix here! – Cicerone

@gniourf_gniourf: I have included your suggestions in my answer. Thanks. – Putamen

A simple files=(*) will work fine. Also, a delayed evaluation of * will avoid playing around with IFS: IFS=, command eval 'list=${files[*]}'; echo "<$list>". Of course, IFS=, command eval echo '"${files[*]}"' is shorter, but doesn't set $list variable string. – Zoellick

Of course, expansion of * works with only one character. – Zoellick

Answer 4

I think this one is awesome

ls -1 | awk 'ORS=","'

ORS is the "output record separator" so now your lines will be joined with a comma.

Fashionable

This does not exclude the trailing delimiter. – Wayless

This is especially awesome due to handling multi-character record separators (e.g., " OR ") – Spreadeagle

Answer 5

Parsing ls in general is not advised, so alternative better way is to use find, for example:

find . -type f -print0 | tr '\0' ','

Or by using find and paste:

find . -type f | paste -d, -s

For general joining multiple lines (not related to file system), check: Concise and portable “join” on the Unix command-line.

Abrahamsen

While true, the other examples are general to any list that you want to combine into one line. – Fayefayette

Answer 6

The combination of setting IFS and use of "$*" can do what you want. I'm using a subshell so I don't interfere with this shell's $IFS

(set -- *; IFS=,; echo "$*")

To capture the output,

output=$(set -- *; IFS=,; echo "$*")

Gaskill

Do you have some more information regarding how set works? Looks a bit like voodoo to me. shallow look through man set didn't net me much information either. – Fletafletch

If you give set a bunch of arguments but no options, it sets the positional parameters ($1, $2, ...). -- is there to protect set in case the first argument (or filename in this case) happens to start with a dash. See the description of the -- option in help set. I find the positional parameters a convenient way to handle a list of things. I could also have implemented this with an array: output=$( files=(*); IFS=,; echo "${files[*]}" ) – Gaskill

This is great since it doesn't require executing any additional programs and it works with file names that contain spaces or even newlines. – Microhenry

@EhteshChoudhury As type set will tell you, set is a shell builtin. So, man set will not help, but help set will do. Answer: "-- Assign any remaining arguments to the positional parameters." – Laynelayney

After a set -- *. Delaying expansion of * one level you may get the correct output without the need of a sub shell: IFS=',' eval echo '"$*"'. Of course that will change the positional parameters. – Zoellick

Answer 7

Adding on top of majkinetor's answer, here is the way of removing trailing delimiter(since I cannot just comment under his answer yet):

ls -1 | awk 'ORS=","' | head -c -1

Just remove as many trailing bytes as your delimiter counts for.

I like this approach because I can use multi character delimiters + other benefits of awk:

ls -1 | awk 'ORS=", "' | head -c -2

EDIT

As Peter has noticed, negative byte count is not supported in native MacOS version of head. This however can be easily fixed.

First, install coreutils. "The GNU Core Utilities are the basic file, shell and text manipulation utilities of the GNU operating system."

brew install coreutils

Commands also provided by MacOS are installed with the prefix "g". For example gls.

Once you have done this you can use ghead which has negative byte count, or better, make alias:

alias head="ghead"

Righthander

Note: negative byte counts are only supported on certain versions of head, so this won't work on e.g. macos. – Winder

Thanks, for pointing that out. I have added a workaround for MacOS. – Righthander

Answer 8

Don't reinvent the wheel.

ls -m

It does exactly that.

Abshier

The OP wanted any delimiter so you would still need a tr to convert the commas. It also adds a space after the commas i.e. file1, file2, file3 – Eserine

so using ls -m and tr to remove the space after the comma you would do ls -m | tr -d ' ' – Nereid

that use of tr will delete spaces inside filenames. better to use sed 's/, /,/g – Gaskill

Answer 9

just bash

mystring=$(printf "%s|" *)
echo ${mystring%|}

Canaliculus

Slightly more efficient would be to use "printf -v mystring "%s|" *" - that avoids a fork for the $() – Hacker

But notably doesn't chomp the trailing |, @camh. – Sievers

Well, just bash and gnu coreutils printf – Pledget

@Hacker But printf -v will work only in bash, while the presented answer works on many shell types. – Zoellick

@Sievers Yes, that will remove the trailing |, provided that both lines are used: printf -v mystring "%s|" * ; echo ${mystring%|}. – Zoellick

Quoting the echoed variable ${mystring%|} seems like a good idea, though. – Zoellick

@sorontar The question is tagged "bash" and the answer my comment is on says "just bash". So why would I avoid bash? – Hacker

Answer 10

This command is for the PERL fans :

ls -1 | perl -l40pe0

Here 40 is the octal ascii code for space.

-p will process line by line and print

-l will take care of replacing the trailing \n with the ascii character we provide.

-e is to inform PERL we are doing command line execution.

0 means that there is actually no command to execute.

perl -e0 is same as perl -e ' '

Dolliedolloff

this still has one issue: it doesn't exclude the trailing delimiter – Gley

Answer 11

The sed way,

sed -e ':a; N; $!ba; s/\n/,/g'
  # :a         # label called 'a'
  # N          # append next line into Pattern Space (see info sed)
  # $!ba       # if it's the last line ($) do not (!) jump to (b) label :a (a) - break loop
  # s/\n/,/g   # any substitution you want

Note:

This is linear in complexity, substituting only once after all lines are appended into sed's Pattern Space.

@AnandRajaseka's answer, and some other similar answers, such as here, are O(n²), because sed has to do substitute every time a new line is appended into the Pattern Space.

To compare,

seq 1 100000 | sed ':a; N; $!ba; s/\n/,/g' | head -c 80
  # linear, in less than 0.1s
seq 1 100000 | sed ':a; /$/N; s/\n/,/; ta' | head -c 80
  # quadratic, hung

Nickolenicks

Answer 12

To avoid potential newline confusion for tr we could add the -b flag to ls:

ls -1b | tr '\n' ';'

Selfcontent

Answer 13

It looks like the answers already exist.

If you want a, b, c format, use ls -m ( Tulains Córdova’s answer)

Or if you want a b c format, use ls | xargs (simpified version of Chris J’s answer)

Or if you want any other delimiter like |, use ls | paste -sd'|' (application of Artem’s answer)

Nib

Answer 14

sed -e :a -e '/$/N; s/\n/\\n/; ta' [filename]

Explanation:

-e - denotes a command to be executed
:a - is a label
/$/N - defines the scope of the match for the current and the (N)ext line
s/\n/\\n/; - replaces all EOL with \n
ta; - goto label a if the match is successful

Taken from my blog.

Couplet

Answer 15

If you version of xargs supports the -d flag then this should work

ls  | xargs -d, -L 1 echo

-d is the delimiter flag

If you do not have -d, then you can try the following

ls | xargs -I {} echo {}, | xargs echo

The first xargs allows you to specify your delimiter which is a comma in this example.

Denudate

-d specifies the input delimiter with GNU xargs, so will not work. The second example exhibits the same issue as other solutions here of a stray delimiter at the end. – Dollfuss

Answer 16

ls produces one column output when connected to a pipe, so the -1 is redundant.

Here's another perl answer using the builtin join function which doesn't leave a trailing delimiter:

ls | perl -F'\n' -0777 -anE 'say join ",", @F'

The obscure -0777 makes perl read all the input before running the program.

sed alternative that doesn't leave a trailing delimiter

ls | sed '$!s/$/,/' | tr -d '\n'

Dollfuss

Answer 17

Python answer above is interesting, but the own language can even make the output nice:

ls -1 | python -c "import sys; print(sys.stdin.read().splitlines())"

Allocution

I can't believe a Python oneliner is the best way I can find for a multicharacter delimiter... but it is: python -c "import sys; print(' | '.join(sys.stdin.read().splitlines()))". POSIX failed me again. – Thoughtless

Answer 18

If Python3 is your cup of tea, you can do this (but please explain why you would?):

ls -1 | python -c "import sys; print(','.join(sys.stdin.read().splitlines()))"

Scheel

I don't know why the OP wants to do it, but I know why I need to do it: in order to copy all the filenames separated by a space to use them as parameters for rubocop, eslint, stylelint, haml-lint, etc. – Lean

Answer 19

You can use:

ls -1 | perl -pe 's/\n$/some_delimiter/'

Breastpin

This does not exclude the trailing delimiter. – Wayless

Answer 20

ls has the option -m to delimit the output with ", " a comma and a space.

ls -m | tr -d ' ' | tr ',' ';'

piping this result to tr to remove either the space or the comma will allow you to pipe the result again to tr to replace the delimiter.

in my example i replace the delimiter , with the delimiter ;

replace ; with whatever one character delimiter you prefer since tr only accounts for the first character in the strings you pass in as arguments.

Nereid

Answer 21

You can use chomp to merge multiple line in single line:

perl -e 'while (<>) { if (/\\$/ ) { chomp; } print ;}' bad0 >test

put line break condition in if statement.It can be special character or any delimiter.

Alb

Answer 22

Quick Perl version with trailing slash handling:

ls -1 | perl -E 'say join ", ", map {chomp; $_} <>'

To explain:

• perl -E: execute Perl with features supports (say, ...)
• say: print with a carrier return
• join ", ", ARRAY_HERE: join an array with ", "
• map {chomp; $_} ROWS: remove from each line the carrier return and return the result
• <>: stdin, each line is a ROW, coupling with a map it will create an array of each ROW

Erleneerlewine

Works as it should. – Lushy

Answer 23

Here is a solution that allows any arbitrary character sequence as the delimiter, and processes input as it arrives, without needing to buffer multiple lines.

---

This input:

line1
line2
[...]
lineN

becomes this output:

prefix line1 delim line2 delim [...] delim lineN suffix

(The spaces above are just used for clarity of display.)

By default, prefix is empty string and delim and suffix are each newline. They are only written for non-empty input.

Shell-only:

joinlines_sh()(
    # delim/prefix/suffix are printf formats
    delim=${1:-'\n'} suffix=${2:-'\n'} prefix=$3
    IFS=
    if read -r || [ -n "$REPLY" ]; then
        printf "$prefix"
        while
            printf '%s' "$REPLY"
            read -r || [ -n "$REPLY" ]
        do
            printf "$delim"
        done
        printf "$suffix"
    fi
)

With awk:

joinlines_awk()(
    # delim/prefix/suffix are awk strings
    awk '
        BEGIN { if (getline) print prefix $0 }
        { print delim $0 }
        END { if (NR) print suffix }
    ' ORS= delim="${1:-\n}" suffix="${2:-\n}" prefix="$3"
)

The two implementations should behave the same except if escape sequences are used that have different meanings in printf formats and awk strings.

---

Sample usage:

$ seq 5 | joinlines_awk
1
2
3
4
5
$ printf '' | joinlines_sh === '///\n'
$ printf 1 | joinlines_sh === '///\n'
1///
$ seq 5 | joinlines_awk ' :: '
1 :: 2 :: 3 :: 4 :: 5
$ seq 5 | joinlines_sh ', ' ' )\n' '( '
( 1, 2, 3, 4, 5 )
$ seq 5 | joinlines_sh '\0' '\0' | od -c
0000000   1  \0   2  \0   3  \0   4  \0   5  \0
0000012
$

---

To use another (single) character instead of newline as the input "line" separator:

• with bash, add the -d option to read
• with awk, set RS

Renfroe

passing nulls in awk strings is not portable. eg: busybox and original awk can't handle them as used here – Renfroe