What's the best way to join two lists?

Question

I have two lists that contain some data (numeric data or/and strings)?

How do I join these two lists, assuming that the lists do not contain sublists?

Which choice is preferred and why?

1. set first [concat $first $second]

2. lappend first $second

3. append first " $second"

Ludhiana

1. Wrong. Don't do that. 2. Wrong. Append the entire second list as a single element to the first list. 3. Wrong. Does the same as 1. -- I suggest you use lappend first {*}$second – Diapositive

Johannes Kuhnthanks for the help. But what about tcl 8.4? – Ludhiana

For 8.4 that would be foreach x $second { lappend first $x } – Citronellal

or evil eval: eval [linsert $second 0 lappend first] – Diapositive

@Kuhn: please enlighten us on why #1 is wrong. Is there any case that it fails to work? – Courtly

@HaiVu, I was just reading wiki.tcl.tk/concat and one phrase jumped out at me: "concat is defined in terms of string concatenation". This might surprise: set l [concat a b \{ c]; llength $l errors with unmatched open brace in list – Janot

@glennjackman, I believe you are most probably wrong on this one: the implementation of concat should be smart enough to detect pure lists and avoid shimmering in this case, just literally concatenation them. In the example you cited neither of the arguments is a pure list, so the first step concat does is to get list rep. of them, and fails to do this with the third one, obviously. Just replace \{ with [list \{] and see it working. – Citronellal

@glennjackman, just to be clear: the OP stated they are operating on lists, so we can assume the arguments to concat are indeed lists. – Citronellal

The error does not occur with concat, but with llength. I was simply pointing out one of the gotcha's associated with concat. – Janot

Answer 1

It is fine to use concat and that is even highly efficient in some cases (it is the recommended technique in 8.4 and before, and not too bad in later versions). However, your second option with lappend will not work at all, and the version with append will work, but will also be horribly inefficient.

Other versions that will work:

# Strongly recommended from 8.6.1 on
set first [list {*}$first {*}$second]

lappend first {*}$second

The reason why the first of those is recommended from 8.6.1 onwards is that the compiler is able to optimise it to a direct "list-concatenate" operation.

Caddaric

I refer to this answer a few times a month. Should probably nail it to my screen. – Waldrup

Answer 2

Examples

% set first {a b c}
a b c
% set second {1 2 3}
1 2 3
% set first [concat $first $second]; # #1 is correct
a b c 1 2 3
% set first {a b c}
a b c
% lappend first $second; # #2 is wrong: appends the whole `second` list to `first
a b c {1 2 3}

Discussion

I looked up the documentation, also experiment with some lists and found out that:

1. Your first choice, concat is correct
2. lappend does not work because it treats $second as one element, not a list
3. append works, but you are treating your lists as string. I don't know what the implications are, but it does not communicate the intention that first and second are lists.

Courtly

Answer 3

Maybe a bit old, but wanted to clarify;

As already stated, the standard way to merge 2 lists is via concat pre-v8.6. However please note that concat gets very inefficient when dealing with long lists, since it analyzes the lists as part of the merge. eg when merging lists, the larger they get the slower they merge.

Both appends do not merge "lists", they just add to an existing list (lappend) or variable (append). Both appends have no impact to speed, since they do not analyze anything when appending.

If merging single entry list elements, one could merge them via set first [join [lappend first $second]] but only if dealing with simple/single elements within each list (ie no spaces per element).

Photolithography

Answer 4

To add to the other answers, I ran a rough benchmark comparing the different versions (tclsh 8.6.13).

#! /usr/bin/env tclsh

set a {1}

for {set i 0} {$i < 25} {incr i} {
  switch $argv {
    list {
      set a [list {*}$a {*}$a]
    }

    concat {
      set a [concat $a $a]
    }

    lappend {
      lappend a {*}$a
    }

    append {
      append a " $a"
    }
  }
}

Results:

./test.tcl lappend  0.28s user 0.51s system 99% cpu 0.795 total
./test.tcl list     0.22s user 0.29s system 99% cpu 0.511 total
./test.tcl append   0.04s user 0.08s system 99% cpu 0.115 total
./test.tcl concat   0.04s user 0.08s system 99% cpu 0.112 total

Note that the semantics aren't quite the same between the different versions. For example, list will re-quote list elements.

Chronometry