Concatenation of tables in Lua

Question

ORIGINAL POST

Given that there is no built in function in Lua, I am in search of a function that allows me to append tables together. I have googled quite a bit and have tried every solutions I stumbled across but none seem to work properly.

The scenario goes like this: I am using Lua embeded in an application. An internal command of the application returns a list of values in the form of a table.

What I am trying to do is call that command recursively in a loop and append the returned values, again in the form of a table, to the table from previous iterations.

---

EDIT

For those who come across this post in the future, please note what @gimf posted. Since Tables in Lua are as much like arrays than anything else (even in a list context), there is no real correct way to append one table to another. The closest concept is merging of tables. Please see the post, "Lua - merge tables?" for help in that regard.

Exorable

Possible dupe: https://mcmap.net/q/235361/-how-to-merge-two-tables-overwriting-the-elements-which-are-in-both. You mention "recursivly in a loop". Do you search for a deep-copy + merge? – Prominence

The following are the links I found that offered solutions: ardoris.wordpress.com/2008/08/10/… idevgames.com/forum/archive/index.php/t-10223.html Though I understand the approach of each, neither seem to work. Do you have a working solution? – Exorable

gimpf, maybe I am not being completely clear. Merging tables and concatinating tables are similar but very different. I am interested in appending one table to another, thus the use of the word concatenate. – Exorable

Please see my edit; an example of what you want to do in the form of 3 lua tables (2 in, 1 out) would be very helpful. – Prominence

Answer 1

Overcomplicated answers much?

Here is my implementation:

function TableConcat(t1,t2)
    for i=1,#t2 do
        t1[#t1+1] = t2[i]
    end
    return t1
end

Hachman

wouldn't ipairs iteration with table.insert be better (more readable and/or faster)? – Bibby

ipairs is barely more costly than normal, the reason not to use it is it doesn't guarantee order of items in the table. the reason not to use insert is it's dramatically more costly than a standard index set on a table, because insert will call a routine that pushes values in the table back from the index it was called on, while there are no values past [#t+1], the routine is still called, causing a performance issue, on a compiled language, there is no difference, but using an interpreted language, we have to be careful what all we ask the computer to do for us – Hachman

From what i know, ipairs guarantees iteration order be for i=1 ... till the first t[i]==nil, no? Which for non-degenerate cases is same as for i=1,#t. Re insert vs indexing set though, you are right - i measured and there is 5-6x performance difference – Bibby

I'm pretty sure I read somewhere that pairs doesn't guarantee order, but you may be right, although it's most likely a safe assumption that it will be ordered, whether guaranteed or not- The code is correct, it should not be [#t1+i], because #t1 will increase as the table size goes up, leaving a sparse table (almost always a bad thing in Lua) – Hachman

NB: ipairs ~= pairs. There is a reason for the "i". pairs is disorderly, ipairs isn't – Bibby

@Cyanogen no bug. t1 is the table being inserted into, thus #t1+1 is the length of t1 plus 1 (i.e. where you should be inserting the next item). – Carissacarita

@Carissacarita Right! Should I delete my comment to avoid confusion? – Cyanogen

@Cyanogen might be an idea to update your comment to show its not a bug. If you can't then I'm not sure. – Carissacarita

@Carissacarita Yeah, I couldn't, so I deleted it. Yet. this discussion remains as archeological evidence for its previous existence :) Thanks for catching that! – Cyanogen

Shouldn't the third line be t1[#t1+i] = t2[i]? – Yon

The fact that this both mutates the first argument AND returns it is a little confusing, although I guess the OP didn't say what interface they wanted. – Baziotes

the third line should be 't1[#t1+1] = t2[i]', otherwise in the 2nd iteration #t1 is 2 and i is 2 and the index 3 is skipped. I'll edit the answer – Clericals

Does not work for t1={a={1},2} t2={3,d={4}} – Jetta

Answer 2

If you want to concatenate an existing table to a new one, this is the most concise way to do it:

local t = {3, 4, 5}
local concatenation = {1, 2, table.unpack(t)}

Although I'm not sure how good this is performance-wise.

Intercommunicate

Just note that this only work if table.unpack is the last argument (unlike Python's *t or JavaScript's ...t), see #37372682 – Ashraf

Answer 3

And one more way:

for _,v in ipairs(t2) do 
    table.insert(t1, v)
end

It seems to me the most readable one - it iterates over the 2nd table and appends its values to the 1st one, end of story. Curious how it fares in speed to the explicit indexing [] above

Bibby

Does not work for t1={a={1},2} t2={3,d={4}} – Jetta

Answer 4

A simple way to do what you want:

local t1 = {1, 2, 3, 4, 5}
local t2 = {6, 7, 8, 9, 10}

local t3 = {unpack(t1)}
for I = 1,#t2 do
    t3[#t1+I] = t2[I]
end

Polacre

why the {unpack(t1)} ?! all it does i make copy of t1 but question implied updating in-place? – Bibby

@NasBanov He asked how to concatenate. When you concatenate strings you get a new string. I assumed he wanted something like that. – Polacre

hmm, you are right regarding the word "concatenation" in the title. But the question speaks about "appending", which is a mutator. Still { unpack(tbl) } is a neat trick for cloning a table - with limitations (something like 1M elements) – Bibby

@NasBanov However this contradiction in terms is the questioner's fault, so I see accepting either answer as valid and correct is fine. – Polacre

Answer 5

To add two tables together do this

    ii=0
for i=#firsttable, #secondtable+#firsttable do
    ii=ii+1
    firsttable[i]=secondtable[ii]
end

use the first table as the variable you wanted to add as code adds the second one on to the end of the first table in order.

• i is the start number of the table or list.
• #secondtable+#firsttable is what to end at.

It starts at the end of the first table you want to add to, and ends at the end of the second table in a for loop so it works with any size table or list.

Readjust

This is wrong. You have to start with i=(#firsttable+1), or you will munch over the last element in the first table. In case of the first table being empty you will even try to access firsttable[0], but arrays are indexed starting with 1 in lua. – Sienna

Answer 6

In general the notion of concatenating arbitrary tables does not make sense in Lua because a single key can only have one value.

There are special cases in which concatenation does make sense. One such is for tables containing simple arrays, which might be the natural result of a function intended to return a list of results.

In that case, you can write:

-- return a new array containing the concatenation of all of its 
-- parameters. Scalar parameters are included in place, and array 
-- parameters have their values shallow-copied to the final array.
-- Note that userdata and function values are treated as scalar.
function array_concat(...) 
    local t = {}
    for n = 1,select("#",...) do
        local arg = select(n,...)
        if type(arg)=="table" then
            for _,v in ipairs(arg) do
                t[#t+1] = v
            end
        else
            t[#t+1] = arg
        end
    end
    return t
end

This is a shallow copy, and makes no attempt to find out if a userdata or function value is a container or object of some kind that might need different treatment.

An alternative implementation might modify the first argument rather than creating a new table. This would save the cost of copying, and make array_concat different from the .. operator on strings.

Edit: As observed in a comment by Joseph Kingry, I failed to properly extract the actual value of each argument from .... I also failed to return the merged table from the function at all. That's what I get for coding in the answer box and not testing the code at all.

Acronym

+1 on the notion for "natural result of a function ... return a list of results". This is quite probable. – Prominence

I think there is an error in this function, I think you need another select in there after the for to get the actual value out of .... lua-users.org/wiki/VarargTheSecondClassCitizen See Issue 8 – Comfit

Yup. Apparently I didn't test this code before posting, or that defect would have been obvious. More obvious in hindsight is the missing return t before the last end. – Acronym

Answer 7

If you want to merge two tables, but need a deep copy of the result table, for whatever reason, use the merge from another SO question on merging tables plus some deep copy code from lua-users.

(edit Well, maybe you can edit your question to provide a minimal example... If you mean that a table

 { a = 1, b = 2 }

concatenated with another table

{ a = 5, b = 10 }

should result in

{ a = 1, b = 2, a = 5, b = 10 }

then you're out of luck. Keys are unique.

It seems you want to have a list of pairs, like { { a, 1 }, { b, 2 }, { a, 5 }, { b, 10 } }. You could also use a final structure like { a = { 1, 5 }, b = { 2, 10 } }, depending on your application.

But the simple of notion of "concatenating" tables does not make sense with Lua tables. )

Prominence

gimf, you were right. I was misinterpreting the use of lists in Tables to think that they could simply be concatenated. Further testing led me to the conclusion that what I really needed to be doing was a merge. Thank you for your help and patience with a Lua newbie. – Exorable

@John, we were all newbies once... coming from complex languages, it is sometimes surprising how much power is hiding inside Lua's simplicity. It can take a while to grok it. – Acronym

Answer 8

Here is an implementation I've done similar to RBerteig's above, but using the hidden parameter arg which is available when a function receives a variable number of arguments. Personally, I think this is more readable vs the select syntax.

function array_concat(...)
    local t = {}

    for i = 1, arg.n do
        local array = arg[i]
        if (type(array) == "table") then
            for j = 1, #array do
                t[#t+1] = array[j]
            end
        else
            t[#t+1] = array
        end
    end

    return t
end

Hemidemisemiquaver

Answer 9

Here is my implementation to concatenate a set of pure-integer-indexing tables, FYI.

1. define a function to concatenate two tables, concat_2tables

2. another recursive function concatenateTables: split the table list by unpack, and call concat_2tables to concatenate table1 and restTableList

   t1 = {1, 2, 3}
   t2 = {4, 5}
   t3 = {6}
   
   concat_2tables = function(table1, table2)
       len = table.getn(table1)
       for key, val in pairs(table2)do
           table1[key+len] = val
       end
       return table1
   end
   
   concatenateTables = function( tableList )
       if tableList==nil then
           return  nil
       elseif table.getn(tableList) == 1 then
           return  tableList[1]
       else
           table1 = tableList[1]
           restTableList = {unpack(tableList, 2)}
           return concat_2tables(table1, concatenateTables(restTableList))
       end
   end
   
   tt = {t1, t2, t3}
   t = concatenateTables(tt)  
   

Damien

Answer 10

EDIT

---

Here's a better solution, the other one tended to overwrite numeric keys, the usage is still the same:

function merge(...)
  local temp = {}
  local index = 1
  local result = {}
  
  math.randomseed(os.time())

  for i, tbl in ipairs({ ... }) do
    for k, v in pairs(tbl) do
      if type(k) == 'number' then
        -- randomize numeric keys
        k = math.random() * i * k
      end
      
      temp[k] = v
    end
  end
  
  for k, v in pairs(temp) do
    if type(k) == "number" then
      -- Sort numeric keys into order
      if result[index] then
        index = index + 1
      end
      
      k = index
    end
    
    result[k] = v
  end

  return result
end

ORIGINAL

---

A wee bit late to the game, but this seems to work for me:

function concat(...)
  local result = {}
  
  for i, tbl in ipairs({...}) do
    for k, v in pairs(tbl) do
      if type(k) ~= "number" then
        result[k] = v
      else
        result[i] = v
      end
    end
  end
  
  return result
end

It might be a bit overcomplicated, but it takes an infinite amount of arguments, and works for both key-value pairs and regular "arrays" (numbers as keys). Here's an example

Longevous

This is the only solution that worked for tables with key/indexes t1={a={a=1}, b=2, 5} t2={c={c=3}, d=4, 6} – Jetta

However note that it doesn't maintain the order of the elements in the table when merging them together. – Jetta

Answer 11

The other solutions here suffered from 3 issues:

1. Did not work with tables that contain key/values
2. Didn't maintain the order of the elements in the tables while merging
3. Did not work with tables with mixed numeric indices and key based indices

This solution is a variant of the original solution proposed by @kaptcha which addresses the shortcomings noted above:

--- Function to merge/join tables
--- @param ... table List of tables to be merged
--- @return table Merged table
function MergeTables(...)
    local result = {}

    for i, tbl in ipairs({...}) do
      for k, v in pairs(tbl) do
        if type(k) ~= "number" then
          result[k] = v
        else
          table.insert(result, v)
        end
      end
    end

    return result
end

Usage:

local t1 = { a={1}, b={b=2} }
local t2 = { c={3}, d={d=4} }
local tMerged = MergeTables(t1, t2)

Jetta

Answer 12

-- Lua 5.1+
function TableAppend(t1, t2)
    -- A numeric for loop is faster than pairs, but it only gets the sequential part of t2
    for i = 1, #t2 do
        t1[#t1 + 1] = t2[i] -- this is slightly faster than table.insert
    end

    -- This loop gets the non-sequential part (e.g. ['a'] = 1), if it exists
    local k, v = next(t2, #t2 ~= 0 and #t2 or nil)
    while k do
        t1[k] = v -- if index k already exists in t1 then it will be overwritten
        k, v = next(t2, k)
    end
end

Mellie

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. – Axil

Answer 13

I like the simplicity in @Weeve Ferrelaine answer, but mutations may cause many issues and in general, are not desirable.

Version with NO MUTATION.

---@param t1 {}
---@param t2 {}
function TableConcat(t1,t2)
    local tOut = {}

    for i = 1, #t1 do
        tOut[i] = t1[i]
    end

    for i = #t1, #t1 + #t2 do
        tOut[i] = t2[i]
    end

    return tOut
end

Original implementation, that's mutating t1.

function TableConcat(t1,t2)
    for i=1,#t2 do
        t1[#t1+1] = t2[i]
    end
    return t1
end

Suction

Answer 14

use table.insert() function

table1 = {
  "param1=value1",
  "param2=value2",
  "param3=value3"
}
table2 = {
  "param21=value1",
  "param23=value2",
  "param23=value3"
}
table.insert(table1, table.concat(table2, ","))
print(table.unpack(table1));

Harper

Answer 15

My approach on concatenating tables

local table_fuse = function (...)
    local result = {}
    local i = 1

    for _, obj in next, {...} do
        if type(obj) ~= "table" then
            result[i] = obj
            i = i + 1
        else
            for k, v in next, obj do
                if type(k) ~= "number" then
                    result[k] = v
                else
                    result[i] = v
                    i = i + 1
                end
            end
        end
    end

    return result
end

You can pass tables and non-table objects and it will happily merge them into one table. Example:

local ex_1 = {1, 66, "hello", {"333", 41}}
local ex_2 = {"test_1", "another test"}
ex_2[":33"] = "i have no idea what to put here"
ex_2["UwU OwO"] = 1337
local ex_3 = 12345

local tbl = table_fuse(
    {"foo", "bar"}, ex_1, ex_2, "something",
    {"another something", "aaaa"}, ex_3,
    "haii :3", {{123, "123"}}
)

print(inspect(tbl)) -- github.com/kikito/inspect.lua

-- { "foo", "bar", 1, 66, "hello", { "333", 41 }, "test_1", "another test", "something", "another something", "aaaa", 12345, "haii :3", { 123, "123" },
--   [":33"] = "i have no idea what to put here",
--   ["UwU OwO"] = 1337
-- }

Lamia