How do you copy a Lua table by value?

Question

Recently I wrote a bit of Lua code something like:

local a = {}
for i = 1, n do
   local copy = a
   -- alter the values in the copy
end

Obviously, that wasn't what I wanted to do since variables hold references to an anonymous table not the values of the table themselves in Lua. This is clearly laid out in Programming in Lua, but I'd forgotten about it.

So the question is what should I write instead of copy = a to get a copy of the values in a?

Tace

Answer 1

To play a little readable-code-golf, here's a short version that handles the standard tricky cases:

• tables as keys,
• preserving metatables, and
• recursive tables.

We can do this in 7 lines:

function copy(obj, seen)
  if type(obj) ~= 'table' then return obj end
  if seen and seen[obj] then return seen[obj] end
  local s = seen or {}
  local res = setmetatable({}, getmetatable(obj))
  s[obj] = res
  for k, v in pairs(obj) do res[copy(k, s)] = copy(v, s) end
  return res
end

There is a short write-up of Lua deep-copy operations in this gist.

Another useful reference is this Lua-users wiki page, which includes an example on how to avoid the __pairs metamethod.

Clisthenes

Why are there two parameters if he's copying one object? – Alcmene

@Alcmene The second parameter is meant to be ignored by the "outside" caller and only used for recursive calls. It avoids repeated deep copying of tables that occur more than once in a single table. E.g. a = {<any values>} b = {k1=a, k2=a}. If b2 is a copy of b, it's nice to keep b2.k1 == b2.k2. The extra seen parameter helps to make this possible. – Clisthenes

This can go horribly wrong with metatables, especially since you're setting the metatable first and then assigning the fields… (__newindex could do all sorts of funny things). Further, __pairs or __index might be futzing with the iteration, so you'd better go for k, v in next, obj, nil do to copy the actual contents (that's sort of the equivalent of rawget for pairs), before setting the metatable at the end (which will then make things look the same). Also: why not seen = seen or {} as the first line (that's more readable) and then if seen and seen[x] --> if seen[x]? – Scoundrelly

@nobody: Short answer: this is short reasonable code that will work for many people finding this SO page. If they want to deep copy a table with surprising-behavior assignment / iteration metamethods, I'd say they've voluntarily entered territory where they ought to know what they're doing. Your suggestion about putting the seen line first is good. I think I wrote it as it is because I wanted to keep the lines short (hence use s as a variable name) but also keep clear what that variable meant; giving it two names was my solution (so line count would be the same). – Clisthenes

@Clisthenes haha as someone who've voluntarily entered that territory I'm still googling "how to copy a table lua." On a more serious note though, this kind of thing would be an issue for people using libraries that might use __newindex and __index for accessor-like syntax. – Absolutely

Answer 2

Table copy has many potential definitions. It depends on whether you want simple or deep copy, whether you want to copy, share or ignore metatables, etc. There is no single implementation that could satisfy everybody.

One approach is to simply create a new table and duplicate all key/value pairs:

function table.shallow_copy(t)
  local t2 = {}
  for k,v in pairs(t) do
    t2[k] = v
  end
  return t2
end

copy = table.shallow_copy(a)

Note that you should use pairs instead of ipairs, since ipairs only iterate over a subset of the table keys (ie. consecutive positive integer keys starting at one in increasing order).

Corollaceous

This code is incorrect. I used it and it fails on one specific issue: what if you have tables inside tables. You need to check the type of v and use table.copy recursively when it's a table. I will post the code in a separate message below. – Siderolite

The code is not incorrect, it's a perfectly valid way to copy a table if you don't need recursive copy. If you need recursive copy, then make the function recursive. The very reason it's not included in standard libraries is that there is no single definition of "table copy". – Corollaceous

I can see @Siderolite point. Perhaps what's needed is a name that better indicates the type of copying that's being done? I'll update the name to shallow_copy; let me know if there's any disagreements. – Oliva

No, that is exactly my point. Good choice @greatwolf. I removed my downvote. – Siderolite

Do the key values have to be numeric? – Lennielenno

No, pairs will iterate over all keys with non-nil values. – Corollaceous

Answer 3

Just to illustrate the point, my personal table.copy also pays attention to metatables:

function table.copy(t)
  local u = { }
  for k, v in pairs(t) do u[k] = v end
  return setmetatable(u, getmetatable(t))
end

There is no copy function sufficiently widely agreed upon to be called "standard".

Prong

Note that it is considered a bad practice to change or extend standard namespaces like table. – Arithmomancy

This works until someone sets the __metatable property of the metatable. – Objective

Also – as of Lua 5.2 – you should use the next function directly, as the behaviour of pairs may be affected by the metamethod __pairs. Also, you could solve the __metatable problem by using debug.getmetatable... but I'm not sure if that's OK to do. – Phototelegraph

Answer 4

The full version of deep copy, handling all the 3 situations:

1. Table circular reference
2. Keys which are also tables
3. Metatable

The general version:

local function deepcopy(o, seen)
  seen = seen or {}
  if o == nil then return nil end
  if seen[o] then return seen[o] end

  local no
  if type(o) == 'table' then
    no = {}
    seen[o] = no

    for k, v in next, o, nil do
      no[deepcopy(k, seen)] = deepcopy(v, seen)
    end
    setmetatable(no, deepcopy(getmetatable(o), seen))
  else -- number, string, boolean, etc
    no = o
  end
  return no
end

Or the table version:

function table.deepcopy(o, seen)
  seen = seen or {}
  if o == nil then return nil end
  if seen[o] then return seen[o] end


  local no = {}
  seen[o] = no
  setmetatable(no, deepcopy(getmetatable(o), seen))

  for k, v in next, o, nil do
    k = (type(k) == 'table') and k:deepcopy(seen) or k
    v = (type(v) == 'table') and v:deepcopy(seen) or v
    no[k] = v
  end
  return no
end

Based on the lua-users.org/wiki/CopyTable's and Alan Yates' functions.

Succulent

Welcome to Stack Overflow and thank you for the suggestions! It looks like the seen variable is a sort of caching system. I'll have to try it out. – Tace

This works well in my project and hope your feedback if there're pitfalls. – Succulent

Hah, wish I'd scrolled down further before writing this myself! (IMO this should be the accepted answer.) – Staffer

Answer 5

An optionally deep, graph-general, recursive version:

function table.copy(t, deep, seen)
    seen = seen or {}
    if t == nil then return nil end
    if seen[t] then return seen[t] end

    local nt = {}
    for k, v in pairs(t) do
        if deep and type(v) == 'table' then
            nt[k] = table.copy(v, deep, seen)
        else
            nt[k] = v
        end
    end
    setmetatable(nt, table.copy(getmetatable(t), deep, seen))
    seen[t] = nt
    return nt
end

Perhaps metatable copy should be optional also?

Galton

You need to add nt to seen before iterating or there are circumstances where you'll overflow the stack. Basically, remove like 15 and insert it after line 6. – Hamitic

I was only thinking about "but what if t is its own metatable" but of course its also a problem if it's simply local t={}; t[1] = t or something. Also, after looking at lua-users.org/wiki/CopyTable, I see you forgot to copy the keys, which can be tables themselves. – Brockwell

Answer 6

Here's what I actually did:

for j,x in ipairs(a) do copy[j] = x end

As Doub mentions, if your table keys are not strictly monotonically increasing, it should be pairs not ipairs.

I also found a deepcopy function that is more robust:

function deepcopy(orig)
    local orig_type = type(orig)
    local copy
    if orig_type == 'table' then
        copy = {}
        for orig_key, orig_value in next, orig, nil do
            copy[deepcopy(orig_key)] = deepcopy(orig_value)
        end
        setmetatable(copy, deepcopy(getmetatable(orig)))
    else -- number, string, boolean, etc
        copy = orig
    end
    return copy
end

It handles tables and metatables by calling itself recursively (which is its own reward). One of the clever bits is that you can pass it any value (whether a table or not) and it will be copied correctly. However, the cost is that it could potentially overflow the stack. So and even more robust (non-recursive) function might be needed.

But that's overkill for the very simple case of wanting to copy an array into another variable.

Tace

Note that this version of deepcopy does not work for self-referential (recursive) tables. (Which is what you get if you follow accepted practice for object oriented Lua.) – Staffer

For the stack overflow problem, transforming the function to use tail calls would solve this. However, may not be possible. – Carmelitacarmelite

Answer 7

The (unfortunately lightly documented) stdlib project has a number of valuable extensions to several of the libraries shipped with the standard Lua distribution. Among them are several variations on the theme of table copying and merging.

This library is also included in the Lua for Windows distribution, and should probably be a part of any serious Lua user's toolbox.

One thing to make sure of when implementing things like this by hand is the proper handling of metatables. For simple table-as-structure applications you probably don't have any metatables, and a simple loop using pairs() is an acceptable answer. But if the table is used as a tree, or contains circular references, or has metatables, then things get more complex.

Put

Answer 8

Don't forget that functions are also references, so if you wanted to completely 'copy' all of the values you'd need to get separate functions, too; however, the only way I know to copy a function is to use loadstring(string.dump(func)), which according to the Lua reference manual, doesn't work for functions with upvalues.

do
    local function table_copy (tbl)
        local new_tbl = {}
        for key,value in pairs(tbl) do
            local value_type = type(value)
            local new_value
            if value_type == "function" then
                new_value = loadstring(string.dump(value))
                -- Problems may occur if the function has upvalues.
            elseif value_type == "table" then
                new_value = table_copy(value)
            else
                new_value = value
            end
            new_tbl[key] = new_value
        end
        return new_tbl
    end
    table.copy = table_copy
end

Yarkand

Answer 9

I think the reason why Lua doesn't have 'table.copy()' in its standard libraries is because the task is not precise to define. As shown already here, one can either make a copy "one level deep" (which you did), a deepcopy with or without caring of possible duplicate references. And then there's metatables.

Personally, I would still like them to offer a built-in function. Only if people wouldn't be pleased with its semantics, they would need to go do it themselves. Not very often, though, one actually has the copy-by-value need.

Cartoon

Answer 10

Warning: the marked solution is INCORRECT!

When the table contains tables, references to those tables will still be used instead. I have been searching two hours for a mistake that I was making, while it was because of using the above code.

So you need to check if the value is a table or not. If it is, you should call table.copy recursively!

This is the correct table.copy function:

function table.copy(t)
  local t2 = {};
  for k,v in pairs(t) do
    if type(v) == "table" then
        t2[k] = table.copy(v);
    else
        t2[k] = v;
    end
  end
  return t2;
end

Note: This might also be incomplete when the table contains functions or other special types, but that is possible something most of us don't need. The above code is easily adaptable for those who need it.

Siderolite

I think it's a little harsh to say that it's "incorrect". It depends on what you have inside the table and what you want to copy. You'll notice that several other answers also account for recursive copying of tables. Others account for metatables and functions. – Tace

IMO. the OP doesn't say anything about the contents of the table. Hence, the answer should be as complete as possible, which is not the case with the selected answer. Others like me will search for this question and will take the selected answer, searching for an error in their own code while it's not in theirs. – Siderolite

Well, for what it's worth, I am the OP and I had not considered the situation of recursively copying a table containing tables. Several of the other answers covered that concern quite well. – Tace

Answer 11

That's as good as you'll get for basic tables. Use something like deepcopy if you need to copy tables with metatables.

Lansing

Answer 12

In most of the cases when I needed to copy a table, I wanted to have a copy that doesn't share anything with the original, such that any modification of the original table has no impact on the copy (and vice versa).

All the snippets that have been shown so far fail at creating a copy for a table that may have shared keys or keys with tables as those are going to be left pointing to the original table. It's easy to see if you try to copy a table created as: a = {}; a[a] = a. deepcopy function referenced by Jon takes care of that, so if you need to create a real/full copy, deepcopy should be used.

Valence

How deep should deepcopy go? Should deepcopy({m = _G.mymodule}) copy whole mymodule table recursively or should it copy only reference to mymodule once it realized that mymodule is accessible from global env? – Orebro

Answer 13

Use penlight library here: https://stevedonovan.github.io/Penlight/api/libraries/pl.tablex.html#deepcopy

local pl = require 'pl.import_into'()
local newTable = pl.tablex.deepcopy(oldTable)

Albertina

Answer 14

Just use the

local unpack = unpack or table.unpack

list2 = {unpack (list)}

Justiciable

note that this will fail if the list has more than 8000 items – Flooded

Answer 15

This might be the simplest method:

local data = {DIN1 = "Input(z)", DIN2 = "Input(y)", AINA1 = "Input(x)"}

function table.copy(mytable)  --mytable = the table you need to copy

    newtable = {}

    for k,v in pairs(mytable) do
        newtable[k] = v
    end
    return newtable
end

new_table = table.copy(data)  --copys the table "data"

Nefarious

Answer 16

In my situation, when the information in the table is only data and other tables (excluding functions, ...), is the following line of code the winning solution:

local copyOfTable = json.decode( json.encode( sourceTable ) )

I'm writing Lua code for some home automation on a Fibaro Home Center 2. The implementation of Lua is very limited with no central library of functions you can refer to. Every function needs to be declared in the code so to keep the code serviceable, so one line solutions like this are favorable.

Crispate