Iterate through Lua Table

Question

I am trying to iterate through a lua table but I keep getting this error:

invalid key to 'next'

I know that index starts off as -8 and I know that there is a table there because it gets the first (and only) value in it. However, it tries to loop round again even though I know there is only one string in the table.

if (lua_istable(L, index))
{
    lua_pushnil(L);

    // This is needed for it to even get the first value
    index--;

    while (lua_next(L, index) != 0)
    {
        const char *item = luaL_checkstring(L, -1);
        lua_pop(L, 1);

        printf("%s\n", item);
    }
}
else
{
    luaL_typerror(L, index, "string table");
}

Any help would be appreciated.

This works fine when I use a positive index (as long as I don't remove 1 from it)

Edit: I've noticed that I don't get this error if I leave the value of item alone. Only when I start reading the value of item do I get this error. When I've got the value from the table, I call another Lua function, could this be disrupting lua_next?

Samira

Answer 1

Do not use luaL_checkstring with negative arguments. Use lua_tostring instead.

Also, make sure the stack remains the same after you call a function in the loop: lua_next expects the previous table key at the top of the stack so that it can resume the traversal.

Marimaria

Yes, after dumping the contents of the stack before and after, I've noticed that there are remains from when I called the function. – Samira

Answer 2

There are 2 things you need to watch:

• ensure the original key is left on the stack before the next call to lua_next. luaL_checkstring will convert non-string keys to strings (as the resulting string is not in the table, it becomes an invalid key.) This is most easily done by passing luaL_checkstring a copy of the key instead of the original.
• ensure you preserve the stack structure (i.e. pop as many values as you push) on each pass through the loop

Your function will only work for negative values of index. You are correct that index--; will ensure that index still points to the table after pushing the key, but only if index was negative (i.e. relative to the top of the stack.) If index is an absolute or pseudo index then it this will cause it to point to the wrong item. The easiest workaround is to push another reference to the table onto the top of the stack.

Here's a minimal C program to demonstrate:

#include <lauxlib.h>
#include <lua.h>

static void iterate_and_print(lua_State *L, int index);

int main(int ac, char **av)
{
   lua_State *L = luaL_newstate();
   luaL_openlibs(L);

   // Create a table and put it on the top of the stack
   luaL_loadstring(L, "return {one=1,[2]='two',three=3}");
   lua_call(L, 0, 1);

   iterate_and_print(L, -1);
   return 0;
}

static void iterate_and_print(lua_State *L, int index)
{
    // Push another reference to the table on top of the stack (so we know
    // where it is, and this function can work for negative, positive and
    // pseudo indices
    lua_pushvalue(L, index);
    // stack now contains: -1 => table
    lua_pushnil(L);
    // stack now contains: -1 => nil; -2 => table
    while (lua_next(L, -2))
    {
        // stack now contains: -1 => value; -2 => key; -3 => table
        // copy the key so that lua_tostring does not modify the original
        lua_pushvalue(L, -2);
        // stack now contains: -1 => key; -2 => value; -3 => key; -4 => table
        const char *key = lua_tostring(L, -1);
        const char *value = lua_tostring(L, -2);
        printf("%s => %s\n", key, value);
        // pop value + copy of key, leaving original key
        lua_pop(L, 2);
        // stack now contains: -1 => key; -2 => table
    }
    // stack now contains: -1 => table (when lua_next returns 0 it pops the key
    // but does not push anything.)
    // Pop table
    lua_pop(L, 1);
    // Stack is now the same as it was on entry to this function
}

Minor

luaL_checkstring does accept negative indices but it aborts and gives the wrong message if it fails because it is meant to check function arguments, not for general conversion. – Marimaria

@lhf, OK there seems to be no advantage in using luaL_checkstring other than that it was in the code sample in the question. Replaced it with lua_tostring. – Minor

Answer 3

From the manual:

const char *lua_tolstring (lua_State *L, int index, size_t *len);

Converts the Lua value at the given acceptable index to a C string. If len is not NULL, it also sets *len with the string length. The Lua value must be a string or a number; otherwise, the function returns NULL. If the value is a number, then lua_tolstring also changes the actual value in the stack to a string. (This change confuses lua_next when lua_tolstring is applied to keys during a table traversal.)

luaL_checkstring calls lua_tolstring.

Designedly

Sorry, as the quote states, this only applies to the key, and you're running this on the value. – Designedly

Answer 4

See also the example from the docs for lua_next, excerpted here:

int lua_next (lua_State *L, int index);

Pops a key from the stack, and pushes a key–value pair from the table at the given index (the "next" pair after the given key). If there are no more elements in the table, then lua_next returns 0 (and pushes nothing).

A typical traversal looks like this:

/* table is in the stack at index 't' */
 lua_pushnil(L);  /* first key */
 while (lua_next(L, t) != 0) {
   /* uses 'key' (at index -2) and 'value' (at index -1) */
   printf("%s - %s\n",
          lua_typename(L, lua_type(L, -2)),
          lua_typename(L, lua_type(L, -1)));
   /* removes 'value'; keeps 'key' for next iteration */
   lua_pop(L, 1);
 }

While traversing a table, do not call lua_tolstring directly on a key, unless you know that the key is actually a string. Recall that lua_tolstring may change the value at the given index; this confuses the next call to lua_next.

See function next for the caveats of modifying the table during its traversal.

Thumb