hot swap code in lua

Question

I've heard mumblings around the internets about being able to hot-swap code in Lua similar to how it's done in Java, Erlang, Lisp, etc. However, 30 minutes of googling for it has turned up nothing. Has anyone read anything substantial about this? Anyone have any experience doing it? Does it work in LuaJIT or only in the reference VM?

I'm more interested in the technique as a shortcut in development/debugging than an upgrade path in a live environment.

Gilda

What do you mean by "hot-swap code"? – Xanthein

en.wikipedia.org/wiki/Hot_swapping#Software – Gilda

I've built this prototype github.com/meric/l2l/tree/master/samples/sample09 debugger.lua that allows you to hotswap code and maintain state. It's a prototype and there will probably be bugs or cases when it doesn't work, but it demonstrates the concept. It takes care of some of the issues illustrated by Nicol's answer. – Bankable

Answer 1

Lua, and most scripting languages for that matter, do not support the most generalized form of "hot swapping" as you define it. That is, you cannot guaranteeably change a file on disk and have any changes in it propagate itself into an executing program.

However, Lua, and most scripting languages for that matter, are perfectly capable of controlled forms of hot swapping. Global functions are global functions. Modules simply load global functions (if you use them that way). So if a module loads global functions, you can reload the module again if it is changed, and those global function references will change to the newly loaded functions.

However, Lua, and most scripting languages for that matter, makes no guarantees about this. All that's happening is the changing of global state data. If someone copied an old function into a local variable, they can still access it. If your module uses local state data, the new version of the module cannot access the old module's state. If a module creates some kind of object that has member functions, unless those members are fetched from globals, these objects will always refer to the old functions, not the new ones. And so forth.

Also, Lua is not thread safe; you can't just interrupt a lua_State at some point and try to load a module again. So you would have to set up some specific point in time for it to check stuff out and reload changed files.

So you can do it, but it isn't "supported" in the sense that it can just happen. You have to work for it, and you have to be careful about how you write things and what you put in local vs. global functions.

Xanthein

Answer 2

As Nicol said, the language itself doesn't do it for you.

If you want to implement something like this yourself though, it's not that hard, the only thing "preventing" you is any "leftover" references (which will still point to the old code) and the fact require caches its return value in package.loaded.

The way I'd do it is by dividing your code into 3 modules:

• the reloading logic at entry point (main.lua)
• any data you want to preserve across reloads (data.lua)
• the actual code to reload (payload.lua), making sure you don't keep any references to that (which is sometimes not possible when you e.g. have to give callbacks to some library; see below).

-- main.lua:
local PL = require("payload")
local D = require("data")

function reload(module)
  package.loaded[module]=nil -- this makes `require` forget about its cache
  return require(module)
end

PL.setX(5)
PL.setY(10)

PL.printX()
PL.printY()

-- .... somehow detect you want to reload:
print "reloading"
PL = reload("payload") -- make sure you don't keep references to PL elsewhere, e.g. as a function upvalue!

PL.printX()
PL.printY()

-- data.lua:
return {} -- this is a pretty dumb module, it's literally just a table stored in `package.loaded.data` to make sure everyone gets the same instance when requiring it.

-- payload.lua:
local D = require("data")
local y = 0
return {
  setX = function(nx) D.x = nx end, -- using the data module is preserved
  setY = function(ny) y = ny end, -- using a local is reset upon reload
  printX = function() print("x:",D.x) end,
  printY = function() print("y:", y) end
}

output:

x: 5
y: 10
reloading
x: 5
y: 0

you could flesh out that logic a bit better by having a "registry module" that keeps track of all the requiring/reloading for you and abstracts away any access into modules (thus allowing you to replace the references), and, using the __index metatable on that registry you could make it pretty much transparent without having to call ugly getters all over the place. this also means you can supply "one liner" callbacks that then actually just tail-call through the registry, if any 3rd party library needs that.

Aetna