How to compile Lua scripts into a single executable, while still gaining the fast LuaJIT compiler?

Question

How can I compile my Lua scripts into a single executable file, while also gaining the super fast performance benefits of LuaJIT?

Background:

• My Lua scripts are for a web application I created (e.g. to host http://example.com)
• My current technology stack is NGINX (web server), Lua/LuaJIT (language to retrieve dynamic content)
• I have around 50+ .lua files that make up my web application (from Models/Views/Controllers)
• FreeBSD 9 operating system

For simplicity sake in deployment, I'd like to compile down all of my .lua scripts that run my web application down to a single executable.

1. Is this possible and how?

   It appears that Lua official comes with a library called SRLua

2. What are the negatives to compiling down my .lua to a single executable (e.g. would performance be worse, etc)?

Amygdaloid

It's a web app, right? Doesn't that mean that you run it... from your web browser? So it's not an executable you download, right? – Policyholder

Yes, a web application. No download. This is all server side. – Amygdaloid

It's to simplify my deployment on the server side for patching, etc of my code. It takes over 50+ .lua files to run my web application today. Which means I have to host 50+ .lua files on my server and ensure I have all the correct files. If I could instead (1) consolidate that down to a single compiled file, it makes my server maintenance and versioning MUCH easier, and (2) if there are performance benefits (?) that would also be great. – Amygdaloid

I know it's not considered a best practice but having all of your files as separate text allows you to make emergency fixes on the live site without having to go through a whole rebuild and redeploy cycle. If you're having trouble pushing a coherent set of 50 files to the server during deployment, you might want to look at your deployment procedure - building a unified executable isn't any easier. – Bacchius

Answer 1

Translate all of the Lua source code files to object files and put them in a static library:

for f in *.lua; do
    luajit -b $f `basename $f .lua`.o
done
ar rcus libmyluafiles.a *.o

Then link the libmyluafiles.a library into your main program using -Wl,--whole-archive -lmyluafiles -Wl,--no-whole-archive -Wl,-E.

This line forces the linker to include all object files from the archive and to export all symbols.

For example, a file named foo.lua can now be loaded with local foo = require("foo") from within your application.

Details about the -b option can be found on Running LuaJIT.

Clod

Wahoo! Mike Pall from LuaJit answered this question :) – Amygdaloid

many thanks for all your work on LuaJIT! It's truly incredible what you have done. – Amygdaloid

That basename command doesn't work on Ubuntu 13.04. It ends up overwriting the original lua file with the bytecoded version – Publia

I think what you want is something like: for f in *.lua; do luajit -b $f basename $f .lua.o; done – Publia

What should you do if you have two files with the same name, but they live in different directories? i.e., a/foo.lua and b/foo.lua. When I try to link the generated .o files, I get duplicate symbol errors (duplicated symbol _luaJIT_BC_foo) – Reich

I have a luajit setup in windows32 built with gcc (tdm sjlj)... Would this recipe work for me? – Samuelson

Answer 2

For a web app that you are currently deploying as a nest of related .lua files, your easiest answer will be to condense them into a single file. This can often be done for simple cases with luac. However, for complex applications with a mix of modules you want something smarter.

I personally use Mathew Wild's utility squish to do something similar.

After running squish, you will have a single .lua file containing all the Lua source code bundled up conveniently. You could just deploy that single file.

If you need to also bundle any binary modules, or the Lua or LuaJIT interpreter, then you can easily use SRLua to bundle it with the Lua interpreter, or similar techniques to bundle it with LuaJIT.

Barbary

So, this isn't really compiling down to a single executable as in my original question but instead - merging all of my .lua files into a single .lua file? – Amygdaloid

Although you can go all the way to an executable, and can also use squish to simplify the collection of dependencies to link into that executable, you can also stop with just the single .lua file. Mike Pall has provided a good recipe for bundling LuaJIT with .lua files. – Barbary