How to define C functions with LuaJIT?

Question

This:

local ffi = require "ffi"

ffi.cdef[[
  int return_one_two_four(){
    return 124;
  }
]]

local function print124()
  print(ffi.C.return_one_two_four())
end

print124()

Throws an error:

Error: main.lua:10: cannot resolve symbol 'return_one_two_four': The specified procedure could not be found.

I have a sort-of moderate grasp on C and wanted to use some of it's good sides for a few things, but I couldn't find many examples on LuaJIT's FFI library. It seems like cdef is only used for function declarations and not definitions. How can I make functions in C and then use them in Lua?

Freethinker

You have to write them in C and compile them as shared libraries (which you then load via ffi.load), you can't write them directly in Lua – Age

There is one workaround - to write functions in machine code instead of in C, but it works only if you're using Windows (x64 or x86) and your LuaJIT is x86 and is built with MinGW. – Concision

Answer 1

LuaJIT is a Lua compiler, but not a C compiler. You have to compile your C code into a shared library first. For example with

gcc -shared -fPIC -o libtest.so test.c
luajit test.lua

with the files test.c and test.lua as below.

test.c

int return_one_two_four(){
    return 124;
}

test.lua

local ffi = require"ffi"

local ltest = ffi.load"./libtest.so"

ffi.cdef[[
int return_one_two_four();
]]

local function print124()
    print(ltest.return_one_two_four())
end

print124()

Live example on Wandbox

A JIT within LuaJIT

In the comments under the question, someone mentioned a workaround to write functions in machine code and have them executed within LuaJIT on Windows. Actually, the same is possible in Linux by essentially implementing a JIT within LuaJIT. While on Windows you can just insert opcodes into a string, cast it to a function pointer and call it, the same is not possible on Linux due to page restrictions. On Linux, memory is either writeable or executable, but never both at the same time, so we have to allocate a page in read-write mode, insert the assembly and then change the mode to read-execute. To this end, simply use the Linux kernel functions to get the page size and mapped memory. However, if you make even the tiniest mistake, like a typo in one of the opcodes, the program will segfault. I'm using 64-bit assembly because I'm using a 64-bit operating system.

Important: Before executing this on your machine, check the magic numbers in <bits/mman-linux.h>. They are not the same on every system.

local ffi = require"ffi"

ffi.cdef[[
typedef unsigned char uint8_t;
typedef long int off_t;

// from <sys/mman.h>
void *mmap(void *addr, size_t length, int prot, int flags,
           int fd, off_t offset);
int munmap(void *addr, size_t length);
int mprotect(void *addr, size_t len, int prot);

// from <unistd.h>
int getpagesize(void);
]]

-- magic numbers from <bits/mman-linux.h>
local PROT_READ     = 0x1  -- Page can be read.
local PROT_WRITE    = 0x2  -- Page can be written.
local PROT_EXEC     = 0x4  -- Page can be executed.
local MAP_PRIVATE   = 0x02 -- Changes are private.
local MAP_ANONYMOUS = 0x20 -- Don't use a file.

local page_size = ffi.C.getpagesize()
local prot = bit.bor(PROT_READ, PROT_WRITE)
local flags = bit.bor(MAP_ANONYMOUS, MAP_PRIVATE)
local code = ffi.new("uint8_t *", ffi.C.mmap(ffi.NULL, page_size, prot, flags, -1, 0))

local count = 0
local asmins = function(...)
    for _,v in ipairs{ ... } do
        assert(count < page_size)
        code[count] = v
        count = count + 1
    end
end

asmins(0xb8, 0x7c, 0x00, 0x00, 0x00) -- mov rax, 124
asmins(0xc3) -- ret

ffi.C.mprotect(code, page_size, bit.bor(PROT_READ, PROT_EXEC))

local fun = ffi.cast("int(*)(void)", code)
print(fun())

ffi.C.munmap(code, page_size)

Live example on Wandbox

How to find opcodes

I see that this answer has attracted some interest, so I want to add something which I was having a hard time with at first, namely how to find opcodes for the instructions you want to perform. There are some resources online most notably the Intel® 64 and IA-32 Architectures Software Developer Manuals but nobody wants to go through thousands of PDF pages just to find out how to do mov rax, 124. Therefore some people have made tables which list instructions and corresponding opcodes, e.g. http://ref.x86asm.net/, but looking up opcodes in a table is cumbersome as well because even mov can have many different opcodes depending on what the target and source operands are. So what I do instead is I write a short assembly file, for example

mov rax, 124
ret

You might wonder, why there are no functions and no things like segment .text in my assembly file. Well, since I don't want to ever link it, I can just leave all of that out and save some typing. Then just assemble it using

$ nasm -felf64 -l test.lst test.s

The -felf64 option tells the assembler that I'm using 64-bit syntax, the -l test.lst option that I want to have a listing of the generated code in the file test.lst. The listing looks similar to this:

$ cat test.lst
     1 00000000 B87C000000              mov rax, 124
     2 00000005 C3                      ret

The third column contains the opcodes I am interested in. Just split these into units of 1 byte and insert them into you program, i.e. B87C000000 becomes 0xb8, 0x7c, 0x00, 0x00, 0x00 (hexadecimal numbers are luckily case-insensitive in Lua and I like lowercase better).

Cowage

Answer 2

LuaJIT includes a recognizer for C declarations, but it isn't a full-fledged C compiler. The purpose of its FFI system is to be able to define what C functions a particular DLL exports so that it can load that DLL (via ffi.load) and allow you to call those functions from Lua.

LuaJIT can load pre-compiled code through a DLL C-based interface, but it cannot compile C itself.

Mandler

Answer 3

Technically you can do the sorts of things you want to do without too much trouble (as long as the code is simple enough). Using something like this: https://github.com/nucular/tcclua With tcc (which is very small, and you can even deploy with it easily) its quite a nice way to have the best of both worlds, all in a single package :)

Submarginal