dynamic linking with LLVM

Question

I want to execute functions in a module, this module will have dependencies resolved in other modules. the modules might change (dynamic compilation environment) so i would prefer not not link all the dependencies in a single monolithic module, that is, if it can be avoided

I hope to use Linker::linkModules but this is always destructive on the source module. That is ok for one module depending on a single one, since if that one changed, is no big deal, but isn't it overkill to rebuild and relink N-1 modules that did not change just because of a single one that changed?

I wonder if there is a non-destructive version of linkModules that can work for JIT execution.

Letourneau

Answer 1

Try this:

Linker::LinkModules(destinationModule, sourceModule, Linker::PreserveSource, &error);

If you pass Linker::PreserveSource instead of Linker::DestroySource, you can continue to use sourceModule after the call.

Laruelarum

Answer 2

We have done something similar in the dynamic compilation environment inside our Fabric Engine product (http://fabricengine.com/). LLVM is not currently very well adapted to this kind of complex "JIT" environment, but we managed to make it work by linking through an extra level of indirection (ie. a double pointer) and then subclassing llvm::MemoryManager to overload llvm::MemoryManager::getPointerToNamedFunction to resolve symbols globally between modules. By using a double pointer, you can change one module without changing any others. You have to be slightly careful but it isn't too bad.

Malo

Answer 3

I dont think this is possible the way you are describing he problem.

In your ideal solution, if modules A and B were linked, changing B would be immediately observable in A?

If this is the case I do not believe this to be possible. (try looking at the contents of A after linking B. The symbols of B have been copied into A)

If you simply want to perserve the information in B, you may copy B first with llvm::CloneModule, passing the result to Linker::linkModules.

Hospitality

as in normal shared libraries, if Bchanges, A would still need to be relinked to the new B. My point is: if A is linked against B0, B1... BN and one of them changes, i should only have to relink the references to that one, since the rest did not change. The current linkModules is working as a static linker (copying everything into the target module) – Letourneau

The Linux kernel, with its modules, allows something like this (unload a module, reload a new version). But there the process is under the kernel's control, and there are interlocks in place to ensure the code isn't being used. – Paramagnetism