LLVM JIT Symbols not found

Question

I am trying to write a program to JIT some code. The JITTed code needs to make calls back into the running application for run-time support and the run-time support symbols are not found when the function is materialized.

I tried to follow the Kaleidoscope tutorial. I need to call a function in the run-time from some IR generated code. For example, I want to call this function from some llvm IR.

extern "C" void* llvmNewVector() {
    return new vector<int>();
}

According to the Kaleidoscope tutorial it should be declared extern "C" and in the run-time of the application. Within the LLVM IR I have created a function prototype and the IR is correctly generated (no errors after checking the function I am jitting).

It would seem to me that there would be something more to do to link this function to the jitted code, but the Kaleidoscope tutorial doesn't seem to do that.

My problem is that the jitted code fails to materialize because the external symbols are not resolved.

The following code prints "made it here" but gets no further.

cerr << "made it here." << endl;
auto Sym = ExitOnErr(TheJIT->lookup(name));
NativeCodePtr FP = (NativeCodePtr)Sym.getAddress();
assert(FP && "Failed to find function ");
cerr << "returning jitted function " << name << endl;
return FP;

I am sure I am doing something wrong or missing some step, but I have not been able to find it.

The output I get is:

made it here.
JIT session error: Symbols not found: { llvmNewVector }
Failed to materialize symbols: { my_test }

The code was compiled using LLVM-9 with the following flags:

clang++ -I. -g -I../include/ -std=c++11 -fexceptions -fvisibility=hidden -fno-rtti -D_DEBUG -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.cpp 

For linking the following was used:

llvm-config --libs

North

Do you have your C functions in a separate file and command do you issue when you compile your stuff? – Nutmeg

I have had the C functions in both a separate file that I link in and I have moved them into an existing source file (more like the Kaleidoscope example) and neither worked. Same issue both times with not finding the symbol. – North

Could you attach the command that you use to build your source and also which LLVM version that you use – Nutmeg

I am trying to use the latest LLVM (LLVM 9). The compile command is clang++ -I. -g -I../include/ -std=c++11 -fexceptions -fvisibility=hidden -fno-rtti -D_DEBUG -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.cpp and the linking command links with all the libraries you get from running llvm-config --libs with ncurses, pthread, and z libraries added. – North

I updated your posts with this information. Currently, I have some stuff to do, I will star and get back to you when I get some time : ). In the meantime, if you find out the reason for your issues, please feel free to answer it here so that it helps someone else! – Nutmeg

Answer 1

I ran into this same issue and could solve it the following way:

The following lines of code in the tutorial, whose goal is to resolve symbols in the host process, does not seem to work.

ES.getMainJITDylib().setGenerator(
    cantFail(DynamicLibrarySearchGenerator::GetForCurrentProcess(DL)));

So instead, I manually registered the symbols that I wanted linked like this:

SymbolMap M;
// Register every symbol that can be accessed from the JIT'ed code.
M[Mangle("llvmNewVector")] = JITEvaluatedSymbol(
    pointerToJITTargetAddress(&llvmNewVector), JITSymbolFlags());
}
cantFail(ES.getMainJITDylib().define(absoluteSymbols(M)));

I added this code right after the two lines of code that I mentioned above, from the tutorial.

Furgeson

Answer 2

How about adding -Xlinker --export-dynamic option for clang?

I met the similar problem in the tutorial. In my environment (Ubuntu 20.04), sin and cos can be resolved, but printd or putchard (the functions defined in the source code of Kaleidoscope processor) cannot.

After compilation, can you see the function name in the dynamic symbol table of the program?

objdump -T program | grep llvmNewVector

If there are no -T option in objdump (e.g., Mac), it might not be the case. In my case, printd nor putchard do not appear in the dynamic symbol table (but appear in symbol table).

To add these function names into the dynamic symbol table, you need to pass -Xlinker --export-dynamic option for clang (actually, the option is passed to ld), for example (this is one for the tutorial),

clang++ -Xlinker --export-dynamic -g toy.cpp `llvm-config --ldflags --system-libs --libs all` -O3 -o toy

After compilation, the function names appear in dynamic symbol table, and the examples of the tutorial work well.

Subsistent

Answer 3

It depends on which llvm version you use. LLVM 10 has LLJIT class and it was working for me the following way

      auto J = ExitOnErr(LLJITBuilder().create());
      auto M = createDemoModule();
    
      auto &dl = J->getDataLayout();
      MangleAndInterner Mangle(J->getExecutionSession(), dl);
      auto &jd = J->getMainJITDylib();
    
      auto s = absoluteSymbols({{ Mangle("printd"), JITEvaluatedSymbol(pointerToJITTargetAddress(&printd), JITSymbolFlags::Exported)}});
      jd.define(s);

the printd function was defined in the same file

extern "C" int32_t printd() {
    std::cout << "calling " << __FUNCTION__ << "...\n";
    return 11;
}

Celluloid

Answer 4

For anyone using LLVM-16, here is a solution:

int main() {
    ...

    TheJIT = ExitOnError(llvm::orc::KaleidoscopeJIT::Create());

    auto &jd = TheJIT->getMainJITDylib();
    auto mangle = llvm::orc::MangleAndInterner(jd.getExecutionSession(), TheJIT->getDataLayout());

    auto s = [](llvm::orc::MangleAndInterner interner) {
        llvm::orc::SymbolMap symbolMap;
        symbolMap[interner("putchard")] = {
            llvm::pointerToJITTargetAddress(&putchard),
            llvm::JITSymbolFlags(),
        };
        symbolMap[interner("printd")] = {
            llvm::pointerToJITTargetAddress(&printd),
            llvm::JITSymbolFlags(),
        };
        return llvm::orc::absoluteSymbols(symbolMap);
    }(mangle);

    ExitOnError(jd.define(s));

    
    ...
    MainLoop();
}

Hylo

For anyone using /LLVM-18/, was able to adapt MrZ's LLVM-16 workaround: – Googly

Answer 5

Adapting MrZ's answer for LLVM-16, was able to get this to work with LLVM-18.1.5:

struct Jit {
    ...
    MangleAndInterner & mangler_;
    JitDylib & dest_dynamic_lib_;

    ...
    /** intern @p symbol, binding it to address @p dest **/
    template <typename T>
    llvm::Error intern_symbol(const std::string & symbol, T * dest) {
        llvm::orc::SymbolMap symbol_map;
        symbol_map[mangler_(symbol)]
            = llvm::orc::ExecutorSymbolDef(llvm::orc::ExecutorAddr::fromPtr(dest),
                             llvm::JITSymbolFlags());

       auto materializer = llvm::orc::absoluteSymbols(symbol_map);

       return dest_dynamic_lib_.define(materializer);
    } /*intern_symbol*/
};

P.S. I did not have problems with the kaleidoscope demos themselves, but ran into inexplicable-to-me resolution problems when both Jit and symbol-to-be-resolved come from a shared-library-loaded-into-running-process instead of kaleidoscope executable.

P.P.S. See https://github.com/Rconybea/xo-pyjit for complete example.

Googly