Calling C++ (not C) from Common Lisp?

Question

I am wondering if there is some way to call C++ code from Common Lisp (preferably portably, and if not, preferably in SBCL, and if not, well, then Clozure, CLisp or ECL).

The C++ would be called inside loops for numeric computation, so it would be nice if calls were fast.

CFFI seems to not support this:

"The concept can be generalized to other languages; at the time of writing, only CFFI's C support is fairly complete, but C++ support is being worked on."

(chapter 4 of the manual)

SBCL's manual doesn't mention C++ either; it actually says

This chapter describes SBCL's interface to C programs and libraries (and, since C interfaces are a sort of lingua franca of the Unix world, to other programs and libraries in general.)

The C++ code uses OO and operator overloading, so it really needs to be compiled with g++.

And as far as I know, I can have a C++ main() function and write wrappers for C functions, but not the other way around -- is that true?

Anyway... Is there some way to do this?

Thank you!

Caucus

Answer 1

Oh, wait!

It seems that there is a trick I can use!

I write a wrapper in C++, declaring wrapper functions extern "C":

#include "lib.h"

extern "C" int lib_operate (int i, double *x) {
...
}

The header file lib.h, which can be called from both C and C++, is:

#if __cplusplus
extern "C" {
#endif

int lib_operate (int i, double *x);

#if __cplusplus
}
#endif

Then compile with:

g++ -c lib.cpp
gcc -c prog.c
gcc lib.o prog.o -lstdc++ -o prog

Seems to work for a toy example! :-)

So, in Common Lisp I'd call the wrapper after loading libstdc++.

Anyway, thank you for your answers!

Caucus

Answer 2

After compiling, most C++ functions actually boil down to regular C function calls. Due to function overloading and other features, C++ compilers use name mangling to distinguish between similarly named functions. Given an object dump utility and sufficient knowledge about your C++ compiler, you can call C++ code directly from the outside world.

Having said that though, you may find it easier to write a C-compatible layer between Lisp and your C++ code. You would do that using extern "C" like this:

extern "C" Foo *new_Foo(int x)
{
    return new Foo(x);
}

This makes the new_Foo() function follow the C calling convention so that you can call it from external sources.

Vorticella

The second option would be good, but the C++ code (which isn't mine) uses operator overloading. I suppose I can't decalare an extern "C" operator +... The first option depends on the compiler being used, but is a good suggestion! – Caucus

You can still call overloaded operators from outside, you just have to get a bit creative with your API. For example: extern "C" void add_Foo(Foo *result, const Foo *foo1, const Foo *foo2) { *result = *foo1 + *foo2; } – Vorticella

Not all C++ functions "boil down to regular C". Member functions typically use a different calling convention in addition to the name mangling. (I believe in MSVC on x86, the this parameter is passed in a register, whereas all params in a C function are passed on the stack) – Perce

Right -- but I can still write a wrapper with C-like functions and do as I explained in my own answer... – Caucus

Answer 3

The main difference in calling C++ functions instead of C functions apart from the name mangling are the 'hidden' features like this pointers that are implicitly passed to member functions. The C runtime layer doesn't know anything about these, implicit type conversions and other fun C++ features, so if you intend to call C++ through a C interface, you might have to fake these features if necessary.

Assuming that you can hold at least a void * to the object you intend to call and the data it requires, you can degrade the following C++ call

matrix->multiply(avector);

to a C call if you create a C wrapper function:

extern "C"
void matrix_multiply(void *cpp_matrix, void *cpp_vector) {
  reinterpret_cast<matrix_type *>(cpp_matrix)->multiply(reinterpret_cast<vector_type *>(cpp_vector);
}

Obviously the function matrix_multiply would sit in the C++ source code and compiled as such but it does expose a C interface to the outside world. As long as you can interact with the opaque pointers, you're OK with the translation shims above.

Admittedly this is not necessarily the most elegant solution for a problem like this but I've used it in the past in situations like yours.

The other option would be to make the C++ calls directly by treating them as C calls with additional parameters and supplying all the required information yourself, but that does move you into the realm of compiler-specific code very quickly. Basically, you would still be holding the opaque pointers to C++ objects, but you'd have to work out the mangled name of the function you want to call. Once you've got that function name, you'll have to supply the this pointer (which is implicit in C++ and semi-implicit in the example above) and the correct parameters and then call the function. It can be done but as mentioned, puts you deeply in to the realm of compiler and even compiler-version specific behaviour.

Craddock

Answer 4

Update 2021:CL-CXX-JIT which handles most of the work on the lisp side.

Example:

(ql:quickload :cxx-jit)
(in-package cxx-jit)
(from '("<cmath>") 'import '("static_cast<double(*)(double)>(std::sin)" . "cpp-sin"))
(cpp-sin 0d0)
(from nil 'import "struct C{ auto hi(){return \"Hello, World\\n\";} auto bye(){return \"Bye\";} };" '("&C::bye" . "bye") '("&C::hi" . "hi") '("[](){static C x; return x;}" . "cc"))
(cc)
(hi *)
(bye **)

---

You can use cl-cxx which is like writing pybind11 for python.

example in c++ 'std >= c++14', compiled as shared lib:

#include <string>

#include "clcxx/clcxx.hpp"

class xx {
 public:
  xx(int xx, int yy) : y(yy), x(xx) {}
  std::string greet() { return "Hello, World"; }

  int y;
  int x;
};

std::string greet() { return "Hello, World"; }
int Int(int x) { return x + 100; }
float Float(float y) { return y + 100.34; }
auto gr(std::complex<float> x) { return x; }
std::string hi(char* s) { return std::string("hi, " + std::string(s)); }

void ref_class(xx& x) { x.y = 1000000; }

CLCXX_PACKAGE TEST(clcxx::Package& pack) {
  pack.defun("hi", F_PTR(&hi));
  pack.defun("test-int", F_PTR(&Int));
  pack.defun("greet", F_PTR(&greet));
  pack.defun("test-float", F_PTR(&Float));
  pack.defun("test-complex", F_PTR(&gr));
  pack.defun("ref-class", F_PTR(&ref_class));
  pack.defclass<xx, false>("xx")
      .member("y", &xx::y)
      .defmethod("greet-from-class", F_PTR(&xx::greet))
      .constructor<int, int>();
}

usage in lisp:

(cffi:use-foreign-library my-lib)
(cxx:init)
(cxx:add-package "TEST" "TEST")
(test:greet)
(setf my-class (test:create-xx2 10 20))
(test:y.get myclass)

That would take care of all conversions, extern "C", ... for you.

Division

And now I know how to instantiate a class and access its members in cl-cxx :-) Maybe you can add that part of the example to the git repo... – Hedgcock

great. sure I will. – Division

Answer 5

Depending on your C++ ABI, your wrapper (lib_operate above) might need to somehow handle any C++ exceptions that might occur. If your ABI does table-drive exception handling, unhandled exceptions will simply crash the (Lisp) process. If it instead does dynamic registration, you might not even notice that anything went wrong. Either way, it's bad.

Or, if you've got the no-throw guarantee for the wrapped code, you can ignore all this.

Randolphrandom

I agree. But what to do about it? – Antibes