Creating C++ API Library

Question

I'm trying to understand the correct way, or right approach, to provide a reasonably large C++ API for a non-open source project. I do not want to provide a "header only" library as the code base is fairly large and meant to be closed source. The goals are as follows:

• Provide a native C++ API that users may instantiate C++ classes, pass data around, all in C++, without a C-only wraper
• Allow methods to take as parameters and return C++ objects, especially STL types (std::string, std::vector, etc)
• No custom allocators
• Most industry standard / canonical method of doing this possible, if such a standard exists
• No re-creating COM or using MS COM
• Assume all C++ compilers are at least C++11 "compliant"

I am targeting Windows as well as other platforms (Linux).

My understanding is creating a DLL or shared library is out of the question because of the DLL boundary issue. To deal with DLL boundary issue, the DLL and the calling code must be compiled with dynamically linked runtime (and the right version, multithreaded/debug/etc on Windows), and then all compiler settings would need to match with respect to debug symbols (iterator debugging settings, etc). One question I have this is whether, if, say on Windows, we ensure that the compiler settings match in terms of /MD using either default "Debug" and "Releas" settings in Visual Studio, can we really be "safe" in using the DLL in this way (that is passing STL objects back and forth and various things that would certainly be dangerous/fail if there was a mismatch)? Do shared object, *.so in Linux under gcc have the same problem?

Does using static libraries solve this problem? How much do compiler settings need to match between a static library and calling code for which it is linked? Is it nearly the same problem as the DLL (on Windows)?

I have tried to find examples of libraries online but cannot find much guidance on this. Many resources discuss Open Source solution, which seems to be copying header and implementation files into the code base (for non-header-only), which does not work for closed source.

What's the right way to do this? It seems like it should be a common issue; although I wonder if most commercial vendors just use C interfaces.

I am ok with static libraries if that solves the problem. I could also buy into the idea of having a set of X compilers with Y variations of settings (where X and Y are pre-determined list of options to support) and having a build system that generated X * Y shared binary libraries, if that was "safe".

Is the answer is really only to do either C interfaces or create Pure Abstract interfaces with factories? (if so, is there a canonical book or guide for doing this write, that is not implementing Microsoft COM?).

I am aware of Stefanus DuToit's Hourglass Pattern: https://www.youtube.com/watch?v=PVYdHDm0q6Y

I worry that it is a lot of code duplication.

I'm not complaining about the state of things, I just want to understand the "right" way and hopefully this will serve as a good question for others in similar position.

I have reviewed these Stackoverflow references:

When to use dynamic vs. static libraries

How do I safely pass objects, especially STL objects, to and from a DLL?

Distributing Windows C++ library: how to decide whether to create static or dynamic library?

Static library API question (std::string vs. char*)

Easy way to guarantee binary compatibility for C++ library, C linkage?

Also have reviewed:

https://www.acodersjourney.com/cplusplus-static-vs-dynamic-libraries/

https://blogs.msmvps.com/gdicanio/2016/07/11/the-perils-of-c-interface-dlls/

Juieta

DLL is a way to avoid the static linking problems. – Proficient

Answer 1

Given your requirements, you'll need a static library (e.g. .lib under windows) that is linked into your program during the build phase.

The interface to that library will need to be in a header file that declares types and functions.

You might choose to distribute as a set of libraries and header files, if they can be cleanly broken into separate pieces - if you manage the dependencies between the pieces. That's optional.

You won't be able to define your own templated functions or classes, since (with most compilers) that requires distributing source for those functions or classes. Or, if you do, you won't be able to use them as part of the interface to the library (e.g. you might use templated functions/classes internally within the library, but not expose them in the library header file to users of the library).

The main down side is that you will need to build and distribute a version of the library for each compiler and host system (in combination that you support). The C++ standard specifically encourages various types of incompatibilities (e.g. different name mangling) between compilers so, generally speaking, code built with one compiler will not interoperate with code built with another C++ compiler. Practically, your library will not work with a compiler other than the one it is built with, unless those compilers are specifically implemented (e.g. by agreement by both vendors) to be compatible. If you support a compiler that supports a different ABI between versions (e.g. g++) then you'll need to distribute a version of your library built with each ABI version (e.g. the most recent version of the compiler that supports each ABI)

An upside - which follows from having a build of your library for every compiler and host you support - is that there will be no problem using the standard library, including templated types or functions in the standard library. Passing arguments will work. Standard library types can be member of your classes. This is simply because the library and the program using it will be built with the same compiler.

You will need to be rigorous in including standard headers correctly (e.g. not relying on one standard header including another, unless the standard says it actually does - some library vendors are less than rigorous about this, which can cause your code to break when built with different compilers and their standard libraries).

There will mostly be no need for "Debug" and "Release" versions (or versions with other optimisation settings) of your library. Generally speaking, there is no problem with having parts of a program being linked that are compiled with different optimisation settings. (It is possible to cause such things to break,if you - or a programmer using your library in their program - use exotic combinations of build options, so keep those to a minimum). Distributing a "Debug" version of your library will permit stepping through your library with a debugger, which seems counter to your wishes.

None of the above prevents you using custom allocators, but doesn't require it either.

You will not need to recreate COM unless you really want to. In fact, you should aim to ensure your code is as standard as possible - minimise use of compiler-specific features, don't make particular assumptions about sizes of types or layout of types, etc. Using vendor specific features like COM is a no-no, unless those features are supported uniformly by all target compilers and systems you support. Which COM is not.

I don't know if there is a "standard/canonical" way of doing this. Writing code that works for multiple compilers and host systems is a non-trivial task, because there are variations between how different compiler vendors interpret or support the standard. Keeping your code simple is best - the more exotic or recent the language or standard library feature you use, the more likely you are to encounter bugs in some compilers.

Also, take the time to set up a test suite for your library, and maintain it regularly, to be as complete as possible. Test your library with that suite on every combination of compiler/system you support.

Spa

Answer 2

• Provide a native C++ API that users may instantiate C++ classes, pass data around, all in C++, without a C-only wraper

This excludes COM.

• Allow methods to take as parameters and return C++ objects, especially STL types (std::string, std::vector, etc)

This excludes DLLs

• Most industry standard / canonical method of doing this possible, if such a standard exists

Not something "standard", but common practises are there. For example, in DLL, pass only raw C stuff.

• No re-creating COM or using MS COM

This requires DLL/COM servers

• Assume all C++ compilers are at least C++11 "compliant"

Perhaps. Normally yes.

Generally: If source is to be available, use header only (if templates) or h +cpp. If no source, the best is DLL. Static libraries - you have to build for many compilers and one has to carry on your lib everywhere and link to it.

Proficient

Answer 3

On linux, gcc uses libstdc++ for the c++ standard library while clang can use libc++ or libstdc++. If your users are building with clang and libc++, they won't be able to link if you only build with gcc and libstdc++ (libc++ and libstdc++ are not binary compatible). So if you wan't to get target both you need two version of you library, one for libstdc++, and another for libc++.

Also, binaries on linux (executable, static library, dynamic library) are not binary compatible between distros. It might work on your distro and not work on someone else distro or even a different version of your distro. Be super careful to test that it work on whichever distro you want to target. Holy Build Box could help you to produce cross-distribution binaries. I heard good thing about it, just never tried it. Worst case you might need to build on all the linux distro you want to support.

https://phusion.github.io/holy-build-box/

Marduk