Is it safe to link C++17, C++14, and C++11 objects
Asked Answered
H

3

159

Suppose I have three compiled objects, all produced by the same compiler/version:

  1. A was compiled with the C++11 standard
  2. B was compiled with the C++14 standard
  3. C was compiled with the C++17 standard

For simplicity, let's assume all headers were written in C++11, using only constructs whose semantics haven't changed between all three standard versions, and so any interdependencies were correctly expressed with header inclusion and the compiler did not object.

Which combinations of these objects is it and isn't it safe to link into a single binary? Why?


EDIT: answers covering major compilers (e.g. gcc, clang, vs++) are welcome

Hackery answered 14/10, 2017 at 16:36 Comment(24)
What do you think? Why do you think that? And why do you wonder? What is the actual problem you have? What makes you ask this? Is it just a dump of a school assignment or an interview question?Aileenailene
Not a school/interview question. The question stems from a particular case: I am working on a project which depends on an open-source library. I build this library from source, but its build system only accepts a flag to choose between C++03/C++11 building. The compiler I use supports other standards though, and I am considering upgrading my own project to C++17. I am unsure whether it is a safe decision. Can there be a break in ABI or some other way in which the approach is not advisable? I did not find a clear answer and decided to post a question about the general case.Hackery
Somewhat related questions: here, here.Steviestevy
The answer to this question is probably best looked up in your compiler's manual. I don't think there are any standard guarantees for this.Florindaflorine
This depends entirely on the compiler. There's nothing in the formal C++ specifications that governs this situation. There is also a small possibility that code that was written to C++03 or C+11 standards will have some issues at the C++14 and C++17 level. With sufficient knowledge and experience (and well-written code to start with), it should be possible to fix any of these issues. If you are not, however, very familiar with the newer C++ standards, you are better off sticking to what the build system supports, and is tested to work with.Glyceride
It is a straight shot in the leg. I guess the proper example of breaking changes would be a copy elision treatment in different standards. If you want to utilize libraries built with drastically different compiler settings then the only way is to communicate through a plain C interface.Unashamed
So if one wants to distribute a library in header/binary form, do different versions need to be provided for different standards?Hackery
Software is generally designed to be backward-compatible, so I'd expect a C++14 compiler to be able to link object files from earlier versions, but not vice versa.Bakst
ABI changes may or may not be linked to changes in source file standards-compliance, and in any case it will depend on which standard library features have been used in each of the source files.Angiosperm
@Someprogrammerdude: It's an extremely worthwhile question. I wish I had an answer. All I know is that libstdc++ via RHEL devtoolset is backward compatible by design, by statically linking in the newer stuff and leaving the older stuff to resolve dynamically at runtime using the distro's "native" libstdc++. But that doesn't answer the question.Selfcontradiction
In general, standard library containers are subject to ABI-breaking layout changes even just changing compiler switches; VC++ breaks liberally between debug and release mode, and I'm quite sure it broke between different VC++ versions; libstdc++ changes ABI when enabling the "debug STL" and when switching from C++03 to C++11 mode (std::string in particular). For libstdc++ the various set of containers have distinct names at link time (so if your modules don't have containers on interface boundaries you may be fine), but what std::string refers to can differ.Wedge
It is safe to use all of them together. If it wasn't the case, it would be rather impossible to ship a single C++ binary library.Hurry
@n.m.: ... which is mostly the case... pretty much everyone who distributes distribution-independent C++ libraries does so (1) in dynamic library form and (2) without C++ standard library containers on interface boundaries. Libraries that come from a Linux distribution have it easy as they are all built with the same compiler, same standard library and pretty much same default set of flags.Wedge
@MatteoItalia please pick any major C++ library that offers binary downloads and point me to the separate C++11-compatible, C++14-compatible and C++17-compatible downloads (note these three only, not debug/release or libc++/libstdc++ or gcc4/gcc6 or any other variation). Boost OpenCV Qt SFML or whatever. Or a disclaimer/warning that a library is not compatible with binaries built to one of these standards. I have searched and found none.Hurry
@MatteoItalia "Libraries that come from a Linux distribution have it easy as they are all built with the same compiler, same standard library and pretty much same default set of flags." I run Gentoo and I know for a fact that different libraries are buit with different -std flags. Additionally, users build their own programs with whatever -std flags they want, and everything is magically compatible.Hurry
@n.m.: AFAIK Qt, up until to a few years ago, made a point not to have standard library containers on binary interface boundaries, period, for this exact reason; all the Qt containers <-> standard library helpers (e.g. QString::toStdString()) were implemented inline in headers. I heard that this changed in recent versions, but I don't know how they handle it.Wedge
So ok, it turns out that, as long as you are compiling everything with the same g++ version, the ABI changes I was thinking about should be handled fine, even with different -std= flags. You are still in for a world of pain if you have a library compiled with an older g++ version.Wedge
The only case of incompatibility I can think of is when a library checks for a specific language standard and deliberately uses different compilation paths with different standards (with the preprocessor or with template magic).Hurry
@MatteoItalia the ABI change in gcc/libstdc++ induced by c++03 to c++11 transition was real and painful, but hopefully lessons were learned.Hurry
@MatteoItalia I think non-crossing of interface boundaries has more to do with how Microsoft handle different compiler versions. Language standard switches were not supported at all by MS compilers until recently.Hurry
all headers were written in C++11 - I think you should specify what that means. Were they written so that they compile in C++11 mode, or were they written using only constructs whose semantics haven't changed between all three standard versions? Having just the former and not the latter sounds like a recipe for trouble, unless special precautions were taken in the code (which implies that its version is more recent than the relevant standard changes).Spile
@bogdan, you are right, very good point. The question was meant to concern link time, so it needs to assert that included headers have the same interpretation in all standards. I will edit the question to reflect that. Please ignore my earlier comment, I was not seeing this clearly (now deleted).Hackery
Just to clarify the earlier comment from @MatteoItalia "and when switching from C++03 to C++11 mode (std::string in particular)." This is not true, the active std::string implementation in libstdc++ is independent of the -std mode used. This is an important property, precisely to support situations like the OP's. You can use the new std::string in C++03 code, and you can use the old std::string in C++11 code (see the link in Matteo's later comment).Ables
Related post: cullmann.io/posts/cpp-standard-version-mix-upHackery
A
208

Which combinations of these objects is it and isn't it safe to link into a single binary? Why?

For GCC it is safe to link together any combination of objects A, B, and C. If they are all built with the same version then they are ABI compatible, the standard version (i.e. the -std option) doesn't make any difference.

Why? Because that's an important property of our implementation which we work hard to ensure.

Where you have problems is if you link together objects compiled with different versions of GCC and you have used unstable features from a new C++ standard before GCC's support for that standard is complete. For example, if you compile an object using GCC 4.9 and -std=c++11 and another object with GCC 5 and -std=c++11 you will have problems. The C++11 support was experimental in GCC 4.x, and so there were incompatible changes between the GCC 4.9 and 5 versions of C++11 features. Similarly, if you compile one object with GCC 7 and -std=c++17 and another object with GCC 8 and -std=c++17 you will have problems, because C++17 support in GCC 7 and 8 is still experimental and evolving.

On the other hand, any combination of the following objects will work (although see note below about libstdc++.so version):

  • object D compiled with GCC 4.9 and -std=c++03
  • object E compiled with GCC 5 and -std=c++11
  • object F compiled with GCC 7 and -std=c++17

This is because C++03 support is stable in all three compiler versions used, and so the C++03 components are compatible between all the objects. C++11 support is stable since GCC 5, but object D doesn't use any C++11 features, and objects E and F both use versions where C++11 support is stable. C++17 support is not stable in any of the used compiler versions, but only object F uses C++17 features and so there is no compatibility issue with the other two objects (the only features they share come from C++03 or C++11, and the versions used make those parts OK). If you later wanted to compile a fourth object, G, using GCC 8 and -std=c++17 then you would need to recompile F with the same version (or not link to F) because the C++17 symbols in F and G are incompatible.

The only caveat for the compatibility described above between D, E and F is that your program must use the libstdc++.so shared library from GCC 7 (or later). Because object F was compiled with GCC 7, you need to use the shared library from that release, because compiling any part of the program with GCC 7 might introduce dependencies on symbols that are not present in the libstdc++.so from GCC 4.9 or GCC 5. Similarly, if you linked to object G, built with GCC 8, you would need to use the libstdc++.so from GCC 8 to ensure all symbols needed by G are found. The simple rule is to ensure the shared library the program uses at run-time is at least as new as the version used to compile any of the objects.

Another caveat when using GCC, already mentioned in the comments on your question, is that since GCC 5 there are two implementations of std::string available in libstdc++. The two implementations are not link-compatible (they have different mangled names, so can't be linked together) but can co-exist in the same binary (they have different mangled names, so don't conflict if one object uses std::string and the other uses std::__cxx11::string). If your objects use std::string then usually they should all be compiled with the same string implementation. Compile with -D_GLIBCXX_USE_CXX11_ABI=0 to select the original gcc4-compatible implementation, or -D_GLIBCXX_USE_CXX11_ABI=1 to select the new cxx11 implementation (don't be fooled by the name, it can be used in C++03 too, it's called cxx11 because it conforms to the C++11 requirements). Which implementation is the default depends on how GCC was configured, but the default can always be overridden at compile-time with the macro.

Ables answered 5/3, 2018 at 21:38 Comment(17)
"because compiling any part of the program with GCC 7 might introduce dependencies on symbols that present in the libstdc++.so from GCC 4.9 or GCC 5" you meant that are NOT present from GCC 4.9 or GCC 5, right? Does this also apply to static linking? Thanks for the info on compatibility across compiler versions.Pirtle
Thanks for the clear answer. You don't happen to know what the case is for other major compilers?Hackery
@Hackery I'm 90% sure the answer is the same for Clang/libc++, but I have no idea about MSVC.Ables
If I am not mistaken there are some mangling issues with noexcept in C++17 vs C++14/11 (or at least, gcc -std=c++14 emits a warning regarding forward compatibility under certain conditions). Is it correct?Macy
@Macy even in C++17 noexcept does not affect a function's mangled name. Templates that have arguments of function type (or pointer/reference to function) can change mangled name but it's not very common in practice. And it doesn't make anything unsafe necessarily, it typically just causes some linker errors due to unresolved symbols.Ables
This answer is stellar. Is it documented somewhere that 5.0+ is stable for 11/14?Curst
Not very clearly or in one place. gcc.gnu.org/gcc-5/changes.html#libstdcxx and gcc.gnu.org/onlinedocs/libstdc++/manual/api.html#api.rel_51 declare the library support for C++11 to be complete (the language support was feature-complete earlier, but still "experimental"). C++14 library support is still listed as experimental until 6.1, but I think in practice nothing changed between 5.x and 6.x that affects ABI.Ables
@JonathanWakely I have a followup question about language ABI if you've got the time and interest :-)Curst
Wow, very detailed and educational. Thank you. Would you say creating a shared library is the least riskier approach?Sheerness
@afp_2008 that doesn't change anything.Ables
Is it safe to mix ISO and GNU dialects? For example build one static library with -std=gnu99 and then build executable target with -std=c11 and link this executable against mentioned library?Parr
@Parr yesAbles
Amazing answer. In your example, if object F is compiled with Clang16 and -std=c++17, all others keep the same, how to guarantee compatibility? Force to use the corresponding libstdc++.so provided by Clang16? I'm not sure if Clang16 has that one...Looney
@kaixinliu clang never provides any libstdc++.so so I'm not sure what you mean. When you compile with clang and use libstdc++ headers, what matters for the purposes of this answer is the version of GCC that the libstdc++ headers came from. The version of clang is (mostly) irrelevant. Clang16 with libstdc++ headers from GCC 7 behaves like GCC 7. Clang16 with libstdc++ headers from GCC 6 behaves like GCC 6.Ables
So, according to @JonathanWakely, following setup does not introduce ABI issues? Lib A - C++ 11 (gcc 8.4) <- Lib B - C++ 14 (gcc 8.4) <- Lib C - C++ 17 (gcc 8.4) .... Description, libC use libB, libB use libA.. All libs compiled by the same compiler (gcc 8.4)Foregone
@AdnaKateg the first paragraph covers that, is it not clear enough? If there are any incompatibilities there it's just a bug that should be fixed in GCC, and almost certainly has already been fixed in current releases.Ables
@JonathanWakely Thanks for the reply, yes I missed/slipped the first paragraph while reading through the more elaborate later paragraphs, it's very much clear enough.. Thanks again.Foregone
P
22

There are two parts to the answer. Compatibility at the compiler level and compatibility at the linker level. Let's start with the former.

let's assume all headers were written in C++11

Using the same compiler means that the same standard library header and source files (the onces associated with the compiler) will be used irrespective of the target C++ standard. Therefore, the header files of the standard library are written to be compatible with all C++ versions supported by the compiler.

That said, if the compiler options used to compile a translation unit specify a particular C++ standard, then the any features that are only available in newer standards should not be accessible. This is done using the __cplusplus directive. See the vector source file for an interesting example of how it's used. Similarly, the compiler will reject any syntactic features offered by newer versions of the standard.

All of that means that your assumption can only apply to the header files you wrote. These header files can cause incompatibilities when included in different translation units targeting different C++ standards. This is discussed in Annex C of the C++ standard. There are 4 clauses, I'll only discuss the first one, and briefly mention the rest.

C.3.1 Clause 2: lexical conventions

Single quotes delimit a character literal in C++11, whereas they are digit separators in C++14 and C++17. Assume you have the following macro definition in one of the pure C++11 header files:

#define M(x, ...) __VA_ARGS__

// Maybe defined as a field in a template or a type.
int x[2] = { M(1'2,3'4) };

Consider two translation units that include the header file, but target C++11 and C++14, respectively. When targeting C++11, the comma within the quotes is not considered to be a parameter separator; there is only once parameter. Therefore, the code would be equivalent to:

int x[2] = { 0 }; // C++11

On the other hand, when targeting C++14, the single quotes are interpreted as digit separators. Therefore, the code would be equivalent to:

int x[2] = { 34, 0 }; // C++14 and C++17

The point here is that using single quotes in one of the pure C++11 header files can result in surprising bugs in the translation units that target C++14/17. Therefore, even if a header file is written in C++11, it has to be written carefully to ensure that it's compatible with later versions of the standard. The __cplusplus directive may be useful here.

The other three clauses from the standard include:

C.3.2 Clause 3: basic concepts

Change: New usual (non-placement) deallocator

Rationale: Required for sized deallocation.

Effect on original feature: Valid C++2011 code could declare a global placement allocation function and deallocation function as follows:

void operator new(std::size_t, std::size_t); 
void operator delete(void*, std::size_t) noexcept;

In this International Standard, however, the declaration of operator delete might match a predefined usual (non-placement) operator delete (3.7.4). If so, the program is ill-formed, as it was for class member allocation functions and deallocation functions (5.3.4).

C.3.3 Clause 7: declarations

Change: constexpr non-static member functions are not implicitly const member functions.

Rationale: Necessary to allow constexpr member functions to mutate the object.

Effect on original feature: Valid C++2011 code may fail to compile in this International Standard.

For example, the following code is valid in C++2011 but invalid in this International Standard because it declares the same member function twice with different return types:

struct S {
constexpr const int &f();
int &f();
};

C.3.4 Clause 27: input/output library

Change: gets is not defined.

Rationale: Use of gets is considered dangerous.

Effect on original feature: Valid C++2011 code that uses the gets function may fail to compile in this International Standard.

Potential incompatibilities between C++14 and C++17 are discussed in C.4. Since all the non-standard header files are written in C++11 (as specified in the question), these issues will not occur, so I will not mention them here.

Now I'll discuss compatibility at the linker level. In general, potential reasons for incompatibilities include the following:

  • The format of the object files.
  • Program startup and termination routines and the main entry point.
  • Whole program optimization (WPO).

If the format of the resulting object file depends on the target C++ standard, the linker must be able to link the different object files. In GCC, LLVM, and VC++, this is fortunately not the case. That is, the format of objects files is the same irrespective of the target standard, although it is highly dependent on the compiler itself. In fact, none of the linkers of GCC, LLVM, and VC++ require knowledge about the target C++ standard. This also means that we can link object files that are already compiled (statically linking the runtime).

If the program startup routine (the function that calls main) is different for different C++ standards and the different routines are not compatible with each other, then it would not be possible to link the object files. In GCC, LLVM, and VC++, this is fortunately not the case. In addition, the signature of the main function (and the restrictions that apply on it, see Section 3.6 of the standard) is the same in all C++ standards, so it doesn't matter in which translation unit it exists.

In general, WPO may not work well with object files compiled using different C++ standards. This depends on exactly which stages of the compiler require knowledge of the target standard and which stages don't and the impact that it has on inter-procedural optimizations that cross object files. Fortunately, GCC, LLVM, and VC++ are well designed and don't have this issue (not that I'm aware of).

Therefore, GCC, LLVM, and VC++ have been designed to enable binary compatibility across different versions of the C++ standard. This is not really a requirement of the standard itself though.

By the way, although the VC++ compiler offers the std switch, which enables you to target a particular version of the C++ standard, it does not support targeting C++11. The minimum version that can be specified is C++14, which is the default starting from Visual C++ 2013 Update 3. You could use an older version of VC++ to target C++11, but then you would have to use different VC++ compilers to compile different translation units that target different versions of the C++ standard, which would at the very least break WPO.

CAVEAT: My answer may not be complete or very precise.

Pirtle answered 5/3, 2018 at 20:26 Comment(6)
The question was really meant to concern linking rather than compilation. I recognize (thanks to this comment) that was perhaps not clear and have edited it to make it clear that any included headers have the same interpretation in all three standards.Hackery
@Hackery The answer covers both compilation and linking. I thought you were asking about both.Pirtle
Indeed, but I find the answer is way too long and confusing, especially until "Now I'll discuss compatibility at the linker level". You could replace everything above that with something like if the included headers cannot be postulated to have the same meaning in C++11 and C++14/17, then it is not safe to include them in the first place. For the remaining part, do you have a source showing that those three bullet points are the only potential reasons for incompatibility? Thanks for the answer in any case, I am still voting upHackery
@Hackery I cannot say for sure. That's why I added the caveat at the end of the answer. Anyone else is welcome to expand the answer to make it more precise or complete in case I missed something.Pirtle
This confuses me: "Using the same compiler means that the same standard library header and source files (...) will be used". How can that be the case? If I have old code compiled with gcc5, the 'compiler files' that belonged to that version can not be future proof. For source code compiled at (wildly) different times with different compiler versions, we can be pretty sure the library header and source files are different. With your rule that these should be the same, you have to recompile older source code with gcc5, ...and make sure they all use the latest (same) 'compiler files'.Mezzosoprano
If you have a huge company with many teams, they may not all switch at the same time to the latest version of their compiler (whether gcc, clang or cl.exe). So what is interesting to know is whether we can link object files produced by different versions of a compiler, which may have different standard library header and source files.Mezzosoprano
D
2

New C++ standards are come in two parts: language features and standard library components.

As you mean by new standard, changes in language itself (e.g. ranged-for) there's almost no problem (sometimes conflicts are exists in 3rd party library headers with newer standard language features).

But standard library...

Each compiler version comes with an implementation of C++ standard library (libstdc++ with gcc, libc++ with clang, MS C++ standard library with VC++,...) and exactly one implementaion, not many implementation for each standard version. Also in some cases you may use other implementation of standard library than compiler provided. What you should care is linking an older standard library implementation with a newer one.

The conflict that could occur between 3rd party libraries and your code is the standard library (and other libraries) that links to that 3rd party libraries.

Dereliction answered 5/3, 2018 at 10:15 Comment(4)
"Each compiler version comes with an implementation of STL" No they don'tSelfcontradiction
@LightnessRacesinOrbit Do you mean that there is no ralation between e.g. libstdc++ and gcc?Dereliction
No, I mean the STL has been effectively obsolete for just over twenty years. You meant the C++ Standard Library. As for the rest of the answer, can you provide some references/evidence to back up your claim? I think for a question like this it's important.Selfcontradiction
Sorry, no, it's not clear from the text. You have made some interesting assertions, but have not yet backed them up with any evidence.Selfcontradiction

© 2022 - 2024 — McMap. All rights reserved.