Why is sizeof( std::variant< char > ) == 8 when using libc++ and not 2 (like with MSVC's STL and libstdc++)?

Question

Consider this example on Compiler explorer.

Basically, we have this code snippet:

#include <cstdint>
#include <variant>

enum class Enum1 : std::uint8_t { A, B };

enum class Enum2 : std::uint8_t { C, D };

using Var = std::variant< Enum1, Enum2 >;
using Var2 = std::variant< char >;

template< std::size_t s >
struct print_size;

void func() {
    print_size< sizeof( Var ) >{};
    print_size< sizeof( Var2 ) >{};
}

If we compile this with GCC's libstdc++ (using either clang or GCC), we get the expected compile error:

error: implicit instantiation of undefined template 'print_size<2>'

Also, similar with MSVC (as expected):

error C2027: use of undefined type 'print_size<2>'

However, when using clang with libc++, I get this error:

error: implicit instantiation of undefined template 'print_size<8>'

This indicates that sizeof( std::variant< char > ) == 8 when using libc++. I've confirmed that on Linux (see compiler explorer link above), but also with Android's NDK r18 and Xcode 10 (both iOS and MacOS).

Is there a reason for libc++'s implementation of std::variant to use so much memory or is this simply a bug in libc++ and should be reported to libc++ developers?

Triumvir

This is a difference of quality. The quality of libc++ is probably comparable to the quality of clang. Also check the ugly implementation of visitor – Zirkle

When I wrote the formatter for std::variant for lldb and I documented the implementation of std::variant and it has two members __index and __data and I don't remember why but __index is usually unsigned int` which explains the size see godbolt – Rancorous

I believe the question is pretty clear, not sure why the downvotes or votes to close. – Rancorous

@Shafik because they asked a question that is implementation specific and can't be answered with a reference to the standard, and many of the more active denziens of the C++ tag(s) hate that. I often just leave the C++ tag off for that reason. The close reason "of unclear" is just a bogus reason these people use because none of the close reasons really fit. The question is clear. – Kamikamikaze

Answer 1

The reason seems to be that in libc++'s original std::variant implementation an unsigned int is always used to store the active type index of the std::variant, while libstdc++ chooses the smallest unsigned integer type able to store the largest index.

In current libc++ this optimization is also available, but it doesn't seem to be enabled by default. The macro enabling the optimization (_LIBCPP_ABI_VARIANT_INDEX_TYPE_OPTIMIZATION) is only set if _LIBCPP_ABI_VERSION >= 2 or _LIBCPP_ABI_UNSTABLE is defined.

I suppose that, because the original implementation didn't make this optimization and it breaks compatibility of std::variant in both directions due to data layout change, it wasn't enabled by default to keep binary compatibility with older versions. The newer ABI can be enabled by setting the mentioned ABI version macro, but of course all libraries will need to be compiled with this new ABI version as well.

See https://reviews.llvm.org/D40210

Wavy

Thank you! This really solves my problem, as I don't need ABI compatibility in my specific case, but size is very important. – Triumvir

Unfortunately, this only works on ArchLinux (libc++ shipped with clang 7), but not on Xcode 10 and NDK r18. Looks like for iOS, Android and MacOS I'll have to stick with the huge sizes of std::variant. – Triumvir

@Triumvir Because of the libc++ version or because it breaks binary compatibility with some pre-compiled library (like libc++ itself)? I think a recent clang should be using a recent enough libc++ and the binary compatibility issue may be solved by compiling all C++ libraries with the macro set (possibly statically linked to your executable), but I have no experience with these platforms. – Wavy

@ user10605163, defining _LIBCPP_ABI_VERSION to 2 and compiling my code on Android NDK r18 or Xcode 10 (macos/ios) I get tons of compile errors stating that std::__2::move does not exist and similar for lots of other standard functions. My solution will be to use mapbox variant on platforms that use libc++ (Android, iOS, MacOS) and std::variant elsewhere. – Triumvir