how to extend std::formatter without sometimes introducing conflicts (can concepts when re-evaluated later in program return diff answer)

Question

I am familiar with how to specialize std::formatter; it's relatively easy and clear with explicit specialization.

But with partial specialization - its a challenge - due to shifting nature of various libraries (including the c++ standard library) and what partial specializations are already included (and sometimes even implementation details of how they are implemented - just what they match).

I have a library, with a bunch of provided types, and which attempts to work with a variety of std c++ versions (20, 23, 26) and have those types work with formatter<>.

Background (From Stroika 'ToString.h')

// Does type 'T' support the Stroika ToString() API
template <typename T>
concept IToString = requires (T t) { { ToString (t) } -> convertible_to<Characters::String>; };
...
template <IToString T>
struct ToStringFormatter /* : std::formatter<std::wstring>*/{...}

Then I want to say something like - if the standard formatter is not already defined, then use ToStringFormatter (never mind about the namespaces for simplicity):

template <typename T>
requires (!std::formattable<T,wchar_t> && IToString<T>)
struct std::formatter<T, wchar_t> : ToStringFormatter<T> {};

So if the type isn't already formattable, format it with ToStringFormatter<>. This appears like it should be the most simple, and natural approach, but doesn't work on visual studio ( error C7608: atomic constraint should be a constant expression) presumably because the definition actually changes the value returned by the constraint.

OR

template <IToString T>
struct std::formatter<T, wchar_t> : ToStringFormatter<T> {};

The problem with the second approach is that it introduces ambiguity, if used with a type T which is BOTH IToString and formattable.

So - I've settled on an ad-hoc approach:

template <typename T>
    concept IUseToStringFormatterForFormatter_ =
        // most user-defined types captured by this rule - just add a ToString() method!
        requires (T t) {
            {
                t.ToString ()
            } -> convertible_to<Characters::String>;
        }

        // Stroika types not captured by std-c++ rules
        or Common::IKeyValuePair<remove_cvref_t<T>> or Common::ICountedValue<remove_cvref_t<T>>

    // c++ 23 features which may not be present with current compilers
    // value with clang++16 was 202101L and cpp2b and libc++ (ubuntu 23.10 and 24.04) flag... and it had at least the pair<> code supported.
    // this stuff needed for clang++-18-debug-libstdc++-c++23
#if !__cpp_lib_format_ranges
#if !qHasFeature_fmtlib or (FMT_VERSION < 110000)
        or (ranges::range<decay_t<T>> and
            not Configuration::IAnyOf<decay_t<T>, string, wstring, string_view, wstring_view, const char[], const wchar_t[],
                                      qStroika_Foundation_Characters_FMT_PREFIX_::string_view, qStroika_Foundation_Characters_FMT_PREFIX_::wstring_view>)
#endif
#endif

// sadly MSFT doesn't support all, and doesn't support __cplusplus with right value
// 202302L is right value to check for C++ 23, but 202101L needed for clang++16 ;-(
#if _MSC_VER || __cplusplus < 202101L /*202302L 202100L 202300L*/ || (__clang__ != 0 && __GLIBCXX__ != 0 && __GLIBCXX__ <= 20240412) ||    \
    (!defined(__clang__) && __cplusplus == 202302L && __GLIBCXX__ <= 20240412) and (!defined(_LIBCPP_STD_VER) || _LIBCPP_STD_VER < 23)
#if !qHasFeature_fmtlib or (FMT_VERSION < 110000)
        // available in C++23
        or Configuration::IPair<remove_cvref_t<T>> or
        Configuration::ITuple<remove_cvref_t<T>>
#endif
#endif

// need to check _LIBCPP_STD_VER for LIBC++ and clang++16 on ubuntu 23.10
#if (!defined(__cpp_lib_formatters) || __cpp_lib_formatters < 202302L) and (!defined(_LIBCPP_STD_VER) || _LIBCPP_STD_VER < 23)
        // available in C++23
        or Configuration::IAnyOf<remove_cvref_t<T>, thread::id>
#endif

    // features added in C++26
    // unsure what to check - __cpp_lib_format - test c++26  __cpp_lib_formatters < 202601L  -- 202302L  is c++23
#if __cplusplus < 202400L
        or Configuration::IAnyOf<remove_cvref_t<T>, std::filesystem::path>
#endif

        // Features from std-c++ that probably should have been added
        // NOTE - we WANT to support type_info (so typeid works directly) - but run into trouble because type_info(const type_info)=delete, so not
        // sure how to make that work with formatters (besides wrapping in type_index).
        or is_enum_v<remove_cvref_t<T>> or Configuration::IOptional<remove_cvref_t<T>> or Configuration::IVariant<remove_cvref_t<T>>

#if qCompilerAndStdLib_ITimepointConfusesFormatWithFloats_Buggy
        or same_as<T, std::chrono::time_point<chrono::steady_clock, chrono::duration<double>>>
#else
            or Configuration::ITimePoint<T>
#endif
        or Configuration::IAnyOf<remove_cvref_t<T>, exception_ptr, type_index> or derived_from<T, exception>;

};
template <Private_::IUseToStringFormatterForFormatter_ T>
struct std::formatter<T, wchar_t> : ToStringFormatter<T> {};

This sort of approach appears to perhaps work - but with lots of exceptions and problems (principally that different libraries - like fmtlib - and clang libc++ etc) all seem to implement the 'ranges' support differently, so the fact that ranges are integrated automatically into std::formater<> - is very complicated to check for each given type and combination of libraries, which checks to throw into IUseToStringFormatterForFormatter_.

Even the complexity above is not nearly enough (and yet seems too much to be very convenient).

Really, what I think would be ideal, is (what I tried first) - something along the lines of:

template <typename T>
requires (!std::formattable_SO_FAR<T,wchar_t>)
struct std::formatter<T, wchar_t> : ToStringFormatter<T> {};

where formattable_SO_FAR is allowed to change meaning as the compilation progresses.

Is there some way with the current C++ language to conditionally extend std::formatter using checks whose value depends on 'formattable' - so that the return value of formattable would change in different parts of the program?

Esmeralda

Yes, but, such a possible concept would only behave consistently across compilers when instantiated from a non-dependent context. In a dependent context, when and how the compiler chooses to return the result from the previous instantiation can differ observably. – Wilfordwilfred

@PatrickRoberts i assume your 'Yes' refers to the last question (about if there is a way with current c++). Even then - I'm not fully understanding your comment. Could you kindly offer an explicit example (even if incomplete - a basis I could play with) of an alternative approach? – Esmeralda

Hey, I posted and answer and decided to delete it. After testing the approach with an actual specialization of std::formatter, the immediate problem you mention is solved, but another problem replaces it: Even if the concept itself is atomic, it will ODR-use the instantiation of std::formatter<T> when testing, probably using the primary template definition which doesn't even satisfy std::semiregular<std::formatter<T>>, but that ODR-usage prevents the specialization from being accessible because it's already defined as the primary template at that point. – Wilfordwilfred

FYI, the approach used a typename TUnique = decltype([]() -> void {}) default parameter and named typename TUnique; within the requires expression, so that each instantiation is a unique atomic constraint which can yield a different result in different parts of the program. But like I said, unfortunately that's not enough on its own to achieve what you want here :\ If I think of a way around that problem too, then I'll edit and undelete my answer, but I don't currently have a solution for you. – Wilfordwilfred

I had hoped it would be as successful as this: godbolt.org/z/3G94anob7 – Wilfordwilfred

Thanks @PatrickRoberts! I now understand what you meant in your first comment. Very clever. And seems a good start. I will also ponder how to use that trick with template partial specialization. – Esmeralda

@Esmeralda I am not sure this is a good approach. It has the potential to give different results in each translation unit depending on which files were included and in which order - leading to a bunch of ODR problems. So, I definitely do not recommend any approach of this kind. What you can do is define a dedicated class my_formatter and use a function like my_format that first try to format variables via my_formatter and then delegate to std::formatter. This way, your library and formatting rules are encapsulated from other libraries, so there won't be interferences. – Fasces

@Fasces I need to think about this some more. What you propose maybe a good solution for my problem. THANK YOU - either way... More in the next day when I've had a chance to ponder/test. – Esmeralda

@Fasces I've thought more about this, and I'm not sure your approach works well. The problem with the my_formatter approach is that I don't think it will work with std c++ apis which themselves use formatting (such as print, or other formatters). – Esmeralda

@Lewis the question is, what do you try to write? It is unclear. Do you write a separate component, like an app or separated library? Or you want to write like a header-only library? These have very different approaches – Fasces

@Fasces largish library (github.com/SophistSolutions/Stroika) – Esmeralda

@Esmeralda Okay, as I understand it, you have a library with its own printing and formatting rules, and you want library users to be able to use them without causing conflicts with other libraries they utilize. It's a bit verbose, but you can create my_lib_wrap<T> template class and put all your formatter specializations under it, i.e., you only create specializations like std::formatter<my_lib_wrap<T>> and when user wants to print, say a range R, using your rules they write std::format("{}", my_lib_wrap(R)). – Fasces

Answer 1

First of all, conditionally extending std::formatter, especially checking if a type is already formattable, can be really complicated. However, we can perform this check with some methods.

You want to check if a type is formattable using the concept of std::formattable, but this is not possible directly. Instead, you need to create your own concept.

Since there is no standard formattable concept, you can create your own and check it with that concept. Below is an example solution.

template <typename T>
concept IUseToStringFormatterForFormatter_ = requires (T t) {
    { t.ToString() } -> std::convertible_to<Stroika::Foundation::Characters::String>;
} || Stroika::Foundation::Common::IKeyValuePair<std::remove_cvref_t<T>> || Stroika::Foundation::Common::ICountedValue<std::remove_cvref_t<T>>;

template <Stroika::Foundation::Characters::Private_::IUseToStringFormatterForFormatter_ T>
struct std::formatter<T, wchar_t> : Stroika::Foundation::Characters::ToStringFormatter<T> {};

Underscore

This appears to be a narrow subset of what I started with, and not an adequate subset. It neglects handling things like pair<>, and tuple<>, and ranges. It really misses the point. I want to extend formatter with all appropriate formatters, but not with those that would create conflicts. This answer appears to just avoid any of the tricky parts (things that might or might not be provided by an STL implementation or other libraries). – Esmeralda