Type erasing type erasure, `any` questions?

Question

So, suppose I want to type erase using type erasure.

I can create pseudo-methods for variants that enable a natural:

pseudo_method print = [](auto&& self, auto&& os){ os << self; };

std::variant<A,B,C> var = // create a variant of type A B or C

(var->*print)(std::cout); // print it out without knowing what it is

My question is, how do I extend this to a std::any?

It cannot be done "in the raw". But at the point where we assign to/construct a std::any we have the type information we need.

So, in theory, an augmented any:

template<class...OperationsToTypeErase>
struct super_any {
  std::any data;
  // or some transformation of OperationsToTypeErase?
  std::tuple<OperationsToTypeErase...> operations;
  // ?? what for ctor/assign/etc?
};

could somehow automatically rebind some code such that the above type of syntax would work.

Ideally it would be as terse in use as the variant case is.

template<class...Ops, class Op,
  // SFINAE filter that an op matches:
  std::enable_if_t< std::disjunction< std::is_same<Ops, Op>... >{}, int>* =nullptr
>
decltype(auto) operator->*( super_any<Ops...>& a, any_method<Op> ) {
  return std::get<Op>(a.operations)(a.data);
}

Now can I keep this to a type, yet reasonably use the lambda syntax to keep things simple?

Ideally I want:

any_method<void(std::ostream&)> print =
  [](auto&& self, auto&& os){ os << self; };

using printable_any = make_super_any<&print>;

printable_any bob = 7; // sets up the printing data attached to the any

int main() {
  (bob->*print)(std::cout); // prints 7
  bob = 3.14159;
  (bob->*print)(std::cout); // prints 3.14159
}

or similar syntax. Is this impossible? Infeasible? Easy?

Pasta

Tangentially related; what implementation are you testing this against? Do any of the major stdlibs have readily available versions? – Overexpose

I feel like when I've tried to do similar things in the past, I eventually realized that intrinsically everything came back to templated virtuals, and the fact that they're not allowed by the language. I'm sure that something is possible but certainly many of the nicer solutions are impossible for this reason. – Breland

@Overexpose Testing? You mean this, or the variant one? I didn't. Did it against boost::variant just now and found a typo. For the most part, barring small differences and syntax uglyness which C++17 cleans up, testing std::any solutions against boost::any will be sufficient to be confident, at least on an online compiler. – Pasta

@NirFriedman Are you talking about trying something similar, where you want to be able to (say) print anything at all and end up using type erasure to do it... or are you talking about type erasing type erasure itself and running into problems then? The templated virtual lack is for the most part solved by type erasure unless you want to multiple dispatch over two unbounded sets of argument types at distinct spots in the code. – Pasta

I'm not sure if I got all pieces together, but here's a very rough sketch: coliru.stacked-crooked.com/a/2ab8d7e41d24e616 – Gimcrackery

Slightly refined with operator overloading: coliru.stacked-crooked.com/a/23a25da83c5ba11d – Gimcrackery

@Gimcrackery yes, that is a proof of concept. Doesn't split the storage of the function pointer from the factory of function pointer (the factory should be "global", the storage of the function pointer local to the any), but that is just a design issue. – Pasta

@Gimcrackery Iterative improvement of your code folded with my (currently deleted) answer below. – Pasta

@Yakk If templated virtuals existed, then any could simply have a templated virtual function called apply which accepted a variadic functor, and in the derived any (which is aware of the type), the implementation of apply would call the functor on the derived type. In that sense, templated virtuals would trivially solve your problem. With variants, the problem is easily solveable because they use switch-case, not virtuals, as their runtime indirection; that's the key difference. – Breland

When i read the documentation example with variant and operator->*, i knew it was you without looking at the name – Muss

If you are allowed to use Boost, you can try Boost.TypeErasure (usage example) – Notochord

@Notochord I am unaware about how to get the slick ->*dofoo or similar syntax with Boost.TypeErasure, but I am unfamiliar with it. – Pasta

Would you consider updating the question to avoid the references to SO Docs? Please flag this comment 'no longer needed' when you're done. (See Removing Documentation: Reputation, Archive and Links if you're not aware of what's going on and what to do.) Thanks. – Sluiter

Answer 1

Here's my solution. It looks shorter than Yakk's, and it does not use std::aligned_storage and placement new. It additionally supports stateful and local functors (which implies that it might never be possible to write super_any<&print>, since print could be a local variable).

any_method:

template<class F, class Sig> struct any_method;

template<class F, class Ret, class... Args> struct any_method<F,Ret(Args...)> {
  F f;
  template<class T>
  static Ret invoker(any_method& self, boost::any& data, Args... args) {
    return self.f(boost::any_cast<T&>(data), std::forward<Args>(args)...);
  }
  using invoker_type = Ret (any_method&, boost::any&, Args...);
};

make_any_method:

template<class Sig, class F>
any_method<std::decay_t<F>,Sig> make_any_method(F&& f) {
  return { std::forward<F>(f) };
}

super_any:

template<class...OperationsToTypeErase>
struct super_any {
  boost::any data;
  std::tuple<typename OperationsToTypeErase::invoker_type*...> operations = {};

  template<class T, class ContainedType = std::decay_t<T>>
  super_any(T&& t)
    : data(std::forward<T>(t))
    , operations((OperationsToTypeErase::template invoker<ContainedType>)...)
  {}

  template<class T, class ContainedType = std::decay_t<T>>
  super_any& operator=(T&& t) {
    data = std::forward<T>(t);
    operations = { (OperationsToTypeErase::template invoker<ContainedType>)... };
    return *this;
  }
};

operator->*:

template<class...Ops, class F, class Sig,
  // SFINAE filter that an op matches:
  std::enable_if_t< std::disjunction< std::is_same<Ops, any_method<F,Sig>>... >{}, int> = 0
>
auto operator->*( super_any<Ops...>& a, any_method<F,Sig> f) {
  auto fptr = std::get<typename any_method<F,Sig>::invoker_type*>(a.operations);
  return [fptr,f, &a](auto&&... args) mutable {
    return fptr(f, a.data, std::forward<decltype(args)>(args)...);
  };
}

Usage:

#include <iostream>
auto print = make_any_method<void(std::ostream&)>(
  [](auto&& self, auto&& os){ os << self; }
);

using printable_any = super_any<decltype(print)>;

printable_any bob = 7; // sets up the printing data attached to the any

int main() {
  (bob->*print)(std::cout); // prints 7
  bob = 3.14159;
  (bob->*print)(std::cout); // prints 3.14159
}

Live

Straighten

Slick, using the signature of a function pointer including the type of the any_method, but it does fundamentally limit you to uniquely typed any_methods (at least on the same object). I don't see a way to migrate to auto* pointer arguments in the super_any<&print> easily. I guess a hack involving an auto* tag. I guess that is a corner case, as we need compile time polymorphism in the any_method, and auto&& self in a lambda is the easiest way to do it. Much smaller than mine! (+1) – Pasta

Answer 2

This is a solution that uses C++14 and boost::any, as I don't have a C++17 compiler.

The syntax we end up with is:

const auto print =
  make_any_method<void(std::ostream&)>([](auto&& p, std::ostream& t){ t << p << "\n"; });

super_any<decltype(print)> a = 7;

(a->*print)(std::cout);

which is almost optimal. With what I believe to be simple C++17 changes, it should look like:

constexpr any_method<void(std::ostream&)> print =
  [](auto&& p, std::ostream& t){ t << p << "\n"; };

super_any<&print> a = 7;

(a->*print)(std::cout);

In C++17 I'd improve this by taking a auto*... of pointers to any_method instead of the decltype noise.

Inheriting publicly from any is a bit risky, as if someone takes the any off the top and modifies it, the tuple of any_method_data will be out of date. Probably we should just mimic the entire any interface rather than inherit publicly.

@dyp wrote a proof of concept in comments to the OP. This is based off his work, cleaned up with value-semantics (stolen from boost::any) added. @cpplearner's pointer-based solution was used to shorten it (thanks!), and then I added the vtable optimization on top of that.

---

First we use a tag to pass around types:

template<class T>struct tag_t{constexpr tag_t(){};};
template<class T>constexpr tag_t<T> tag{};

This trait class gets the signature stored with an any_method:

This creates a function pointer type, and a factory for said function pointers, given an any_method:

template<class any_method, class Sig=any_sig_from_method<any_method>>
struct any_method_function;

template<class any_method, class R, class...Args>
struct any_method_function<any_method, R(Args...)>
{
  using type = R(*)(boost::any&, any_method const*, Args...);
  template<class T>
  type operator()( tag_t<T> )const{
    return [](boost::any& self, any_method const* method, Args...args) {
      return (*method)( boost::any_cast<T&>(self), decltype(args)(args)... );
    };
  }
};

Now we don't want to store a function pointer per operation in our super_any. So we bundle up the function pointers into a vtable:

template<class...any_methods>
using any_method_tuple = std::tuple< typename any_method_function<any_methods>::type... >;

template<class...any_methods, class T>
any_method_tuple<any_methods...> make_vtable( tag_t<T> ) {
  return std::make_tuple(
    any_method_function<any_methods>{}(tag<T>)...
  );
}

template<class...methods>
struct any_methods {
private:
  any_method_tuple<methods...> const* vtable = 0;
  template<class T>
  static any_method_tuple<methods...> const* get_vtable( tag_t<T> ) {
    static const auto table = make_vtable<methods...>(tag<T>);
    return &table;
  }
public:
  any_methods() = default;
  template<class T>
  any_methods( tag_t<T> ): vtable(get_vtable(tag<T>)) {}
  any_methods& operator=(any_methods const&)=default;
  template<class T>
  void change_type( tag_t<T> ={} ) { vtable = get_vtable(tag<T>); }

  template<class any_method>
  auto get_invoker( tag_t<any_method> ={} ) const {
    return std::get<typename any_method_function<any_method>::type>( *vtable );
  }
};

we could specialize this for a cases where the vtable is small (for example, 1 item), and use direct pointers stored in-class in those cases for efficiency.

Now we start the super_any. I use super_any_t to make the declaration of super_any a bit easier.

template<class...methods>
struct super_any_t;

This searches the methods that the super any supports for SFINAE:

template<class super_any, class method>
struct super_method_applies : std::false_type {};

template<class M0, class...Methods, class method>
struct super_method_applies<super_any_t<M0, Methods...>, method> :
    std::integral_constant<bool, std::is_same<M0, method>{}  || super_method_applies<super_any_t<Methods...>, method>{}>
{};

This is the pseudo-method pointer, like print, that we create globally and constly.

We store the object we construct this with inside the any_method. Note that if you construct it with a non-lambda things can get hairy, as the type of this any_method is used as part of the dispatch mechanism.

template<class Sig, class F>
struct any_method {
  using signature=Sig;

private:
  F f;
public:

  template<class Any,
    // SFINAE testing that one of the Anys's matches this type:
    std::enable_if_t< super_method_applies< std::decay_t<Any>, any_method >{}, int>* =nullptr
  >
  friend auto operator->*( Any&& self, any_method const& m ) {
    // we don't use the value of the any_method, because each any_method has
    // a unique type (!) and we check that one of the auto*'s in the super_any
    // already has a pointer to us.  We then dispatch to the corresponding
    // any_method_data...

    return [&self, invoke = self.get_invoker(tag<any_method>), m](auto&&...args)->decltype(auto)
    {
      return invoke( decltype(self)(self), &m, decltype(args)(args)... );
    };
  }
  any_method( F fin ):f(std::move(fin)) {}

  template<class...Args>
  decltype(auto) operator()(Args&&...args)const {
    return f(std::forward<Args>(args)...);
  }
};

A factory method, not needed in C++17 I believe:

template<class Sig, class F>
any_method<Sig, std::decay_t<F>>
make_any_method( F&& f ) {
    return {std::forward<F>(f)};
}

This is the augmented any. It is both an any, and it carries around a bundle of type-erasure function pointers that change whenever the contained any does:

template<class... methods>
struct super_any_t:boost::any, any_methods<methods...> {
private:
  template<class T>
  T* get() { return boost::any_cast<T*>(this); }

public:
  template<class T,
    std::enable_if_t< !std::is_same<std::decay_t<T>, super_any_t>{}, int>* =nullptr
  >
  super_any_t( T&& t ):
    boost::any( std::forward<T>(t) )
  {
    using dT=std::decay_t<T>;
    this->change_type( tag<dT> );
  }

  super_any_t()=default;
  super_any_t(super_any_t&&)=default;
  super_any_t(super_any_t const&)=default;
  super_any_t& operator=(super_any_t&&)=default;
  super_any_t& operator=(super_any_t const&)=default;

  template<class T,
    std::enable_if_t< !std::is_same<std::decay_t<T>, super_any_t>{}, int>* =nullptr
  >
  super_any_t& operator=( T&& t ) {
    ((boost::any&)*this) = std::forward<T>(t);
    using dT=std::decay_t<T>;
    this->change_type( tag<dT> );
    return *this;
  }  
};

Because we store the any_methods as const objects, this makes making a super_any a bit easier:

template<class...Ts>
using super_any = super_any_t< std::remove_const_t<std::remove_reference_t<Ts>>... >;

Test code:

const auto print = make_any_method<void(std::ostream&)>([](auto&& p, std::ostream& t){ t << p << "\n"; });
const auto wprint = make_any_method<void(std::wostream&)>([](auto&& p, std::wostream& os ){ os << p << L"\n"; });

const auto wont_work = make_any_method<void(std::ostream&)>([](auto&& p, std::ostream& t){ t << p << "\n"; });

struct X {};
int main()
{
  super_any<decltype(print), decltype(wprint)> a = 7;
  super_any<decltype(print), decltype(wprint)> a2 = 7;

  (a->*print)(std::cout);

  (a->*wprint)(std::wcout);

  // (a->*wont_work)(std::cout);

  double d = 4.2;
  a = d;

  (a->*print)(std::cout);
  (a->*wprint)(std::wcout);

  (a2->*print)(std::cout);
  (a2->*wprint)(std::wcout);

  // a = X{}; // generates an error if you try to store a non-printable
}

live example.

The error message when I try to store a non-printable struct X{}; inside the super_any seems reasonable at least on clang:

main.cpp:150:87: error: invalid operands to binary expression ('std::ostream' (aka 'basic_ostream<char>') and 'X')
const auto x0 = make_any_method<void(std::ostream&)>([](auto&& p, std::ostream& t){ t << p << "\n"; });

this happens the moment you try to assign the X{} into the super_any<decltype(x0)>.

The structure of the any_method is sufficiently compatible with the pseudo_method that acts similarly on variants that they can probably be merged.

---

I used a manual vtable here to keep the type erasure overhead to 1 pointer per super_any. This adds a redirection cost to every any_method call. We could store the pointers directly in the super_any very easily, and it wouldn't be hard to make that a parameter to super_any. In any case, in the 1 erased method case, we should just store it directly.

---

Two different any_methods of the same type (say, both containing a function pointer) spawn the same kind of super_any. This causes problems at lookup.

Distinguishing between them is a bit tricky. If we changed the super_any to take auto* any_method, we could bundle all of the identical-type any_methods up in the vtable tuple, then do a linear search for a matching pointer if there are more than 1. The linear search should be optimized away by the compiler unless you are doing something crazy like passing a reference or pointer to which particular any_method we are using.

That seems beyond the scope of this answer, however; the existence of that improvement is enough for now.

---

In addition, a ->* that takes a pointer (or even reference!) on the left hand side can be added, letting it detect this and pass that to the lambda as well. This can make it truly an "any method" in that it works on variants, super_anys, and pointers with that method.

With a bit of if constexpr work, the lambda can branch on doing an ADL or a method call in every case.

This should give us:

(7->*print)(std::cout);

((super_any<&print>)(7)->*print)(std::cout); // C++17 version of above syntax

((std::variant<int, double>{7})->*print)(std::cout);

int* ptr = new int(7);
(ptr->*print)(std::cout);

(std::make_unique<int>(7)->*print)(std::cout);
(std::make_shared<int>(7)->*print)(std::cout);

with the any_method just "doing the right thing" (which is feeding the value to std::cout <<).

Pasta

I think it should be possible to use virtual functions instead of the function pointers, by constructing a new type that derives from class template instantiations that create the necessary code within super_any_t::set_operation_to. With multiple inheritance, that should even be about as short as the assignment to function pointers. Since you restrict the input functions to be stateless/pure, storing those pointers in a vtable once per list of input function types seems possible. – Gimcrackery

"as I don't have a C++17 compiler." You mean apart from Wandbox and any of the other online compilers? Also: apt.llvm.org. – Candler

@Candler I think Yakk meant that none of the compiler supports fully C++17 here auto template parameters... – Furlana

@Gimcrackery a manual vtable, where we create a pointer to a static tuple created on a per-type-stored basis, would be better. Overhead of one pointer per super any instance, instead of one per method per instance. This avoids use of new (either placement or not). Placement new is no good as it requires "luck" to get size right (or static asserts, or non-insane compilers), plus adds level of indierction unless very careful. – Pasta

Is this possible to get rid of signature in any_method_function ? So it could be possible to use template parameters? Like std::variant with std::visit. – Wineskin

@tower that us the double dispatch problem. Examibe solutions without tyoe erasure; a similar one could be done here. But they are orthogonal problems. – Pasta

@Yakk-AdamNevraumont Intention is to have this interface coliru.stacked-crooked.com/a/a1cc393ffd2aa879 . In your super_any, during emplace, you store "functor" with tag<T>, and on call you any_cast to this concrete type. But in order to use functor you need to have it signature, which limits you to only concrete types use. Opposite way of doing this, is having "any_visitor" with KNOWN set of types, and try any_cast to them one by one, then call visitor (callbacks may have template arguments). I don't see how this can be done with or without type-erasure. – Wineskin

@Wineskin Have you solved your problem without using super_any kind of type erasure, using normal virtual functions? If you have not, then I don't even know how to talk to you about the solution in this more complex area. I cannot tell from your response. But the solution for double-dispatch in a virtual inheritance case and the solution here is going to be basically the same. Your "intended" solution does not look like the virtual double-dispatch case, hence my confusion. – Pasta

@Yakk-AdamNevraumont To solve it using virtual functions, you need template virtual functions... "intended" is not a solution - that what I'm asking about; here is a little changed version pastebin.com/hXXPLNFP . The only difference from your WORKING solution - make_any_method does not have function signature. Initially I asked does this doable? You said there is some way to do this. I don't see it. – Wineskin

@Wineskin Ok. So calm down and google "double dispatch". Double dispatch is how you virtually dispatch a single function based on the dynamic type of two of its arguments (say, the this pointer and a function argument). There is a whole bunch of work on double dispatch in C++, including a paper by Bjarn. There is a whole continuum of solutions going from manual visitors to using variants and automatic method writing with CRTP all the way to Bjarn's paper. This problem is completely orthogonal to super_any. – Pasta

@Wineskin So it is possible, but it isn't something you should be talking about in a comment thread in an unrelated question on stack overflow. I kept on saying "double dispatch problem". Until you grasp that problem, I don't even have the vocabulary to talk to you about it. And no, I'm not going to solve both problems simultaneously in a comment thread on SO, but I will assert is is solvable depending on the exact details of the problem which again I cannot even talk about until you actually do homework. – Pasta