Why doesn't C++11 implicitly convert lambdas to std::function objects?

Question

I implemented a generic event emitter class which allows code to register callbacks, and emit events with arguments. I used Boost.Any type erasure to store the callbacks so they can have arbitrary parameter signatures.

It all works, but for some reason, lambdas being passed in must first be turned into std::function objects. Why doesn't the compiler infer that the lambda is the function type? Is it because of the way I use variadic templates?

I use Clang (version string: Apple LLVM version 5.0 (clang-500.2.79) (based on LLVM 3.3svn)).

Code:

#include <functional>
#include <iostream>
#include <map>
#include <string>
#include <vector>

#include <boost/any.hpp>


using std::cout;
using std::endl;
using std::function;
using std::map;
using std::string;
using std::vector;


class emitter {

   public:

      template <typename... Args>
      void on(string const& event_type, function<void (Args...)> const& f) {
         _listeners[event_type].push_back(f);
      }

      template <typename... Args>
      void emit(string const& event_type, Args... args) {
         auto listeners = _listeners.find(event_type);
         for (auto l : listeners->second) {
            auto lf = boost::any_cast<function<void (Args...)>>(l);
            lf(args...);
         }
      }

   private:

      map<string, vector<boost::any>> _listeners;

};


int main(int argc, char** argv) {

   emitter e;

   int capture = 6;

   // Not sure why Clang (at least) can't deduce the type of the lambda. I don't
   // think the explicit function<...> business should be necessary.
   e.on("my event",
        function<void ()>( // <--- why is this necessary?
           [&] () {
              cout << "my event occurred " << capture << endl;
           }));
   e.on("my event 2",
        function<void (int)>(
           [&] (int x) {
              cout << "my event 2 occurred: " << x << endl;
           }));
   e.on("my event 3",
        function<void (double)>(
           [&] (double x) {
              cout << "my event 3 occurred: " << x << endl;
           }));
   e.on("my event 4",
        function<void (int, double)>(
           [&] (int x, double y) {
              cout << "my event 4 occurred: " << x << " " << y << endl;
           }));

   e.emit("my event");
   e.emit("my event 2", 1);
   e.emit("my event 3", 3.14159);
   e.emit("my event 4", 10, 3.14159);

   return EXIT_SUCCESS;
}

Radmilla

Answer 1

A lambda is not a std::function, and std::function is not a lambda.

A lambda is syntactic sugar to create an anonymous class that looks like this:

struct my_lambda {
private:
  int captured_int;
  double captured_double;
  char& referenced_char;
public:
  int operator()( float passed_float ) const {
    // code
  }
};
int captured_int = 7;
double captured_double = 3.14;
char referenced_char = 'a';
my_lambda closure {captured_int, captured_double, referenced_char};
closure( 2.7f );

from this:

int captured_int = 7;
double captured_double = 3.14;
char referenced_char = 'a';
auto closure = [=,&referenced_char](float passed_float)->int {
  // code
};
closure(2.7);

with the type name of the my_lambda actually being some unnameable type.

A std::function is a completely different thing. It is an object that does implement operator() with a particular signature, and stores a smart value-semantics pointer to an abstract interface that covers copy/move/invoke operations. It has a templated constructor that can take any type that supports copy/move/operator() with a compatible signature, generates a concrete custom class that implement the abstract internal interface, and stores it in the above mentioned internal value-semantics smart pointer.

It then forwards operations from itself as a value-type to the abstract internal pointer, including perfect forwarding to the invocation method.

As it happens, you can store a lambda in a std::function, just like you can store a function pointer.

But there are a whole myriad of different std::function that could store a given lambda -- anything where the types are convertible to and from the arguments works, and in fact works equally well, as far as the std::function is concerned.

Type deduction in C++ in templates does not work at the level "can you convert into" -- it is pattern matching, pure and simple. As the lambda is a type unrelated to any std::function, no std::function type can be deduced from it.

If C++ tried to do that in the general case, it would have to invert a Turing-complete process to determine what (if any) set of types could be passed to the template in order to generate a conversion-compatible instance.

In theory, we could add "operator deduce template arguments from" to the language, where the implementers of a given template can write code that takes some arbitrary type, and they attempt to tease out "from this type, what template parameters should be used for an instance". But C++ does not have this.

Pongid

It looks like C++ has problems that any other language does not. In Haskell, C#, Python you have a type for lambda and you do not need any explicit conversions. – Matamoros

@trismegistos Yes, those languages all type erase lambdas by default. C++ lambdas are designed to be as fast as writing reasonably optimized C-level code, but much easier, when passed to algorithms, which means the type information of a lambda is not discarded, so algorithms that use them are replicated and distinct for each lambda passed, which makes inlining trivial. – Pongid

does it mean that when I pass create identical lamdba two times and call std::find_if on both lambdas I will have two different find_if function compiled which are binary identical? That does not sound like good optimization. – Matamoros

@trismegistos yes, there are two. Under the as-if if the address (of the resulting functuon) is never taken, the duplicate can be eliminated. MSVC calls that COMDAT folding. (if address is taken, by standard the address must differ (some compilers violate this?), but body can be the same via a goto stub). Also two closure instances of the same lambda are, of course, the same lambda. – Pongid

Answer 2

The compiler doesn't infer anything because the compiler implements the C++ language, and the language's template argument deduction rules do not allow the deduction in the way you want.

Here's a simple example that represents your situation:

template <typename T> struct Foo
{
    Foo(int) {}
};

template <typename T> void magic(Foo<T> const &);

int main()
{
    magic(10);   // what is T?
}

Phenformin

I think this explains why you can't pass a lambda directly into a method taking a templatized function, but I think that the underlying issue here has more to do with any's type deduction rules than with std::function. Or am I mistaken? – Lunde

It certainly seems to me that the compiler should infer that T is an int, since that information is clearly present at compile-time. Are you saying that it just doesn't and that's the end of that? – Radmilla

@Radmilla Why not Foo<char>? or Foo<void>? Every possible instantiation of Foo has a converting constructor from int. – Barrick

Answer 3

When boost::any stores a value, it uses the static type of that object to determine what type of object is being stored. You can then only cast the any back to an object of the right type if you specify the static type of what's being stored.

Each C++ lambda is associated with an implementation-defined type that's opaque to the user. Although lambdas can be called as functions, they don't directly evaluate to std::functions. The cast is necessary when storing the lambda in the any to ensure that the static type of what's stored is a std::function for when you cast back.

Hope this helps!

Lunde