Map function with c++11 constructs
Asked Answered
C

3

6

Both to teach myself about implementing more advanced template constructions than simply basic ones, and becouse they are useful in many circumstances, I'm trying to implement map, filter and similar functions common in functional programming using c++11 constructs like decltype.

I'm having trouble creating a function prototype that the compiler I use can handle, so I have to ask you how you would create something like this:

//
// Takes an iterable, applies a function to every element, and returns a vector of the results
//
template <typename T, typename Func>
auto map(const T& iterable, Func func) -> std::vector< decltype(  func( *iterable.cbegin() ) ) >
{
    // body snipped
}

That is, this function should take any iterable and a function that takes the iterables value type as an argument and returns some sort of value. The result of the function call will be a vector, regardless of the type of iterable passed in, of the type that the passed function returns.

The map function should accept any function with a valid prototype as argument, whether it is a functionpointer, a functor or lambda expression.

Using the function above with this test code:

std::vector<int> intVector;
intVector.push_back(1);
intVector.push_back(2);

map(intVector, [](int& value) { return value + 1; });

makes visual studio spit out a C2893 ("Failed to specialize function template") error and I'm not sure what's wrong.

Update: Applied changes suggested in comments and answers so far to question, new prototype tested but the same error remains.

Cognizance answered 18/2, 2013 at 21:3 Comment(5)
I could be wrong but it looks to me like you are requiring Func to be a function that takes an iterator, not the value at the iterator, as the parameter. Try replacing the decltype with this decltype(func( *(iterable.cbegin())))Aintab
You may also need to use a const reference in your lambda since you are using cbegin() which returns a const iterator.Aintab
It's worth noting that algorithms tend to take two iterator arguments rather than a full container.Softy
@sftrabbit range based algorithms are the new hotness. :)Burnell
Your map function uses cbegin and thus gets a const reference from dereferencing the iterator, but your lambda for some inexplicable reason wants a non-const reference. That's why the current version doesn't work.Upswing
C
8

This might do what you want. It uses std::transform internally, which basically does the whole work. The function I wrote is nothing more than a simple wrapper for containers (not working with C-style arrays, that would require some additional type traits):

#include <vector>
#include <algorithm>
#include <type_traits>

//
// Takes an iterable, applies a function to every element, 
// and returns a vector of the results
//
template <typename T, typename Func>
auto map_container(const T& iterable, Func&& func) ->
    std::vector<decltype(func(std::declval<typename T::value_type>()))>
{
    // Some convenience type definitions
    typedef decltype(func(std::declval<typename T::value_type>())) value_type;
    typedef std::vector<value_type> result_type;

    // Prepares an output vector of the appropriate size
    result_type res(iterable.size());

    // Let std::transform apply `func` to all elements
    // (use perfect forwarding for the function object)
    std::transform(
        begin(iterable), end(iterable), res.begin(),
        std::forward<Func>(func)
        );

    return res;
}

However, notice that your lambda should take a reference to const, or better should take its argument by value in the case of int.

Also, I renamed the function from map into map_container: it is a bad programming practice to reuse names of standard containers of the C++ Standard Library for functions, variables, or anything else in your program.

For me, this gives the desired output:

#include <iostream>

int main()
{
    std::vector<int> intVector;

    intVector.push_back(1);
    intVector.push_back(2);

    auto v = map_container(intVector, [] (int value) { return value + 1; });

    for (int i : v) { std::cout << i << " "; }
}
Colo answered 18/2, 2013 at 21:17 Comment(3)
Fails to work for int x[5] = {0}; a basic C array. This is iterable, as demonstrated by for( auto i:x ) { std::cout << i << " "; }Burnell
How so? You still examine T::value_type directly.Burnell
You can improve the efficiency here. Instead of initializing with result_type res(iterable.size()), which will default-construct iterable.size() elements into your output vector, instead just default-construct res, do a res.reserve(iterable.size()), and make your output iterator a std::back_inserter(res). (Full code in my minor improvement to this excellent answer.)Winthrop
B
4

So there are a whole bunch of corner cases to handle here. What I'd do is first attempt to build some container_traits templates to abstract as much of the work as possible.

A type is a container if it admits calls to the begin and end free functions in which std::begin and std::end are have been brought into play via using, and those two types are the same (that last might not be a requirement).

The traits of a container are mostly derived from the iterators that the container has, plus the types of said iterators. A few other features, like size (or even size_at_least -- see below), are common.

A type is said to be iterable if the const of the type is a container.

The next question is "what kind of type instances are valid for mapping the elements of a container?" -- that is also slightly non-trivial, so I added some traits classes to deal with it.

So, this leads to this implementation:

#include <algorithm>
#include <type_traits>
#include <utility>

namespace aux {
  // calculate the type that calling `begin` and `end` on a type will return
  // in a scope where `std::begin` and `std::end` are visible.  This hack is
  // required to enable argument-dependent lookup.
  using std::begin;
  using std::end;
  template<typename T>
  auto adl_begin(T&&t)->decltype( begin(std::forward<T>(t)) );
  template<typename T>
  auto adl_end(T&&t)->decltype( end(std::forward<T>(t)) );
  template<typename T>
  auto adl_cbegin(T const&t)->decltype( begin(t) );
  template<typename T>
  auto adl_cend(T const&t)->decltype( end(t) );
}

// What is a container?  Something with a `begin`ing and an `end`ing...
template<typename C,typename=void>
struct is_container:std::false_type {};
template<typename C>
struct is_container<C, typename std::enable_if<
   std::is_same<
      decltype(aux::adl_begin(std::declval<C>())),
      decltype(aux::adl_end(std::declval<C>()))
   >::value
>::type >:std::true_type {};


// Default container_traits is empty for SFINAE ease of use:
template<typename C, typename=void>
struct container_traits {};

// if it is a container, go in whole hog:
template<typename C>
struct container_traits<C, typename std::enable_if< is_container<C>::value >::type >
{
   typedef decltype( aux::adl_begin(std::declval<C>()) ) iterator;
   typedef decltype( aux::adl_cbegin(std::declval<C>()) ) const_iterator;
   // I'm lazy, so I'll copy typedefs from `iterator_traits` below:
   typedef typename std::iterator_traits<iterator>::value_type value_type;
   typedef typename std::iterator_traits<iterator>::reference reference;
   // etc

   // TODO: size_at_least is a helper function
   // it returns 0 if it is expensive to calculate the size (say, a range
   // if iterators into a `std::list`), and the size if it is cheap to
   // calculate (say, a `std::vector`, any class with a `.size()` method,
   // or a pair of pointers or other random-access iterators)
   // template<typename C2, typename=typename std::enable_if< std::is_convertable< C2, C const&>::value>::type
   // static std::size_t size_at_least( C2&& c ) { ... }
};

// Can Functor map the elements of C into something we can store elsewhere?
template<typename C, typename Functor, typename=void>
struct can_map:std::false_type {};
// Yes, if the result of calling Functor on C's elements is non-void:
template<typename C, typename Functor>
struct can_map<C, Functor, typename std::enable_if<
  !std::is_same< decltype(std::declval<Functor>()(std::declval<typename container_traits<C>::value_type>())), void >::value
>::type>: std::true_type {};

// The result of mapping the elements of C under Functor
template<typename C, typename Functor, typename=void>
struct map_result {};
template<typename C, typename Functor>
struct map_result<C,Functor,typename std::enable_if< can_map<C,Functor>::value>::type>
{
  typedef
    decltype(
      std::declval<Functor>()(
        *std::declval<
          typename container_traits<C>::const_iterator
        >()
      )
    )
  type;
};

// The actual implementation
// we std::decay the map_result because we want to store
// instances of the type, and std::decay does that quite nicely
// note that some pathological Functors may break this, ie ones
// that return pseudo-references that are intended to be read from
// yet are not std-container safe
template <typename T, typename Func>
auto map_container(T&& iterable, Func&& func) ->
  std::vector<
    typename std::decay<
      typename map_result<T, Func>::type
    >::type
  >
{
  std::vector<
    typename std::decay<
      typename map_result<T, Func>::type
    >::type
  > retval;
  // TODO: use container_traits<T>::size_at_least to reserve space in retval
  // that will bring the efficiency of this function up to near-hand-crafted-C.
  for (auto&& s:iterable) {
    retval.push_back( func(s) );
  }
  return retval;
}

And that is it. Next, test code. We should be able to map_container on C-style arrays, vectors of both conventional types and bool (which uses pseudo-references and packs the bits tightly), and on user-defined types both via the .begin() method and via free floating begin(C) functions.

One issue I had with arrays is that C const& seemed to cause pointer decay in the array, which made it no longer a container: I had to bind to C&& to get the real array type.

#include <iostream>

void test1() {
   std::vector<int> src{1,2,3,4,5};
   auto r = map_container( src, [](int x){return x*2;});
   for (auto&& x:r) {
      std::cout << x << "\n";
   }
}
struct test_buffer {
  int foo[5];
  int* begin() { return foo; }
  int* end() { return &foo[5]; }
  int const* begin() const { return foo; }
  int const* end() const { return &foo[5]; }
};
test_buffer buff1={{1,2,3,4,5}};
struct test_buffer_2 {
  int foo[5];
};
test_buffer_2 buff2={{1,2,3,4,5}};
int* begin(test_buffer_2& t) { return t.foo; }
int* end(test_buffer_2& t) { return &t.foo[5]; }
int const* begin(test_buffer_2 const& t) { return t.foo; }
int const* end(test_buffer_2 const& t) { return &t.foo[5]; }
std::vector<bool> bits{true, false, true, false};   

template<typename Container>
void tester(Container&& c) {
   Container const& src = c;
   auto r = map_container( src, [](int x){return x*2;});
   for (auto&& x:r) {
      std::cout << x << "\n";
   }
}
void test2() {
   tester(buff1);
   tester(buff2);
   tester(bits);
}
template<typename C>
bool is_container_test(C&&) {
   return is_container<C>::value;
}
template<typename C, typename F>
bool can_map_test( C&&, F&& ) {
   return can_map<C, F>::value;
}
template<typename C, typename F>
bool can_map_test2( C const&, F&& ) {
   return can_map<C, F>::value;
}
int array[] = {1,2,3,4,5};
void test3() {
   std::cout << "Array is container:" << is_container_test(array) << "\n";
   auto x2 = [](int x){return x*2;};
   std::cout << "Double can map:" << can_map_test(array, x2) << "\n";
   std::cout << "Double can map:" << can_map_test2(array, x2) << "\n";
}
void test4() {
   tester(array);
}
int main() {
   test1();
   test2();
   test3();
   test4();
}

or something along those lines. Don't do complex SFINAE in the function itself, instead create traits classes that do the work for you.

Other techniques used above: I used std::begin and std::end to get the begin/end iterators. This means I now support raw C arrays. I then wrapped this in some argument dependent lookup helpers whose purpose is to allow you to define begin and end with your class overrides in the same namespace.

Note that the "no acceptance" version of container_traits is an empty struct, not an undefined one. This lets us use container_traits in SFINAE elsewhere.

Oh, and an efficiency improvement would be to write "smart reserve" which takes a container with a reserve method and a container whose size you wish to copy. It does nothing if the container you want to copy lacks random-access iterators and lacks a .size() method, but if it does it does a .reserve( end(...)-begin(...) ) or .reserve(src.size()). We could abstract this for other algorithms by adding it to container_traits as static size_t size_at_least(Container const&), which returns a size_t in O(1) time that is no larger than the size of the Container.

Burnell answered 18/2, 2013 at 21:24 Comment(0)
W
0

A couple minor improvements on Andy Prowl's excellent answer above:

  1. We can reserve the size we need in advance, without actually resizing the vector (thereby avoiding default-constructing n elements). This has two advantages:
    1. It allows you to map over types that aren't default-constructable at all
    2. It could have significant performance advantages (as compilers are rarely able to elide those unused default-constructions), depending on the number of elements and how expensive they are to construct.
  2. We can provide an overload that works on iterator ranges, so that you can operate on part of a container.

You can play with this in Compiler Explorer, but the code itself looks like this:

template <typename Iterator, typename Func> [[nodiscard]]
auto functional_map(Iterator begin, Iterator end, Func && func) ->
std::vector<decltype(func(std::declval<typename Iterator::value_type>()))>
{
    using value_type = decltype(func(std::declval<typename Iterator::value_type>()));

    std::vector<value_type> out_vector;
    out_vector.reserve(std::distance(begin, end));
    
    std::transform(begin, end, std::back_inserter(out_vector),
                   std::forward<Func>(func));
    
    return out_vector;
}

template <typename T, typename Func> [[nodiscard]]
auto functional_map(const T & iterable, Func && func) ->
std::vector<decltype(func(std::declval<typename T::value_type>()))>
{
    return functional_map(std::begin(iterable), std::end(iterable),
                          std::forward<Func>(func));
}

(Note that I'm using C++17's [[nodiscard]] attribute in those declarations, but you could drop that without issue if you're pre-C++17.)

The Compiler Explorer link also includes a couple demonstrative tests:

TEST_CASE("Mapping ints to string")
{
    const std::vector<int> int_version{0, 1, 2, 3, 4};
    const std::vector<std::string> string_version{"0", "1", "2", "3", "4"};

    CHECK(functional_map(int_version, 
                         [](int i) { return std::to_string(i); }) == string_version);
    
    CHECK(functional_map(string_version, 
                         [](const std::string & s) { return std::stoi(s); }) == int_version);
}

TEST_CASE("Mapping over only part of a container")
{
    const std::vector<int> int_version{0, 1, 2, 3, 4, 5, 6, 7, 8, 9};
    const std::vector<std::string> string_version{"0", "1", "2", "3", "4", "5", "6", "7", "8", "9"};

    const std::vector<std::string> first_four_strings(string_version.begin(), string_version.begin() + 4);
    CHECK(functional_map(int_version.begin(), int_version.begin() + 4, 
                         [](int i) { return std::to_string(i); }) == first_four_strings);
    
    const std::vector<int> first_four_ints(int_version.begin(), int_version.begin() + 4);
    CHECK(functional_map(string_version.begin(), string_version.begin() + 4, 
                         [](const std::string & s) { return std::stoi(s); }) == first_four_ints);
}
Winthrop answered 16/12, 2020 at 15:42 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.