Composing adaptors in Boost::range

Question

I started playing with Boost::Range in order to have a pipeline of lazy transforms in C++. My problem now is how to split a pipeline in smaller parts. Suppose I have:

int main(){
  auto map = boost::adaptors::transformed; // shorten the name
  auto sink = generate(1) | map([](int x){ return 2*x; })
                          | map([](int x){ return x+1; })
                          | map([](int x){ return 3*x; });
  for(auto i : sink)
    std::cout << i << "\n";
}

And I want to replace the first two maps with a magic_transform, i.e.:

int main(){
  auto map = boost::adaptors::transformed; // shorten the name
  auto sink = generate(1) | magic_transform()
                          | map([](int x){ return 3*x; });
  for(auto i : sink)
    std::cout << i << "\n";
}

How would one write magic_transform? I looked up Boost::Range's documentation, but I can't get a good grasp of it.

Addendum: I'm looking to write a class like this:

class magic_transform {
    ... run_pipeline(... input) {
        return input | map([](int x){ return 2*x; })
                     | map([](int x){ return x+1; });
};

Tallyman

What is the return type of generate? I am worried that your sink contains a reference to a temporary that expires at the semicolon. – Mcloughlin

generate's return type is boost::iterator_range, please check liveworkspace.org/code/841508d3b54bed4181d4e9fb6058200f for details. – Tallyman

Magic transform does not get an input parameter. You wrote it as a nullary function. – Sylvanite

OK, it would be more a class than a function. See above. – Tallyman

Your needs are still vague. Is it the pipe syntax you want? A transform around chained lamdas gives the functionality and external use of |. Do you actually want | within your magic_transform, and if so why? – Sylvanite

Yes, I want | within my magic_transform - I'm designing a program that consists mostly of a pipeline, and the pipeline can get complex. I want to be able to break it into smaller pipes :) – Tallyman

Answer 1

The most difficult problem is figuring out the return type in the code. decltype and lambdas do not mix well (see here), so we have to think of an alternative way:

auto map = boost::adaptors::transformed;

namespace magic_transform
{
   std::function<int(int)> f1 = [](int x){ return 2*x; };
   std::function<int(int)> f2 = [](int x){ return x+1; };
   template <typename Range>
   auto run_pipeline(Range input) -> decltype(input | map(f1) | map(f1))
   {
        return input | map(f1) | map(f2);
   }
}

...
auto sink = magic_transform::run_pipeline(generate(1))
                          | map([](int x){ return 3*x; });

The simple solution is stick the lambdas into std::function so we can use decltype to deduce the return type. I used namespace magic_transform in the example, but you could adapt this code into a class if you would like also. Here is a link adapting your code to the above.

Also, using std::function might be overkill here. Instead you could just declare two normal functions instead (example).

I was also experimenting with boost::any_range, there seems to be some incompatibilities with C+11 lambdas, etc. The closest I could get was the following (example):

auto map = boost::adaptors::transformed;
using range = boost::any_range<
               const int,
               boost::forward_traversal_tag,
               const int&,
               std::ptrdiff_t
               >;

namespace magic_transform
{
    template <typename Range>
    range run_pipeline(Range r)
    {
        return r | map(std::function<int(int)>([](int x){ return 2*x; }))
             | map(std::function<int(int)>([](int x){ return x+1; }));
    }
}

int main(){
  auto sink = magic_transform::run_pipeline(boost::irange(0, 10))
                          | map([](int x){ return 3*x; });
  for(auto i : sink)
    std::cout << i << "\n";
}

Diocesan

Before asking here, I tried decltype and found it too verbose for this application (you end up writing the pipeline twice). The any_range, OTOH, allows you to only specify input and output, correct? – Tallyman

Now I see, any_range specifies the output (value and reference). Interesting. – Tallyman

Also, you can use any_range for the parameter too. E.g.: range run_pipeline(range r) defines a pipeline that acts on a range of ints and returns a range of ints. – Tallyman

@brunonery: Yes, I couldn't get any_range to work with your generate(1) example though. – Diocesan

@brunonery: I found out you need to change the iterator tag to std::forward_iterator_tag, here is a working example. – Diocesan

Answer 2

I think will work:

auto magic_transform()->decltype(boost::adaptors::transformed(std::function<int(int)>())
{
    std::function<int(int)> retval = [](int x){ return [](int x){ return x+1; }(2*x);
    return boost::adaptors::transformed(retval);
}

but it probably isn't what you are looking for. :) (Jokes in the above code: chained lambdas for 2*x+1, use of decltype on basically the implementation to find the return type),

Looking at the source code for http://www.boost.org/doc/libs/1_46_1/boost/range/adaptor/transformed.hpp, the type that magic_transform wants to return is boost::range_detail::transform_holder<T>, where T is the type of the function.

When you do it on the stack with lambdas, T ends up being some very narrow type. If you want to pass around abstract transformations without exposing all of the details, using std::function<outtype(intype)> may be reasonable (there will be a small run-time overhead).

Hope that works.

Sylvanite

Yes, chaining the lambdas wasn't what I was looking for :), please check my update. I could actually use auto and decltype, but I wanted something more generic. – Tallyman

So you want some way to pipe (well combine in order) transforms together and have the result be a transform? – Sylvanite

Not only transforms, but eventually filters and other range adaptors. – Tallyman

Range adaptor's result type is documentated. for example, transformed returns boost::transformed_range<Range> boost.org/doc/libs/1_52_0/libs/range/doc/html/range/reference/… – Helms