I have been skimming through the news and the source code of the newest LTE Java 17 version and I have encountered with new Stream method called mapMulti. The early-access JavaDoc says it is similar to flatMap.
<R> Stream<R> mapMulti(BiConsumer<? super T,? super Consumer<R>> mapper)
flatMap. When is each one preferable?mapper can be called?Stream::mapMulti is a new method that is classified as an intermediate operation.
It requires a BiConsumer<T, Consumer<R>> mapper of the element about to be processed a Consumer. The latter makes the method look strange at the first glance because it is different from what we are used to at the other intermediate methods such as map, filter, or peek where none of them use any variation of *Consumer.
The purpose of the Consumer provided right within the lambda expression by the API itself is to accept any number elements to be available in the subsequent pipeline. Therefore, all the elements, regardless of how many, will be propagated.
One to some (0..1) mapping (similar to filter)
Using the consumer.accept(R r) for only a few selected items achieves filter-alike pipeline. This might get useful in case of checking the element against a predicate and it's mapping to a different value, which would be otherwise done using a combination of filter and map instead. The following
Stream.of("Java", "Python", "JavaScript", "C#", "Ruby")
.mapMulti((str, consumer) -> {
if (str.length() > 4) {
consumer.accept(str.length()); // lengths larger than 4
}
})
.forEach(i -> System.out.print(i + " "));
// 6 10
One to one mapping (similar to map)
Working with the previous example, when the condition is omitted and every element is mapped into a new one and accepted using the consumer, the method effectively behaves like map:
Stream.of("Java", "Python", "JavaScript", "C#", "Ruby")
.mapMulti((str, consumer) -> consumer.accept(str.length()))
.forEach(i -> System.out.print(i + " "));
// 4 6 10 2 4
One to many mapping (similar to flatMap)
Here things get interesting because one can call consumer.accept(R r) any number of times. Let's say we want to replicate the number representing the String length by itself, i.e. 2 becomes 2, 2. 4 becomes 4, 4, 4, 4. and 0 becomes nothing.
Stream.of("Java", "Python", "JavaScript", "C#", "Ruby", "")
.mapMulti((str, consumer) -> {
for (int i = 0; i < str.length(); i++) {
consumer.accept(str.length());
}
})
.forEach(i -> System.out.print(i + " "));
// 4 4 4 4 6 6 6 6 6 6 10 10 10 10 10 10 10 10 10 10 2 2 4 4 4 4
The very idea of this mechanism is that is can be called multiple times (including zero) and its usage of SpinedBuffer internally allows to push the elements into a single flattened Stream instance without creating a new one for every group of output elements unlike flatMap. The JavaDoc states two use-cases when using this method is preferable over flatMap:
- When replacing each stream element with a small (possibly zero) number of elements. Using this method avoids the overhead of creating a new Stream instance for every group of result elements, as required by flatMap.
- When it is easier to use an imperative approach for generating result elements than it is to return them in the form of a Stream.
Performance-wise, the new method mapMulti is a winner in such cases. Check out the benchmark at the bottom of this answer.
Using this method instead of filter or map separately doesn't make sense due to its verbosity and the fact one intermediate stream is created anyway. The exception might be replacing the .filter(..).map(..) chain called together, which comes handy in the case such as checking the element type and its casting.
int sum = Stream.of(1, 2.0, 3.0, 4F, 5, 6L)
.mapMultiToInt((number, consumer) -> {
if (number instanceof Integer) {
consumer.accept((Integer) number);
}
})
.sum();
// 6
int sum = Stream.of(1, 2.0, 3.0, 4F, 5, 6L)
.filter(number -> number instanceof Integer)
.mapToInt(number -> (Integer) number)
.sum();
As seen above, its variations like mapMultiToDouble, mapMultiToInt and mapMultiToLong were introduced. This comes along the mapMulti methods within the primitive Streams such as IntStream mapMulti(IntStream.IntMapMultiConsumer mapper). Also, three new functional interfaces were introduced. Basically, they are the primitive variations of BiConsumer<T, Consumer<R>>, example:
@FunctionalInterface
interface IntMapMultiConsumer {
void accept(int value, IntConsumer ic);
}
The real power of this method is in its flexibility of usage and creating only one Stream at a time, which is the major advantage over flatMap. The two below snippets represent a flatmapping of Product and its List<Variation> into 0..n offers represented by the Offer class and based on certain conditions (product category and the variation availability).
Product with String name, int basePrice, String category and List<Variation> variations.Variation with String name, int price and boolean availability.List<Product> products = ...
List<Offer> offers = products.stream()
.mapMulti((product, consumer) -> {
if ("PRODUCT_CATEGORY".equals(product.getCategory())) {
for (Variation v : product.getVariations()) {
if (v.isAvailable()) {
Offer offer = new Offer(
product.getName() + "_" + v.getName(),
product.getBasePrice() + v.getPrice());
consumer.accept(offer);
}
}
}
})
.collect(Collectors.toList());
List<Product> products = ...
List<Offer> offers = products.stream()
.filter(product -> "PRODUCT_CATEGORY".equals(product.getCategory()))
.flatMap(product -> product.getVariations().stream()
.filter(Variation::isAvailable)
.map(v -> new Offer(
product.getName() + "_" + v.getName(),
product.getBasePrice() + v.getPrice()
))
)
.collect(Collectors.toList());
The use of mapMulti is more imperatively inclined compared to the declarative approach of the previous-versions Stream methods combination seen in the latter snippet using flatMap, map, and filter. From this perspective, it depends on the use-case whether is easier to use an imperative approach. Recursion is a good example described in the JavaDoc.
As promised, I have wrote a bunch of micro-benchmarks from ideas collected from the comments. As long as there is quite a lot of code to publish, I have created a GitHub repository with the implementation details and I am about to share the results only.
Stream::flatMap(Function) vs Stream::mapMulti(BiConsumer) Source
Here we can see the huge difference and a proof the newer method actually works as described and its usage avoid the overhead of creating a new Stream instance with each processed element.
Benchmark Mode Cnt Score Error Units
MapMulti_FlatMap.flatMap avgt 25 73.852 ± 3.433 ns/op
MapMulti_FlatMap.mapMulti avgt 25 17.495 ± 0.476 ns/op
Stream::filter(Predicate).map(Function) vs Stream::mapMulti(BiConsumer) Source
Using chained pipelines (not nested, though) is fine.
Benchmark Mode Cnt Score Error Units
MapMulti_FilterMap.filterMap avgt 25 7.973 ± 0.378 ns/op
MapMulti_FilterMap.mapMulti avgt 25 7.765 ± 0.633 ns/op
Stream::flatMap(Function) with Optional::stream() vs Stream::mapMulti(BiConsumer) Source
This one is very interesting, escpecially in terms of usage (see the source code): we are now able to flatten using mapMulti(Optional::ifPresent) and as expected, the new method is a bit faster in this case.
Benchmark Mode Cnt Score Error Units
MapMulti_FlatMap_Optional.flatMap avgt 25 20.186 ± 1.305 ns/op
MapMulti_FlatMap_Optional.mapMulti avgt 25 10.498 ± 0.403 ns/op
Consumer into a stream. E.g., an already existing Iterator. But also converting a stream of CompletionStage instances into a stream of results simplified to mapMulti(CompletionStage::thenAccept)… –
Mump SpinnedBuffer is only in the default implementation, ReferencePipeline overrides that and "exposes" the actual Sink (elements are consumed by this), which is what that consumer is. This is rather crazy, never thought about it this way. –
Mielke Optional, i.e. flatMap(Optional::stream) vs. mapMulti(Optional::ifPresent). Only the limited type inference is annoying. It seems, we almost always need .<IntendedType>mapMulti(…)… –
Mump COUNT variable via annotation into a parameter that JMH can vary –
Mump @Forks for lower error. I have a git project that does this against github ( since it takes a lot of time multiple forks ) ... –
Mielke To address the scenario
When it is easier to use an imperative approach for generating result elements than it is to return them in the form of a Stream.
We can see it as now having a limited variant of the yield statement C#. The limitations are that we always need an initial input from a stream, as this is an intermediate operation, further, there’s no short-circuiting for the elements we’re pushing in one function evaluation.
Still, it opens interesting opportunities.
E.g., implementing a stream of Fibonacci number formerly required a solution using temporary objects capable of holding two values.
Now, we can use something like:
IntStream.of(0)
.mapMulti((a,c) -> {
for(int b = 1; a >=0; b = a + (a = b))
c.accept(a);
})
/* additional stream operations here */
.forEach(System.out::println);
It stops when the int values overflow, as said, it won’t short-circuit when we use a terminal operation that does not consume all values, however, this loop producing then-ignored values might still be faster than the other approaches.
Another example inspired by this answer, to iterate over a class hierarchy from root to most specific:
Stream.of(LinkedHashMap.class).mapMulti(MapMultiExamples::hierarchy)
/* additional stream operations here */
.forEach(System.out::println);
}
static void hierarchy(Class<?> cl, Consumer<? super Class<?>> co) {
if(cl != null) {
hierarchy(cl.getSuperclass(), co);
co.accept(cl);
}
}
which unlike the old approaches does not require additional heap storage and will likely run faster (assuming reasonable class depths that do not make recursion backfire).
Also monsters like this
List<A> list = IntStream.range(0, r_i).boxed() .flatMap(i -> IntStream.range(0, r_j).boxed() .flatMap(j -> IntStream.range(0, r_k) .mapToObj(k -> new A(i, j, k)))) .collect(Collectors.toList());
can now be written like
List<A> list = IntStream.range(0, r_i).boxed()
.<A>mapMulti((i,c) -> {
for(int j = 0; j < r_j; j++) {
for(int k = 0; k < r_k; k++) {
c.accept(new A(i, j, k));
}
}
})
.collect(Collectors.toList());
Compared to the nested flatMap steps, it loses some parallelism opportunity, which the reference implementation didn’t exploit anyway. For a non-short-circuiting operation like above, the new method likely will benefit from the reduced boxing and less instantiation of capturing lambda expressions. But of course, it should be used judiciously, not to rewrite every construct to an imperative version (after so many people tried to rewrite every imperative code into a functional version)…
© 2022 - 2024 — McMap. All rights reserved.
flatMap, but they must have had reasons to not overload. Then, I am glad that I am not the only one naming APIs poorly. Bookmarking this to try it out and understand, what stopped them to improve the performance with the existing combination offilter.maporflatMapinstead of providing a new API. Thanks for sharing anyway. – Jewfishfilter.mapagainstmapMulti. – Blamedinstanceof– Yocum