I originally encountered this issue when attempting to create a Rayon ParallelIterator that a flat_map
operation was applied to a dynamic number of times see this post on the Rayon issue tracker.
Note: I asked the same question on the Rust User forum as well (see here), but StackOverflow might be a better venue for this question.
For standard (i.e. that execute serially) Iterator
s the following works:
fn fancy<Input, Output, Item>(input: Input, depth: usize) -> Output
where
Input : IntoIterator<Item = Item>,
Output : std::iter::FromIterator<Item>,
Item : Copy + std::ops::Add<Output = Item>,
{
let mut iter: Box<dyn Iterator<Item = Item>> = Box::new(input.into_iter());
for _ in 0..depth {
iter = Box::new(iter.flat_map(|x| vec![x, x + x].into_iter()));
}
iter.collect()
}
When working with Rayon's ParallelIterator
s, however, things get iffy: ParallelIterator
requires types that implement it to be Sized
, making them non-object safe and thus impossible to pass around as trait object. Thus the following straight translation does not work:
use rayon::prelude::*;
fn fancy2<Input, Output, Item>(input: Input, depth: usize) -> Output
where
Input : IntoParallelIterator<Item = Item>,
Output : FromParallelIterator<Item>,
Item : Copy + std::ops::Add<Output = Item> + Send,
{
let mut iter: Box<dyn rayon::iter::ParallelIterator<Item = Item>> = Box::new(input.into_par_iter());
for _ in 0..depth {
iter = Box::new(iter.flat_map(|x| vec![x, x + x].into_par_iter()));
}
iter.collect()
}
My question is: How can we wrap a non-object-safe trait like ParallelIterator
in a new trait that is object-safe, so we can e.g. perform a dynamic operation on it like in above example?