Is it possible to split a vector into groups of 10 with iterators?

Question

I have let my_vec = (0..25).collect::<Vec<_>>() and I would like to split my_vec into iterators of groups of 10:

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9];
[10, 11, 12, 13, 14, 15, 16, 17, 18, 19];
[20, 21, 22, 23, 24, None, None, None, None, None];

Is it possible to do that with iterators in Rust?

Nittygritty

Answer 1

There is no such helper method on the Iterator trait directly. However, there are two main ways to do it:

1. Use the [T]::chunks() method (which can be called on a Vec<T> directly). However, it has a minor difference: it won't produce None, but the last iteration yields a smaller slice. For always-exact slices that omit the final chunk if it's incomplete, see the [T]::chunks_exact() method.

Example:

    let my_vec = (0..25).collect::<Vec<_>>();

    for chunk in my_vec.chunks(10) {
        println!("{:02?}", chunk);
    }

Result:

```none
[00, 01, 02, 03, 04, 05, 06, 07, 08, 09]
[10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
[20, 21, 22, 23, 24]
```

2. Use the Itertools::chunks() method from the crate itertools. This crate extends the Iterator trait from the standard library so this chunks() method works with all iterators! Note that the usage is slightly more complicated in order to be that general. This has the same behavior as the method described above: in the last iteration, the chunk will be smaller instead of containing Nones.

Example:

    extern crate itertools;
    use itertools::Itertools;
    
    for chunk in &(0..25).chunks(10) {
        println!("{:02?}", chunk.collect::<Vec<_>>());
    }

Result:

```none
[00, 01, 02, 03, 04, 05, 06, 07, 08, 09]
[10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
[20, 21, 22, 23, 24]
```

Althing

Thanks! The first solution was exactly what I was looking for – Nittygritty

chunks seems to be a part of std now, so no need in itertools for it – Gadgeteer

@Gadgeteer chunks still doesn't exist on Iterator. There is [T]::chunks and Iterator::array_chunks, which are both neat, but don't do the same as IterTools::chunks. Or am I missing something? – Althing

I use it like this some_vec[start..end].chunks(split) – Gadgeteer

Answer 2

You can achieve a similar solution as Lukas Kalbertodt's itertools example using only the standard library:

let my_vec = (0..25).collect::<Vec<_>>();

let mut my_values = my_vec.into_iter().peekable();

while my_values.peek().is_some() {
    let chunk: Vec<_> = my_values.by_ref().take(10).collect();
    println!("{:?}", chunk);
}

Result:

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
[10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
[20, 21, 22, 23, 24]

If you don't actually need the Vec of each chunk, you can omit the collect call and operate directly on the iterator created by by_ref.

See also:

• What is the canonical way to implement is_empty for Iterator?
• Are there equivalents to slice::chunks/windows for iterators to loop over pairs, triplets etc?
• Why does Iterator::take_while take ownership of the iterator?
• Using the same iterator multiple times in Rust

Costume

Answer 3

A trick I adapted from Python to get a groupby() effect like Python's itertools provides is to combine two or more iterators using .zip(), .skip() and .step_by(). This approach could produce groups of 10, but it would be quite unaesthetic to see in a code base. But if you need small groups, this may be okay.

Rust does have the itertools crate which has .group_by(), but that's not always available depending on the situation - for instance, submitting a Rust solution to sites like HackerRank.

fn main()
{
    let a = "Hello, World!";

    println!("{:?}", a.chars().step_by(2).zip(
                         a.chars().skip(1).step_by(2) 
                     ).collect::<Vec<_>>());
}

Output:

[('H', 'e'), ('l', 'l'), ('o', ','), (' ', 'W'), ('o', 'r'), ('l', 'd')]

Where n is the number of zipped iterators, the value for .step_by will be n for all iterators, and the value for .skip() will depend on the position of the iterator in the chain; the first iterator will skip 0, the next 1, etc.

You can chain any number of iterators this way, but it starts looking a bit complicated and ugly after the third. In that case, maybe consider just collecting the iterator into a Vec and using its .chunks() method. Or if you can, use the itertools crate.

Caricature

Answer 4

Chunks one vector with or without use rayon.
And then map some function to each part.
In the following example, the sum function was used.

use rayon::prelude::*;

fn main() {
    let my_vec: Vec<_> = (0..25).collect();
    let chunk_size = 10;

    let chunks_without_rayon: Vec<_> = my_vec
        .chunks(chunk_size)
        .enumerate()
        .map(|(i, chunk)| {
            println!("chunk{i}: {chunk:?}");
            chunk.iter().sum::<i32>()
        })
        .collect();

    println!("chunks_without_rayon = {chunks_without_rayon:?}\n");

    let chunks_with_rayon: Vec<_> = my_vec
        .par_chunks(chunk_size)
        .enumerate()
        .map(|(i, chunk)| {
            println!("chunk{i}: {chunk:?}");
            chunk.iter().sum::<i32>()
        })
        .collect();

    println!("chunks_with_rayon: {chunks_with_rayon:?}")
}

The output:

chunk0: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
chunk1: [10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
chunk2: [20, 21, 22, 23, 24]
chunks_without_rayon = [45, 145, 110]

chunk0: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
chunk1: [10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
chunk2: [20, 21, 22, 23, 24]
chunks_with_rayon: [45, 145, 110]

Rectangle