Is there a way to have a Rust closure that moves only some variables into it?

Question

I have a general struct with settings and an extra variable setting that I want to tune and play around with.

For all possible values in an integer range, I want to start a (scoped) thread with this variable set to that value. Depending on this value, they do slightly different work.

Each of these threads should be able to read the general settings struct.

use crossbeam; // 0.7.3

struct Settings {
    // ... many fields
}

const MAX_FEASIBLE_SCORE: u8 = 10;

fn example(settings: Settings) {
    crossbeam::scope(|scope| {
        for score in 0..MAX_FEASIBLE_SCORE {
            scope.spawn(|_| {
                let work_result = do_cool_computation(&settings, score);
                println!("{:?}", work_result);
            });
        }
    })
    .unwrap();
}

fn do_cool_computation(_: &Settings, _: u8) {}

This does not compile:

error[E0373]: closure may outlive the current function, but it borrows `score`, which is owned by the current function
  --> src/lib.rs:12:25
   |
10 |     crossbeam::scope(|scope| {
   |                       ----- has type `&crossbeam_utils::thread::Scope<'1>`
11 |         for score in 0..MAX_FEASIBLE_SCORE {
12 |             scope.spawn(|_| {
   |                         ^^^ may outlive borrowed value `score`
13 |                 let work_result = do_cool_computation(&settings, score);
   |                                                                  ----- `score` is borrowed here
   |
note: function requires argument type to outlive `'1`
  --> src/lib.rs:12:13
   |
12 | /             scope.spawn(|_| {
13 | |                 let work_result = do_cool_computation(&settings, score);
14 | |                 println!("{:?}", work_result);
15 | |             });
   | |______________^
help: to force the closure to take ownership of `score` (and any other referenced variables), use the `move` keyword
   |
12 |             scope.spawn(move |_| {
   |                         ^^^^^^^^

This would invalidate &settings since the first loop iteration will take ownership of settings in a move closure.

The only easy ways to make it work were:

• copy the Settings struct into each thread (which in my real application is rather expensive)
• introduce an Arc around settings, which also feels a bit unfortunate.

Is there a way that we can circumvent reference counting here? Is there a way we can move score into the inner closure while still being allowed to reference settings?

Peacock

Answer 1

Yes, it is possible to move only one or some variables into a closure (rather than all or none).

Yes, this can be used to "circumvent" reference counting.

I found an answer in the documentation of rayon::scope that turns out to be exactly about this problem: 'Accessing the stack data [from within a scoped threads scope]'. That page also has an example that is clearer than the pseudocode in this question.

It turns out that you fix this as follows:

Use a move closure but refer to variables in the outer scope by shadowing them with a reference, therefore capturing them by reference rather than by value, using let settings = &settings:

```
crossbeam::scope(|scope| {
    let settings = &settings; // refer to outer variable by reference
    for score in 0..MAX_FEASIBLE_SCORE {
        scope.spawn(move |_| {
            let work_result = do_cool_computation(settings, score);
            println!("{:?}", work_result);
        });
    }
})
.unwrap();
```

Playground So here, we move 'all used variables' but turn settings into a reference beforehand, so it is borrowed instead. (Pedantically: we 'move the reference' but that is exactly what a 'borrow' is.)

There is also a second possibility: Borrowing all variables (and moving nothing).

This possibility works in many contexts but not here. (Because here, we have to capture score by value as it will not be there during the next iteration of the for loop and the closure passed to scope.spawn will outlive this).

Taken from the documentation of rayon::scope

use rayon;

fn main() {
let ok: Vec<i32> = vec![1, 2, 3];
rayon::scope(|s| {
    let bad: Vec<i32> = vec![4, 5, 6];
    s.spawn(|_| {
        // Transfer ownership of `bad` into a local variable (also named `bad`).
        // This will force the closure to take ownership of `bad` from the environment.
        let bad = bad;
        println!("ok: {:?}", ok); // `ok` is only borrowed.
        println!("bad: {:?}", bad); // refers to our local variable, above.
    });

    s.spawn(|_| println!("ok: {:?}", ok)); // we too can borrow `ok`
});
}

Playground

Peacock

Super minor nitpick here, but you're not really "circumventing" anything, you're just not reference counting. – Ninetieth

@trentcl thanks! You are of course correct. 'Circumvention' is maybe a bit of sloppy language here. I have slightly altered the way it is written in the answer, but if you have better suggestions to make it more clear, feel free to propose one (and/or edit the answer yourself directly) – Peacock

Your second answer does not work. – Bluecollar

Could the second answer be made to work with some modification? – Dotson

@Dotson It cannot. Thanks for bringing my attention to this old answer. I have replaced the second example with the original from rayon::scope's docs it was based on, and added an explanation on why that situation is simpler/different from the one we have here (as here the spawn is nested inside a for loop). – Peacock

Answer 2

The closure! macro from the closure crate gives the ability to selectively reference, move, or clone variables into a closure.

Example taken from the docs:

use closure::closure;

let string = "move".to_string();
let x = 10;
let mut y = 20;
let rc = Rc::new(5);

let closure = closure!(move string, ref x, ref mut y, clone rc, |arg: i32| {
    ...
});

Variables that are captured but not listed default to being moved.

Gree