Is there any way to return a reference to a variable created in a function?
Asked Answered
R

5

133

I want to write a program that will write a file in 2 steps. It is likely that the file may not exist before the program is run. The filename is fixed.

The problem is that OpenOptions.new().write() can fail. In that case, I want to call a custom function trycreate(). The idea is to create the file instead of opening it and return a handle. Since the filename is fixed, trycreate() has no arguments and I cannot set a lifetime of the returned value.

How can I resolve this problem?

use std::io::Write;
use std::fs::OpenOptions;
use std::path::Path;

fn trycreate() -> &OpenOptions {
    let f = OpenOptions::new().write(true).open("foo.txt");
    let mut f = match f {
        Ok(file)  => file,
        Err(_)  => panic!("ERR"),
    };
    f
}

fn main() {
    {
        let f = OpenOptions::new().write(true).open(b"foo.txt");
        let mut f = match f {
            Ok(file)  => file,
            Err(_)  => trycreate("foo.txt"),
        };
        let buf = b"test1\n";
        let _ret = f.write(buf).unwrap();
    }
    println!("50%");
    {
        let f = OpenOptions::new().append(true).open("foo.txt");
        let mut f = match f {
            Ok(file)  => file,
            Err(_)  => panic!("append"),
        };
        let buf = b"test2\n";
        let _ret = f.write(buf).unwrap();
    }
    println!("Ok");
}
Rabat answered 20/9, 2015 at 18:34 Comment(1)
Open this page, Ctrl-F, "Cow", no results?? While you can't return references to variables created in functions, you can use std::borrow::Cow to generalize over owned data and unowned references -- it's a Deref that lets you choose whether a given instance owns or borrows its data. I've found it to be the most reliable way to switch off between returning owned and unowned data.Aweinspiring
W
127

The question you asked

TL;DR: No, you cannot return a reference to a variable that is owned by a function. This applies if you created the variable or if you took ownership of the variable as a function argument.

Solutions

Instead of trying to return a reference, return an owned object. String instead of &str, Vec<T> instead of &[T], T instead of &T, etc.

If you took ownership of the variable via an argument, try taking a (mutable) reference instead and then returning a reference of the same lifetime.

In rare cases, you can use unsafe code to return the owned value and a reference to it. This has a number of delicate requirements you must uphold to ensure you don't cause undefined behavior or memory unsafety.

See also:

Deeper answer

fjh is absolutely correct, but I want to comment a bit more deeply and touch on some of the other errors with your code.

Let's start with a smaller example of returning a reference and look at the errors:

fn try_create<'a>() -> &'a String {
    &String::new()
}

Rust 2015

error[E0597]: borrowed value does not live long enough
 --> src/lib.rs:2:6
  |
2 |     &String::new()
  |      ^^^^^^^^^^^^^ temporary value does not live long enough
3 | }
  | - temporary value only lives until here
  |
note: borrowed value must be valid for the lifetime 'a as defined on the function body at 1:15...
 --> src/lib.rs:1:15
  |
1 | fn try_create<'a>() -> &'a String {
  |               ^^

Rust 2018

error[E0515]: cannot return reference to temporary value
 --> src/lib.rs:2:5
  |
2 |     &String::new()
  |     ^-------------
  |     ||
  |     |temporary value created here
  |     returns a reference to data owned by the current function

Is there any way to return a reference from a function without arguments?

Technically "yes", but for what you want, "no".

A reference points to an existing piece of memory. In a function with no arguments, the only things that could be referenced are global constants (which have the lifetime &'static) and local variables. I'll ignore globals for now.

In a language like C or C++, you could actually take a reference to a local variable and return it. However, as soon as the function returns, there's no guarantee that the memory that you are referencing continues to be what you thought it was. It might stay what you expect for a while, but eventually the memory will get reused for something else. As soon as your code looks at the memory and tries to interpret a username as the amount of money left in the user's bank account, problems will arise!

This is what Rust's lifetimes prevent - you aren't allowed to use a reference beyond how long the referred-to value is valid at its current memory location.

See also:

Your actual problem

Look at the documentation for OpenOptions::open:

fn open<P: AsRef<Path>>(&self, path: P) -> Result<File>

It returns a Result<File>, so I don't know how you'd expect to return an OpenOptions or a reference to one. Your function would work if you rewrote it as:

fn trycreate() -> File {
    OpenOptions::new()
        .write(true)
        .open("foo.txt")
        .expect("Couldn't open")
}

This uses Result::expect to panic with a useful error message. Of course, panicking in the guts of your program isn't super useful, so it's recommended to propagate your errors back out:

fn trycreate() -> io::Result<File> {
    OpenOptions::new().write(true).open("foo.txt")
}

Option and Result have lots of nice methods to deal with chained error logic. Here, you can use or_else:

let f = OpenOptions::new().write(true).open("foo.txt");
let mut f = f.or_else(|_| trycreate()).expect("failed at creating");

I'd also return the Result from main. All together, including fjh's suggestions:

use std::{
    fs::OpenOptions,
    io::{self, Write},
};

fn main() -> io::Result<()> {
    let mut f = OpenOptions::new()
        .create(true)
        .write(true)
        .append(true)
        .open("foo.txt")?;

    f.write_all(b"test1\n")?;
    f.write_all(b"test2\n")?;

    Ok(())
}
Wotton answered 20/9, 2015 at 19:22 Comment(18)
Note: in C++, returning a reference to a stack-local variable is Undefined Behavior; if it appears to work, you are just unlucky. In common situations, the compilers should detect the problem and emit a warning.Levan
@MatthieuM. just a warning... how unsafe :-) Although it shows that it's been a few years since I wrote C for a day job, as I never got to see any of those warnings. Nice to see progress in all camps!Wotton
@Shepmaster: You must have been using an old compiler; I'm still stuck on gcc 4.3.2 and I have it! But yes, only a warning. Most C/C++ compilers take a conservative approach: errors are mandated by the Standard, and the rest is done with warnings (with more or less accuracy ...)Levan
@MatthieuM. We certainly had an older compiler, but maybe not that old. It's possible that I never tried to return a local variable as a reference, of course, but I don't know that I could claim to be that good. :-)Wotton
Could you possibly explain how to use the value returned from Result<File> returned from TryCatch and call the write method? I tried your suggestions but when i unwrap the result using let inside = self.returnedresultsfile.as_ref().unwrap(); and then match to a file it has the wrong type and produces the "error: no method named Write found for type &std::fs::File in the current scope". I would really appreciate an explanation of how you can call the write method from the returned value :)Copro
@Copro there is no method called Write. You are looking for write, and you will need to have the trait in scope. Check out the Rust 1.0 / "write a file" section of What's the de-facto way of reading and writing files in Rust 1.x?.Wotton
Just want to point out that in some cases you can actually return a reference from inside a function, e.g. (below). The details on why are a bit complex, but from what I gather, the compiler creates it in the 'static lifetime in this case. (Try it, it'll compile fine) struct foo { val:i32 } and fn try_create<'a>() -> &'a foo { &foo{val:3} }Vesiculate
@JohnDoe is that not covered in the question linked in the answer?Wotton
@Wotton : Certainly ;) I was mostly just putting that there more as a btw to anyone else reading it in case they missed it (or for Google searches - actually mostly cause I came from a Google search too haha).Vesiculate
Btw, if your closures are returning Option<&T> enums, and you want to .clone() the data so it can be moved to the next call/closure, use .cloned() instead! The latter makes a clone of the value inside the Option enum, versus the former, which simply clones the "wrapper" but keeps holding the contents by-reference (causing the error to remain).Homogenous
I can't thank enough for this solution. I was stuck due to similar issue for 3 days. This tip solved it. :)Dalmatian
"In rare cases, you can use unsafe code to return the owned value and a reference to it."—can you? Surely by returning the owned value, it is being moved—and therefore it is UB for there also to be a reference that borrows from it?Gennygeno
@Gennygeno true, it's a bit of lax wording on my part. You aren't really returning a reference to the owned value itself, but to something "within" it. That something has to be guaranteed to not move when the outer value moves. a Box<T> is the canonical example. Moving the Box itself doesn't move the value located on the heap.Wotton
But is it not UB to move (ownership of) a Box<T> whilst there exists a &T that borrows from its heap alloc?Gennygeno
@Gennygeno to my knowledge, no. Moving the Box<T> doesn't change the address of the T, so the &T should continue to be valid.Wotton
Sure, I understand the T shouldn't relocate... I just thought that moving ownership of the owning entity would be a violation Rust's ownership model giving rise to UB. I don't know this for sure though.Gennygeno
To put it another way, what lifetime could the reference have if the owner is able to be moved whilst that borrow remains alive?Gennygeno
@Gennygeno that's not expressible using safe Rust, thus part of the problem. Lifetimes indicate "how long is this value valid at this specific memory address?". However, Rust doesn't have a way to model the fact that the Box wrapper and the heap-allocated value are related but not precisely one-to-one. That's why unsafe is needed — to say "it's OK compiler, I'm going to guarantee it this time".Wotton
E
23

Is there any way to return a reference from a function without arguments?

No (except references to static values, but those aren't helpful here).

However, you might want to look at OpenOptions::create. If you change your first line in main to

let  f = OpenOptions::new().write(true).create(true).open(b"foo.txt");

the file will be created if it does not yet exist, which should solve your original problem.

Exosmosis answered 20/9, 2015 at 19:5 Comment(0)
D
21

You can not return a reference pointing to a local variable. You have two alternatives, either return the value or use a static variable.

Here is why:

References are pointers to memory locations. Once functions are executed, local variables are popped off the execution stack and resources are de-allocated. After that point, any reference to a local variable will be pointing to some useless data. Since it is de-allocated, it is not in our program's possession any more and OS may have already given it to another process and our data may have been overwritten.

For the following example, x is created when the function runs and dropped off when the function completes executing. It is local to the function and lives on this particular function's stack. Function's stack holds local variables.

When run is pop off the execution stack, any reference to x, &x, will be pointing to some garbage data. That is what people call a dangling pointer. The Rust compiler does not allow to use dangling pointers since it is not safe.

fn run() -> &u32 {
    let x: u32 = 42;

    return &x;
} // x is dropped here

fn main() {
    let x = run();
}

So, that is why we can not return a reference to a local variable. We have two options: either return the value or use a static variable.

Returning the value is the best option here. By returning the value, you will be passing the result of your calculation to the caller, in Rust's terms x will be owned by the caller. In our case it is main. So, no problem there.

Since a static variable lives as long as the process runs, its references will be pointing to the same memory location both inside and outside the function. No problem there either.

Note: @navigaid advises using a box, but it does not make sense because you are moving readily available data to heap by boxing it and then returning it. It does not solve the problem, you are still returning the local variable to the caller but using a pointer when accessing it. It adds an unnecessary indirection due to de-referencing hence incurring additional cost. Basically you will be using & for the sake of using it, nothing more.

Droplight answered 11/9, 2019 at 18:31 Comment(4)
return at the end of a block like that is not idiomatic.Wotton
First answer is unnecessarily verbose, second one is not elaborate enough. return is chosen for the emphasis.Droplight
Despite the non-idiomatic return I found this answer to have the clearest explanation.Spectatress
This makes a lot more sense. The compiler error message could maybe a bit more obvious. I had a case where I was passing in an owned object, but was returning only references to the now moved object which is going to get dropped at the end of the function. I'm more used to reading the more common "x.foo() borrows x here but x is dropped at the end of the function"Soupandfish
B
15

This is an elaboration on snnsnn's answer, which briefly explained the problem without being too specific.

Rust doesn't allow return a reference to a variable created in a function. Is there a workaround? Yes, simply put that variable in a Box then return it. Example:

fn run() -> Box<u32> {
    let x: u32 = 42;
    return Box::new(x);
} 

fn main() {
    println!("{}", run());
}

code in rust playground

As a rule of thumb, to avoid similar problems in Rust, return an owned object (Box, Vec, String, ...) instead of reference to a variable:

  • Box<T> instead of &T
  • Vec<T> instead of &[T]
  • String instead of &str

For other types, refer to The Periodic Table of Rust Types to figure out which owned object to use.

Of course, in this example you can simply return the value (T instead of &T or Box<T>)

fn run() -> u32 {
    let x: u32 = 42;
    return x;
} 
Baugh answered 4/10, 2020 at 14:50 Comment(4)
This answer is wrong and misguiding. Why would you box a readily available data then return it. It adds unnecessary indirection and hence cost.Droplight
@Droplight the u32 variable is only for demonstration purpose. I already pointed it out at the end of the answer.Baugh
Even if you box a variable that is suitable for boxing, the example is wrong all the same because references are for passing a reference from outside scope into the function, in other words for borrowing the outer variable, to avoid the clutter of taking in and returning back. Your example is inverts this logic all the way around. It will be confusing for new comers and has no real use case.Droplight
@Droplight Allocating value on the heap by Box makes sense if the Box's lifetime is properly long enough. In navigaid's way, Box's lifetime is too short, so it might not make sense as you say. Box should be dropped when the scope ends. So we must use Box::into_raw or Box:from_raw. See here too: #66197472Zemstvo
H
3

The question was:

Is there any way to return a reference to a variable created in a function?

Answer: Yes, it's possible! See the examples below for proof.

Disclaimer: It's usually better to just return a value instead, but not always...

You have to find a way to extend the lifetime. One way to do this is to create a dummy/default value outside the function, and then give this as a mutable reference (&mut T) to the function. Now the function can fill/replace the value and then return a reference (&T) to that value. You also have to specify the lifetime so that the returned reference get the lifetime of a the value created outside the function ('a).

This works because the function returns a reference to a memory location that was allocated before calling the function, so it's not deleted (or moved) when the function goes out of scope. Also the value isn't owned by the function.

Examples that proves it's possible:

//&mut T -> &T
fn example2<'a>(life: &'a mut Vec<i32>) -> &'a Vec<i32> {
    *life = vec![1, 2, 3, 4];
    life
}

fn test_example2() {
    //Could also use Vec::new()
    let mut life = Vec::default();
    let res = example2(&mut life);
    println!("{:?}", res)
}

fn test2_example2() {
    let life = &mut Vec::default();
    let res = example2(life);
    println!("{:?}", res)
}

//shows real use case
fn get_check_test_slices2<'a>(
    lifetime: &'a mut Vec<usize>,
    limit: usize,
) -> impl Iterator<Item = (&'a [usize], &'a [usize])> + 'a {
    // create a list of primes using a simple primes sieve
    *lifetime = primes1_iter_bitvec(limit).collect::<Vec<_>>();
    // iterate through pairs of sub slices without copying the primes vec
    // slices will be used to check that a complicated sieve is correct
    all_test_check_slices(lifetime)
}

Examples using a wrapper type called LateInit, with helper methods:

fn late_init_example1<'a>(holder: &'a mut LateInit<Vec<i32>>) -> &'a Vec<i32> {
    //create a new vec inside the function
    let v = vec![1, 2, 3];
    //insert it into holder and get reference to value
    let res = holder.init(v);
    //return reference
    res
}

fn late_init_example2<'a>(holder: &'a mut LateInit<Vec<i32>>) -> &'a Vec<i32> {
    //create new vec, insert it into holder, return a reference
    holder.init(vec![1, 2, 3])
}

fn using_late_init_examples() {
    let mut a = LateInit::new();
    println!("example1: {:?}", late_init_example1(&mut a));
    println!("example1: {:?}", late_init_example1(&mut LateInit::new()));
    let b = &mut LateInit::new();
    println!("example2: {:?}", late_init_example2(b));
    //can re-use the same late init
    for i in 0..=4 {
        println!("example2: {:?}", late_init_example2(b));
    }
}

/// A thin wrapper around Option<T>.
/// Enables returning a reference to a value created inside a function.
/// Note: You probably have to add lifetime annotations.
/// 1: This can be achieved by creating a
/// late initialized value on the stack first (Option<T>),
/// 2: then calling a function with a mutable reference
/// to this late initialized value,
/// 3: then initializing the value inside the function,
/// 4: and finally returning a reference to the now initialized value.
/// The returned reference can even be stored in a struct or
/// given to another function, as long as the
/// lifetime annotations are correct and
/// the late init value isn't deleted or moved.
/// TODO: better name?
/// TODO: better documentation.
pub struct LateInit<T> {
    value: Option<T>,
}

impl<T> LateInit<T> {
    /// Creates a new LateInit with None.
    /// Same as default.
    pub fn new() -> Self {
        Self { value: None }
    }

    /// Inserts the value
    /// and returns a mutable reference with 
    /// the same lifetime as the inserted value.
    pub fn init_mut<'a>(&'a mut self, init_value: T) -> &'a mut T {
        self.value.insert(init_value)
    }

    /// Inserts the value
    /// and returns a reference with 
    /// the same lifetime as the inserted value.
    /// Is non mutable init neccesary?
    pub fn init<'a>(&'a mut self, init_value: T) -> &'a T {
        self.value.insert(init_value)
    }
}

impl<T> Default for LateInit<T> {
    /// Creates a new LateInit with None.
    /// Same as new.
    fn default() -> Self {
        Self { value: None }
    }
}

Possible advantages with LateInit wrapper type (compared to just &mut T):

  • The returned reference is always to an initialized value.
  • You don't need a default constructor for T, which sometimes can be messy.
  • The intention of the code might be clearer.

TODO:
Make a crate for LateInit.
Spread the word to all similar questions that might benefit.

Edits:
&mut T examples was added (thank you Chayim Friedman).
Old wrapper type was replaced with "improved" version called LateInit.

Hernandes answered 4/5, 2022 at 19:28 Comment(4)
How is that different from just giving &mut T to the function?Crosscut
I have updated my answer, to answer that question. Often you can just give &mut T to the function.Hernandes
Update: Maybe you can always give &mut T...Hernandes
The answer has been updated with &mut T examples. The old wrapper type has been replaced with an improved version. Some possible advantages of the wrapper type has been added.Hernandes

© 2022 - 2025 — McMap. All rights reserved.