Is it safe and defined behavior to transmute between a T and an UnsafeCell<T>?

Question

A recent question was looking for the ability to construct self-referential structures. In discussing possible answers for the question, one potential answer involved using an UnsafeCell for interior mutability and then "discarding" the mutability through a transmute.

Here's a small example of such an idea in action. I'm not deeply interested in the example itself, but it's just enough complication to require a bigger hammer like transmute as opposed to just using UnsafeCell::new and/or UnsafeCell::into_inner:

use std::{
    cell::UnsafeCell, mem, rc::{Rc, Weak},
};

// This is our real type.
struct ReallyImmutable {
    value: i32,
    myself: Weak<ReallyImmutable>,
}

fn initialize() -> Rc<ReallyImmutable> {
    // This mirrors ReallyImmutable but we use `UnsafeCell` 
    // to perform some initial interior mutation.
    struct NotReallyImmutable {
        value: i32,
        myself: Weak<UnsafeCell<NotReallyImmutable>>,
    }

    let initial = NotReallyImmutable {
        value: 42,
        myself: Weak::new(),
    };

    // Without interior mutability, we couldn't update the `myself` field
    // after we've created the `Rc`.
    let second = Rc::new(UnsafeCell::new(initial));

    // Tie the recursive knot 
    let new_myself = Rc::downgrade(&second);

    unsafe {
        // Should be safe as there can be no other accesses to this field
        (&mut *second.get()).myself = new_myself;

        // No one outside of this function needs the interior mutability
        // TODO: Is this call safe?
        mem::transmute(second)
    }
}

fn main() {
    let v = initialize();
    println!("{} -> {:?}", v.value, v.myself.upgrade().map(|v| v.value))
}

This code appears to print out what I'd expect, but that doesn't mean that it's safe or using defined semantics.

Is transmuting from a UnsafeCell<T> to a T memory safe? Does it invoke undefined behavior? What about transmuting in the opposite direction, from a T to an UnsafeCell<T>?

Ximenez

I think, though I'm not sure, that Rust's aliasing rules would make this undefined behavior. If the compiler doesn't see an UnsafeCell, it's going to assume no aliasing can happen. – Elery

@SebastianRedl in the block where the mutation happens there still is an UnsafeCell, so the knot tying part is perfectly fine. – Everyplace

afaik: even if ReallyImmutable and NotReallyImmutable were #[repr(C)] neither UnsafeCell nor Rc have a defined reliable layout. – Everyplace

also: transmuting means that the destructor will be called with a different type than the data was constructed with, same for memory allocation. Not sure whether this is explicitly said to be UB, but I don't think it is guaranteed to be safe in any way, even if the data layouts match perfectly. – Everyplace

@Everyplace "neither UnsafeCell nor Rc have a defined reliable layout." — UnsafeCell takes up zero (extra) space. I don't think it is feasible that a future version of Rust would change that. – Ontario

I would definitely be uncomfortable going from T to UnsafeCell<T>... even in the absence of aliasing I could see the compiler making assumptions there. – Manipur

@PeterHall What currently works, what probably keeps working and what is guaranteed to keep working are separate things; afaict OP wants to know mainly about the last part. It would be nice though if we could rely on newtype wrappers (or plain 1-element tuples) to use the inner layout. – Everyplace

Seems like #[repr(transparent)] might specify this. PR #1758 and tracking issue – Scotney

@Everyplace Those arguments are equally valid for the more general question: "is it safe to transmute between X and Y?". You can never be certain that layout won't change in the future, or that someone won't decide to implement Drop, so you should never use transmute! In this particular case, the Rust core developers are constrained - changing how UnsafeCell affects its data's layout would be a breaking change, and so would implementing Drop for UnsafeCell. If these two risks are the only concerns then this feels very safe. – Ontario

@PeterHall I don't see why changing the UnsafeCell data layout would be a breaking change; they made no promises about the layout afaict. Even if the newtype layout would be defined, they'd still need to promise not to add more fields. But I'd worry more about rust eliminating fields from a type that aren't used or optimizing the field order for different access patterns. – Everyplace

Not an answer to the question, but imho a proper fix for the presented usecase: TieWeak on playground, a weak pointer wrapper to tie the knot. – Everyplace

@Everyplace you may wish to promote that to a full answer for the linked question (a far better place to save more complicated code like that). – Ximenez

Answer 1

Disclaimer: The rules for these kinds of things are not (yet) set in stone. So, there is no definitive answer yet. I'm going to make some guesses based on (a) what kinds of compiler transformations LLVM does/we will eventually want to do, and (b) what kind of models I have in my head that would define the answer to this.

Also, I see two parts to this: The data layout perspective, and the aliasing perspective. The layout issue is that NotReallyImmutable could, in principle, have a totally different layout than ReallyImmutable. I don't know much about data layout, but with UnsafeCell becoming repr(transparent) and that being the only difference between the two types, I think the intent is for this to work. You are, however, relying on repr(transparent) being "structural" in the sense that it should allow you to replace things in larger types, which I am not sure has been written down explicitly anywhere. Sounds like a proposal for a follow-up RFC that extends the repr(transparent) guarantees appropriately?

As far as aliasing is concerned, the issue is breaking the rules around &T. I'd say that, as long as you never have a live &T around anywhere when writing through the &UnsafeCell<T>, you are good -- but I don't think we can guarantee that quite yet. Let's look in more detail.

Compiler perspective

The relevant optimizations here are the ones that exploit &T being read-only. So if you reordered the last two lines (transmute and the assignment), that code would likely be UB as we may want the compiler to be able to "pre-fetch" the value behind the shared reference and re-use that value later (i.e. after inlining this).

But in your code, we would only emit "read-only" annotations (noalias in LLVM) after the transmute comes back, and the data is indeed read-only starting there. So, this should be good.

Memory models

The "most aggressive" of my memory models essentially asserts that all values are always valid, and I think even that model should be fine with your code. &UnsafeCell is a special case in that model where validity just stops, and nothing is said about what lives behind this reference. The moment the transmute returns, we grab the memory it points to and make it all read-only, and even if we did that "recursively" through the Rc (which my model doesn't, but only because I couldn't figure out a good way to make it do so) you'd be fine as you don't mutate any more after the transmute. (As you may have noticed, this is the same restriction as in the compiler perspective. The point of these models is to allow compiler optimizations, after all. ;)

(As a side-note, I really wish miri was in better shape right now. Seems I have to try and get validation to work again in there, because then I could tell you to just run your code in miri and it'd tell you if that version of my model is okay with what you are doing :D )

I am thinking about other models currently that only check things "on access", but haven't worked out the UnsafeCell story for that model yet. What this example shows is that the model may have to contain ways for a "phase transition" of memory first being UnsafeCell, but later having normal sharing with read-only guarantees. Thanks for bringing this up, that will make for some nice examples to think about!

So, I think I can say that (at least from my side) there is the intent to allow this kind of code, and doing so does not seem to prevent any optimizations. Whether we'll actually manage to find a model that everybody can agree with and that still allows this, I cannot predict.

The opposite direction: T -> UnsafeCell<T>

Now, this is more interesting. The problem is that, as I said above, you must not have a &T live when writing through an UnsafeCell<T>. But what does "live" mean here? That's a hard question! In some of my models, this could be as weak as "a reference of that type exists somewhere and the lifetime is still active", i.e., it could have nothing to do with whether the reference is actually used. (That's useful because it lets us do more optimizations, like moving a load out of a loop even if we cannot prove that the loop ever runs -- which would introduce a use of an otherwise unused reference.) And since &T is Copy, you cannot even really get rid of such a reference either. So, if you have x: &T, then after let y: &UnsafeCell<T> = transmute(x), the old x is still around and its lifetime still active, so writing through y could well be UB.

I think you'd have to somehow restrict the aliasing that &T allows, very carefully making sure that nobody still holds such a reference. I'm not going to say "this is impossible" because people keep surprising me (especially in this community ;) but TBH I cannot think of a way to make this work. I'd be curious if you have an example though where you think this is reasonable.

Equilibrate