How to borrow a field for serialization but create it during deserialization?

Asked 10/10, 2018 at 5:15 Answered 13/12, 2021 at 21:26

I have a struct like this:

#[derive(Serialize, Deserialize)]
struct Thing {
    pub small_header: Header,
    pub big_body: Body,
}

I want to serialize this Thing to send over the network. I already have a Body available but I can't move it (imagine I am doing something with it, and every now and then I receive a command to temporarily stop what I'm doing and send over whatever data I have now) and I can't copy it (it's too big, possibly hundreds of megabytes).

So I'd like Serde to just borrow the one I have for serializing it, since it shouldn't need to be moved into a struct for that. If I rewrite Thing to take a reference, I obviously can't derive Deserialize!

The workaround I've been using is just using an Arc<Body> in my code, so that I can work with the body in my normal logic and when I need to serialize it, I can just do a cheap clone and put the Arc<Body> into the struct for serialization. During deserialization Serde will make a new Arc with a refcount of 1.

This still involves scattering Arc all over my code, which isn't great, not to mention the unnecessary (though minor) runtime cost. What is the correct solution for this use case?

The funny thing is that if I didn't have to send a header, then this would be a non-issue since I could serialize by reference and deserialize by value, but the presence of a header makes this impossible. I feel like I'm missing something about how Serde borrows data here...

Swinge answered 10/10, 2018 at 5:15 Comment(1)

serde.rs/impl-deserializer.html, it's not clear what you asking please include curent minimal reproducible example that copy. – Thury 10/10, 2018 at 5:43

You could use a Cow, which will be Cow::Borrowed when serializing and will deserialize as Cow::Owned when deserializing.

use std::borrow::Cow;

#[derive(Serialize, Deserialize)]
struct Thing<'a> {
    small_header: Header,
    big_body: Cow<'a, Body>,
}

Alternatively, you could serialize and deserialize as two independent data structures.

#[derive(Serialize)]
struct SerializeThing<'a> {
    small_header: Header,
    big_body: &'a Body,
}

#[derive(Deserialize)]
struct DeserializeThing {
    small_header: Header,
    big_body: Body,
}

Station answered 10/10, 2018 at 6:2 Comment(0)

-1

Keep in mind, that serde always allocate (as default) (https://github.com/serde-rs/serde/issues/1852).

To be sure that you are really borrowing, use the #[serde(borrow)].

As example:

#[derive(Serialize, Deserialize)]
struct Thing<'a> {
    small_header: Header,
    #[serde(borrow)]
    big_body: Cow<'a, Body>,
}

Demineralize answered 13/12, 2021 at 21:26 Comment(0)

Recommended topics

Hot tags