rust serde - flatten path on deserialize

Question

I want to deserialize a pretty deep JSON to Rust struct:

{
  "root": {
    "f1": {
      "f2": {
         "f3": 123
       }
    }
  }
}

When deriving Deserialize, I will have to create too many structs - one for each level for the above JSON :

struct Root {
  f1: Field1
}
struct Field1 {
  f2: Field2
}
struct Field3 {
  f3: Field3
}
// ...

Is there any way to avoid having this number of structs. I didn't find any attribute, which could be helpful with derive. I would like to have something like:

struct Root {
  // some attr?
  f3: u64
}

For sure, it is possible to implementing custom deserialize, but I wonder, whether there is a default way to achieve this.

Yarbrough

I don't think there's any builtin. See this old issue, it references #119 but that's serde(flatten) which AFAIK only works the other way around (though it's possible that at one point it also discussed flattening nested json). – Loram

If you really don't want to create the structs, you can use untyped values and extract the nested values manually. – Riella

Do you want all the data? Or just what's in f3? If the latter then something akin to XPath for JSON sounds more appropriate. – Sy

Got curious, this is how the custom deserializer would look. – Fabri

macros can help reduce the boilerplate :) – Victorvictoria

Not technically a solution per se, but the json_dotpath crate could make life much easier. – Shingle

Answer 1

I thought this was an interesting question/challenge so I wrote a simple proc-macro attribute to do this called serde_flat_path. Here is an example of how it can be used to provide the functionality described in the question:

#[serde_flat_path::flat_path]
#[derive(Serialize, Deserialize)]
struct Root {
    #[flat_path(path = ["f1", "f2", "f3"])]
    f3: u64,
}

The attribute must be placed before deriving Serialize or Deserialize since it will place serde attributes on fields with #[flat_path(...)]. I have attempted to make sure that this attribute plays well with other serde attributes and helper crates as much as possible. It can also be used for more complex types like the one below. To a Serializer or Deserializer it should appear no different from actually writing out all of the structs in the chain. For the specifics, you can check out the crate's readme.

#[serde_flat_path::flat_path]
#[derive(Serialize, Deserialize)]
#[serde(tag = "foobar")]
pub enum Bar {
    Foo {
        #[flat_path(path = ["a", "b", "c", "d", "e"])]
        #[serde(with = "flip_bool")]
        foo: bool,
        #[flat_path(path = ["x", "y", "z"])]
        #[serde(skip_serializing_if = "Option::is_some")]
        x: Option<u64>,
    },
    Bar {
        #[flat_path(path = ["a", "b"])]
        bar: Bar,
    },
    Baz,
    Biz {
        #[serde(with = "add_one")]
        z: f64,
    }
}

Fair warning though as this proc-macro is not perfect. At the moment it can not handle overlapping flattened paths due to the way the macro is expanded. If this is attempted, a compile time error will be emitted unless you use the allow_overlap feature. It also struggles with generics in some cases, but I am looking to improve this.

// This would produce an error since x and y have overlapping paths
#[serde_flat_path::flat_path]
#[derive(Serialize, Deserialize)]
struct Foo {
    z: bool,
    #[flat_path(path = ["a", "b", "x"])]
    x: u64,
    #[flat_path(path = ["a", "c", "y"])]
    y: u64,
}

let foo = Foo { z: true, x: 123, y: 456 };
println!("{}", serde_json::to_string(&foo).unwrap());
// Output:
// {"z":true,"a":{"b":{"x":123}},"a":{"c":{"y":456}}}

Cingulum