Is there is a simpler way to convert a type upon deserialization?

Question

Using serde_json, I have JSON objects with Strings that I need to convert to floats. I've stumbled upon a custom deserializer solution, but it seems like a hack. Here is a working playground example of the code below.

#[macro_use]
extern crate serde_derive;
extern crate serde;
extern crate serde_json;

use serde_json::Error;
use serde::de::{Deserialize, DeserializeOwned, Deserializer};

#[derive(Serialize, Deserialize)]
struct Example {
    #[serde(deserialize_with = "coercible")]
    first: f64,
    second: f64,
}

fn coercible<'de, T, D>(deserializer: D) -> Result<T, D::Error>
where
    T: DeserializeOwned,
    D: Deserializer<'de>,
{
    use serde::de::Error;
    let j = String::deserialize(deserializer)?;
    serde_json::from_str(&j).map_err(Error::custom)
}

fn typed_example() -> Result<(), Error> {
    let data = r#"["3.141",1.618]"#;
    let e: Example = serde_json::from_str(data)?;
    println!("{} {}", e.first * 2.0, e.second * 2.0);
    Ok(())
}

fn main() {
    typed_example().unwrap();
}

The above code compiles and runs as you would expect, outputting two floats.

I'm trying to learn how the deserializer solution works, but I'd like to know if I'm headed in the right direction or if there is a better way to do this.

Lubra

I'm trying to understand serde since one year ago... – Decane

I'm not sure this question is a great fit for SO. You've got a working solution and you're asking for open ended advice. I'd say that will attract the sorts of responses (if any?) that SO tries to avoid. This might be a better fit for CodeReview? – Oxygen

If you are just asking for a review, you should probably send this to Code Review instead. Your approach seems to work, so there is no concrete problem that we can answer to. – Ailurophobe

Thank you for the suggestions. My understanding of the code is improving, so I may be able to reformulate the question into something more directly answerable. – Lubra

Answer 1

The lightweight crate serde-this-or-that makes it a one-liner:

use serde_this_or_that::as_f64;

#[derive(Serialize, Deserialize)]
struct Example {
    #[serde(deserialize_with = "as_f64")]
    first: f64,
    second: f64,
}

Beware that serde-this-or-that resolves edge cases in potentially surprising ways. For example and empty string is coerced to 0.0 when deserializing as_f64.

A more concise custom (and hence customizable) version (adapted from Reddit comment) would look like this:

struct Example {
    #[serde(deserialize_with = "de_f64_or_string_as_f64")]
    first: f64,
    second: f64,
}

fn de_f64_or_string_as_f64<'de, D: Deserializer<'de>>(deserializer: D) -> Result<f64, D::Error> {
  Ok(match Value::deserialize(deserializer)? {
    Value::String(s) => s.parse().map_err(de::Error::custom)?,
    Value::Number(num) => num.as_f64().ok_or_else(|| de::Error::custom("Invalid number"))?,
    _ => return Err(de::Error::custom("wrong type")),
  })
}

It may be safer to return None in an Option<f64> when string parsing fails. The above errors on things like "first": "unparseable".

The below returns None instead:

struct Example {
    #[serde(default, deserialize_with = "de_f64_or_string_as_f64")]
    first: Option<f64>,
    second: Option<f64>,
}

fn de_f64_or_string_as_f64<'de, D: Deserializer<'de>>(deserializer: D) -> Result<Option<f64>, D::Error> {
  Ok(match Value::deserialize(deserializer)? {
    Value::String(s) => s.parse().ok(),
    Value::Number(num) => num.as_f64(),
    _ => None,
  })
}

Note, I derived default so that errors in parsing result in None.

Vinni

Answer 2

Using coercible worked kind-of by accident. With it, the input "3.141" was stripped of its ""s, so I had 3.141 being fed into serde_json::from_str(&j), which appropriately returned a float. This accidental solution broke easily and confusingly when, e.g., the input JSON contained unexpected values.

I read the Serde docs (a great learning exercise) and came up with the appropriate way to convert a string to a f64 upon deserialization of JSON (working playground here):

#[macro_use]
extern crate serde_derive;
extern crate serde;
extern crate serde_json;

use std::fmt;
use serde_json::Error;
use serde::de::{self, Deserializer, Unexpected, Visitor};

#[derive(Serialize, Deserialize)]
struct Example {
    #[serde(deserialize_with = "string_as_f64")]
    first: f64,
    second: f64,
}

fn string_as_f64<'de, D>(deserializer: D) -> Result<f64, D::Error>
where
    D: Deserializer<'de>,
{
    deserializer.deserialize_f64(F64Visitor)
}

struct F64Visitor;
impl<'de> Visitor<'de> for F64Visitor {
    type Value = f64;
    fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
        formatter.write_str("a string representation of a f64")
    }
    fn visit_str<E>(self, value: &str) -> Result<f64, E>
    where
        E: de::Error,
    {
        value.parse::<f64>().map_err(|_err| {
            E::invalid_value(Unexpected::Str(value), &"a string representation of a f64")
        })
    }
}

fn typed_example() -> Result<(), Error> {
    let data = r#"["3.141",1.618]"#;
    let e: Example = serde_json::from_str(data)?;
    println!("{} {}", e.first * 2.0, e.second * 2.0);
    Ok(())
}

fn main() {
    typed_example().unwrap();
}

Kudos to the Serde devs, because although the Serde documentation seemed totally obtuse to my eyes, it actually proved to be very helpful and comprehensible. I just had to start from the top and read through slowly.

Lubra

The link to your playground doesn't work: thread 'main' panicked at 'called Result::unwrap() on an Err value: ErrorImpl { code: Message("invalid type: string \"3.141\", expected a string representation of a f64"), line: 1, column: 8 }', /checkout/src/libcore/result.rs:916:5 note: Run with RUST_BACKTRACE=1 for a backtrace. – Rosebay

There is one minor bug in the code: string_as_f64 should actually call deserializer.deserialize_str(F64Visitor), not deserializer.deserialize_f64(F64Visitor) – Willemstad

Rust has changed a lot since 2017, now there are conciser ways of doing this, luckily! – Vinni