How to unit-test a deserialization function used in serde(deserialize_with)?

Question

I have a struct which implements Deserialize and uses the serde(deserialize_with) on a field:

#[derive(Debug, Deserialize)]
struct Record {
    name: String,
    #[serde(deserialize_with = "deserialize_numeric_bool")]
    is_active: bool,
}

The implementation of deserialize_numeric_bool deserializes a string "0" or "1" to the corresponding boolean value:

pub fn deserialize_numeric_bool<'de, D>(deserializer: D) -> Result<bool, D::Error>
    where D: Deserializer<'de>
{
    struct NumericBoolVisitor;

    impl<'de> Visitor<'de> for NumericBoolVisitor {
        type Value = bool;

        fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
            formatter.write_str("either 0 or 1")
        }

        fn visit_u64<E>(self, value: u64) -> Result<bool, E>
            where E: DeserializeError
        {
            match value {
                0 => Ok(false),
                1 => Ok(true),
                _ => Err(E::custom(format!("invalid bool: {}", value))),
            }
        }
    }

    deserializer.deserialize_u64(NumericBoolVisitor)
}

(I appreciate comments about code improvements)

I'd like to write unit tests for deserialization functions like deserialize_numeric_bool. Of course, my friendly search box revealed the serde_test crate and a documentation page about unit-testing. But these resources couldn't help me in my case, as the crate tests a structure directly implementing Deserialize.

One idea I had was to create a newtype which only contains the output of my deserialize functions and test it with it. But this looks like a unnecessary indirection to me.

#[derive(Deserialize)]
NumericBool {
    #[serde(deserialize_with = "deserialize_numeric_bool")]
    value: bool
};

How do I write idiomatic tests for it?

Shuffleboard

I don't know enough about idiomatic Serde to properly answer this, but you can instantiate a Deserializer implementation via a crate like serde-json, which would then give you something you could pass in to the function. If using a JSON crate in your tests feels weird, you could use something like serde-value. – Internationalism

Did you find a more direct solution? – Fennie

Answer 1

I've come across this question several times while trying to solve a similar problem recently. For future readers, pixunil's answer is nice, straightforward, and works well. However, I'd like to provide a solution using serde_test as the unit testing documentation mentions.

I researched how serde_test is used across a few crates that I found via its reverse dependencies on lib.rs. Several of them define small structs or enums for testing deserialization or serialization as you mentioned in your original post. I suppose doing so is idiomatic when testing would be too verbose otherwise.

Here's a few examples; this is a non-exhaustive list:

• Example from time
• Another example from time
• Example from slab (tokio)
• Example from bitcoin_hashes
• Example from uuid
• Example from euclid

Anyway, let's say I have a function to deserialize a bool from a u8 and another function that serializes a bool to a u8.

use serde::{
    de::{Error as DeError, Unexpected},
    Deserialize, Deserializer, Serialize, Serializer,
};

fn bool_from_int<'de, D>(deserializer: D) -> Result<bool, D::Error>
where
    D: Deserializer<'de>,
{
    match u8::deserialize(deserializer)? {
        0 => Ok(false),
        1 => Ok(true),
        wrong => Err(DeError::invalid_value(
            Unexpected::Unsigned(wrong.into()),
            &"zero or one",
        )),
    }
}

#[inline]
fn bool_to_int<S>(a_bool: &bool, serializer: S) -> Result<S::Ok, S::Error>
where
    S: Serializer,
{
    if *a_bool {
        serializer.serialize_u8(1)
    } else {
        serializer.serialize_u8(0)
    }
}

I can test those functions by defining a struct in my test module. This allows constraining the tests to those functions specifically instead of ser/deserializing a larger object.

#[cfg(test)]
mod tests {
    use super::{bool_from_int, bool_to_int};
    use serde::{Deserialize, Serialize};
    use serde_test::{assert_de_tokens_error, assert_tokens, Token};

    #[derive(Debug, PartialEq, Deserialize, Serialize)]
    #[serde(transparent)]
    struct BoolTest {
        #[serde(deserialize_with = "bool_from_int", serialize_with = "bool_to_int")]
        a_bool: bool,
    }

    const TEST_TRUE: BoolTest = BoolTest { a_bool: true };
    const TEST_FALSE: BoolTest = BoolTest { a_bool: false };

    #[test]
    fn test_true() {
        assert_tokens(&TEST_TRUE, &[Token::U8(1)])
    }

    #[test]
    fn test_false() {
        assert_tokens(&TEST_FALSE, &[Token::U8(0)])
    }

    #[test]
    fn test_de_error() {
        assert_de_tokens_error::<BoolTest>(
            &[Token::U8(14)],
            "invalid value: integer `14`, expected zero or one",
        )
    }
}

BoolTest is within the tests module which is gated by #[cfg(test)] as per usual. This means that BoolTest is only compiled for tests rather than adding bloat. I'm not a Rust expert, but I think this is a good alternative if a programmer wishes to use serde_test as a harness.

Saleable

Answer 2

My current solution uses only structures already provided by serde. In my use case, I only wanted to test that a given string will deserialize successfully into a bool or has a certain error. The serde::de::value provides simple deserializers for fundamental data types, for example U64Deserializer which holds a u64. It also has an Error struct which provides a minimal representation for the Error traits – ready to be used for mocking errors.

My tests look currently like that: I mock the input with a deserializer and pass it to my function under test. I like that I don't need an indirection there and that I have no additional dependencies. It is not as nice as the assert_tokens* provided serde_test, as it needs the error struct and feels less polished. But for my case, where only a single value is deserialized, it fulfills my needs.

use serde::de::IntoDeserializer;
use serde::de::value::{U64Deserializer, StrDeserializer, Error as ValueError};

#[test]
fn test_numeric_true() {
    let deserializer: U64Deserializer<ValueError> = 1u64.into_deserializer();
    assert_eq!(numeric_bool(deserializer), Ok(true));
}

#[test]
fn test_numeric_false() {
    let deserializer: U64Deserializer<ValueError> = 0u64.into_deserializer();
    assert_eq!(numeric_bool(deserializer), Ok(false));
}

#[test]
fn test_numeric_invalid_number() {
    let deserializer: U64Deserializer<ValueError> = 2u64.into_deserializer();
    let error = numeric_bool(deserializer).unwrap_err();
    assert_eq!(error.description(), "invalid bool: 2");
}

#[test]
fn test_numeric_empty() {
    let deserializer: StrDeserializer<ValueError> = "".into_deserializer();
    let error = numeric_bool(deserializer).unwrap_err();
    assert_eq!(error.description(), "invalid type: string \"\", expected either 0 or 1");
}

I hope that it helps other folks too or inspire other people to find a more polished version.

Shuffleboard

Answer 3

serde_test can't be used to test your function directly because serde_test does not expose the Deserializer it uses internally. Consequentially, serde_test can only be used to test Serialize and Deserialize implementations, and can't be used for testing functions meant to be used with deserialize_with.

However, you can do this with the serde_assert crate (disclaimer: I wrote serde_assert). serde_assert does expose a Deserializer to be used directly in tests, which makes it very simple to directly test your function:

use serde_assert::{de, Deserializer, Token, Tokens};

#[test]
fn test_numeric_true() {
    let mut deserializer = Deserializer::builder()
        .tokens(Tokens(vec![Token::U64(1)]))
        .build();

    assert_eq!(deserialize_numeric_bool(&mut deserializer), Ok(true),);
}

#[test]
fn test_numeric_false() {
    let mut deserializer = Deserializer::builder()
        .tokens(Tokens(vec![Token::U64(0)]))
        .build();

    assert_eq!(deserialize_numeric_bool(&mut deserializer), Ok(false),);
}

#[test]
fn test_numeric_invalid_value() {
    let mut deserializer = Deserializer::builder()
        .tokens(Tokens(vec![Token::U64(2)]))
        .build();

    assert_eq!(
        deserialize_numeric_bool(&mut deserializer),
        Err(de::Error::Custom("invalid bool: 2".to_owned())),
    );
}

#[test]
fn test_numeric_invalid_type() {
    let mut deserializer = Deserializer::builder()
        .tokens(Tokens(vec![Token::Str("".to_owned())]))
        .build();

    assert_eq!(
        deserialize_numeric_bool(&mut deserializer),
        Err(de::Error::InvalidType(
            "string \"\"".to_owned(),
            "either 0 or 1".to_owned()
        )),
    );
}

Cade