Deserialize a reqwest::Reponse to JSON but print response text on error

Question

I'm decoding a reqwest::Response to JSON. Usually that works fine, but in some rare cases the remote server returns a response that doesn't fit my struct that I'm using for deserialization. In those cases I'd like to print out the original response text for further debugging.

However, I'm having trouble to both do the JSON deserialization and printing out the response body. What I'd like to do is

#[derive(serde::Deserialize)]
struct MyData {
    // ...
}

async fn get_json(url: &str) -> Result<MyData, reqwest::Error> {
    let response = reqwest::get(url).await?;
    let text = response.text().await?;
    response
        .json::<MyData>().await
        .map_err(|err| {
            println!(
                "Could not decode response from {}: {}", url, text
            );
            err
        })
}

But that doesn't work because response.text takes self, so I cannot re-use response for response.json.

Based on code from another answer (also recommended in this answer) I've found this approach:

let response = reqwest::get(url).await?;
let text = response.text().await?;
serde_json::from_str(&text).map_err(...)

However, serde_json::from_str returns a Result<_, serde_json::Error>, so this approach would complicate my error handling because the calls before all return Result<_, reqwest::Error>. I'd prefer my function to also return the latter, not some custom error wrapper.

What is an idiomatic way of achieving my goal?

Button

Regardless of what's idiomatic, what you'd prefer isn't possible: reqwest doesn't have a way to customize how json parsing is handled and it doesn't allow creating Errors outside the crate. So you have to use a different error type. Perhaps one of the many error-handling crates (anyhow, thiserror, snafu) can help ease that burden though. – Meant

Answer 1

I don't think you'll get around the .text() and then serde_json::from_str(&text) part.

But you can avoid having to define custom error types by using the anyhow library (as kmdreko suggested in his/her comment). Simply use Result<T, anyhow::Error>, or equivalently anyhow::Result<T>, as the return type of any fallible function.

For example (copied from my codebase):

use anyhow::{Context, Result};
use reqwest::Client;
use serde::de::DeserializeOwned;
use serde::Deserialize;
use url::Url;

#[derive(Debug)]
pub struct UrlPath<'a>(&'a str);

pub async fn get<T: DeserializeOwned>(
    client: &Client,
    backend_url: &Url,
    path: &UrlPath<'_>,
) -> Result<T> {
    let response = client
        .get(backend_url.join(path.0).with_context(|| {
            format!(
                "Unable to join base URL \"{}\" with path \"{:?}\"",
                backend_url, path
            )
        })?)
        .send()
        .await?;

    let contents = response.text().await?;
    serde_json::from_str(&contents).with_context(|| {
        format!("Unable to deserialise response. Body was: \"{}\"", contents)
    })
}

This will give you a nice error like this:

Error: Unable to fetch Source Kinds

Caused by:
    0: Unable to deserialise response. Body was: "{"detail":"Authentication credentials were not provided."}"
    1: invalid type: map, expected a sequence at line 1 column 0

with_context() from anyhow allows you to provide custom context to errors. And anywhow will convert most errors such that they "just work".

But only use this for application code, where you don't need to tell the caller what precisely went wrong.

More info: https://docs.rs/anyhow/latest/anyhow/index.html

Climb