How to distinguish long and double-values when deserializing with moshi?

Question

My goal is to synchronize abitrary rows of data by using the JSON-Format. As I do not know the exact scheme for the rows (it is a general sync method), my datamodel apparently has to rely on "Object". So in Java I will have an array of Map<String,Object> to be synchronized with the server.

Translating such a row into JSON would give something like

{{"string":"stringvalue"},{"double1":1234.567},{"double2":1234.0},{"long":1234}}

so far, so good - no problem with moshi - everything works as expected.

Now the Problem: When I try to deserialize that JSON with moshi, I get back a double-value for the "long" member. Moshi converts all numbers to Doubles. But unfortunately not all numbers can be safely converted to doubles. Very big integers (aka longs) have a problem with the limited precision of doubles. And rounding-effects also might exist.

I opened an issue with moshi, but unfortunately that was closed. Maybe I wasn't clear enough. (Issue 192)

JSON has no concept of integer - only numbers and Strings. But the subtle detail from "double2" from the example above might lead to a solution for my problem:
If a number does not contain a decimal-point, it is an integer and should be converted to a long.

As longs can not be losslessly converted to doubles, I need a method to intercept the parser before the value is converted to double. But how to do that?

Moshi has this handy concept of JsonAdapters - but unfortunately I currently do not see how I can use them in this case:
The input-type of such an JsonAdapter would have to be Object because I can not cast a generated double to long. So I have to intercept the parser before he converts any value.
But how to return more than one type from there? (I would have to return String, Double or Long from there - or if I can limit the inputs to only numbers I would at least have to return Longs or Doubles.)

(My backend is written in PHP and automatically produces the desired output: Integers are written without a decimal-point.)

Badman

The JSON file format only stores doubles, it can't store ints. This is because JavaScript does not have ints, only doubles, and JSON means JavaScript Object Notation. – Venetic

Answer 1

I am afraid it's not possible without changing Moshi source code. The JSON string source passes through JsonReader which converts all numbers to double. I could not find a way how to alter this behavior since all subclasses are package-protected.

Crocodilian

no, JsonReader has nextSource(). – Loadstar

Answer 2

using JsonReader.nextSource() to read number written in json source.

package jp.juggler.mastodonInboxFilter

import com.squareup.moshi.JsonAdapter
import com.squareup.moshi.JsonEncodingException
import com.squareup.moshi.JsonReader
import com.squareup.moshi.JsonWriter

/**
 * this adapter encode/decode Number.
 *
 * - Number is interface, actual classes are:
 *   BigDecimal, Double, Float, Long, Int, Short, Byte ,etc.
 */
class NumberAdapter : JsonAdapter<Number>() {
    override fun toJson(writer: JsonWriter, value: Number?) {
        when (value) {
            null -> writer.nullValue()
            else -> writer.value(value)
        }
    }

    override fun fromJson(reader: JsonReader): Number? =
        when (val token = reader.peek()) {
            JsonReader.Token.NULL -> reader.nextNull()
            JsonReader.Token.STRING -> reader.nextString().parseNumber()
            JsonReader.Token.NUMBER -> reader.nextSource().use { it.readUtf8().trim() }.parseNumber()
            else -> throw JsonEncodingException("NumberAdapter.fromJson: unexpected token $token. path=${reader.path}")
        }

    private fun String.parseNumber(): Number =
        toIntOrNull()
            ?: toLongOrNull()
            ?: toDoubleOrNull()?.takeIf { it.isFinite() }
            ?: toBigDecimal()
}

val lazyMoshi :Moshi by lazy {
    Moshi.Builder()
        .add(Number::class.java,NumberAdapter())
        .build()
}

@OptIn(ExperimentalStdlibApi::class)
inline fun <reified T : Any> T.encodeMoshi() =
    lazyMoshi.adapter<T>().toJson(this)

@OptIn(ExperimentalStdlibApi::class)
inline fun <reified T : Any> String.decodeMoshi(): T? =
    lazyMoshi.adapter<T>().fromJson(this)

test code

@JsonClass(generateAdapter = true)
class Entity1(
    val num1: Number?,
)

class TestMoshiNumber {
    @Test
    fun testMoshiNumber() {
        fun t(src: String, expected: Number?, expectedJson: String? = null) {
            val entity: Entity1 = src.decodeMoshi()!!
            assertEquals(
                expected,
                entity.num1,
                "num1 value. src=$src",
            )
            assertEquals(
                expected?.javaClass,
                entity.num1?.javaClass,
                "num1 class. src=$src",
            )
            val encoded = entity.encodeMoshi()
            assertEquals(
                expectedJson ?: src,
                encoded,
                "json encoded. src=$src",
            )
        }

        t("""{"num1":null}""", null, expectedJson = "{}")

        t("""{"num1":1}""", 1)
        t("""{"num1":1.1}""", 1.1)

        // int range
        t("""{"num1":2147483647}""", Int.MAX_VALUE)
        t("""{"num1":-2147483648}""", Int.MIN_VALUE)

        // long range
        t("""{"num1":9223372036854775807}""", Long.MAX_VALUE)
        t("""{"num1":-9223372036854775808}""", Long.MIN_VALUE)

        // out of long range
        t("""{"num1":9223372036854775808}""", 9.223372036854776E18, expectedJson = """{"num1":9.223372036854776E18}""")
        t(
            """{"num1":-9223372036854775809}""",
            -9.223372036854776E18,
            expectedJson = """{"num1":-9.223372036854776E18}"""
        )

        // double range
        t("""{"num1":1.7976931348623157E308}""", Double.MAX_VALUE)
        t("""{"num1":4.9E-324}""", Double.MIN_VALUE)

        // too small value is decoded as 0.0
        t("""{"num1":4.9E-325}""", 0.0, expectedJson = """{"num1":0.0}""")

        // BigDecimal
        t(
            """{"num1":1.7976931348623157E309}""",
            BigDecimal("1.7976931348623157E309"),
            expectedJson = """{"num1":1.7976931348623157E+309}""",
        )
        t(
            """{"num1":-1.7976931348623157E309}""",
            BigDecimal("-1.7976931348623157E+309"),
            expectedJson = """{"num1":-1.7976931348623157E+309}""",
        )
    }
}

Loadstar