How can I JSON encode BigDecimal and BigInteger in Kotlinx Serialization without losing precision?

Question

I'm using Kotlin/JVM 1.8.0 and Kotlinx Serialization 1.4.1.

I need to encode a java.math.BigDecimal and java.math.BigInteger to JSON.

I'm using BigDecimal and BigInteger because the values I want to encode can be larger than a Double can hold, and also I want to avoid errors with floating-point precision. I don't want to encode the numbers as strings because JSON is read by other programs, so it needs to be correct.

The JSON spec places no restriction on the length of numbers, so it should be possible.

When I try and use BigDecimal and BigInteger directly, I get an error

import java.math.*
import kotlinx.serialization.*
import kotlinx.serialization.json.*

@Serializable
data class FooNumbers(
  val decimal: BigDecimal,
  val integer: BigInteger,
)

Serializer has not been found for type 'BigDecimal'. To use context serializer as fallback, explicitly annotate type or property with @Contextual
Serializer has not been found for type 'BigInteger'. To use context serializer as fallback, explicitly annotate type or property with @Contextual

I tried creating custom serializers for BigDecimal and BigInteger (and typealiases for convenience), but because these use toDouble() and toLong() they lose precision!

typealias BigDecimalJson = @Serializable(with = BigDecimalSerializer::class) BigDecimal

private object BigDecimalSerializer : KSerializer<BigDecimal> {

  override val descriptor = PrimitiveSerialDescriptor("java.math.BigDecimal", PrimitiveKind.DOUBLE)

  override fun deserialize(decoder: Decoder): BigDecimal =
    decoder.decodeDouble().toBigDecimal()

  override fun serialize(encoder: Encoder, value: BigDecimal) =
    encoder.encodeDouble(value.toDouble())
}

typealias BigIntegerJson = @Serializable(with = BigIntegerSerializer::class) BigInteger

private object BigIntegerSerializer : KSerializer<BigInteger> {

  override val descriptor = PrimitiveSerialDescriptor("java.math.BigInteger", PrimitiveKind.LONG)

  override fun deserialize(decoder: Decoder): BigInteger =
    decoder.decodeLong().toBigInteger()

  override fun serialize(encoder: Encoder, value: BigInteger) =
    encoder.encodeLong(value.toLong())
}

When I encode and decode an example instance, a different result is returned.

import java.math.*
import kotlinx.serialization.*
import kotlinx.serialization.json.Json

@Serializable
data class FooNumbers(
  val decimal: BigDecimalJson,
  val integer: BigIntegerJson,
)

fun main() {
  val fooDecimal = BigDecimal("0.1234567890123456789012345678901234567890")
  val fooInteger = BigInteger("9876543210987654321098765432109876543210")

  val fooNumbers = FooNumbers(fooDecimal, fooInteger)
  println("$fooNumbers")

  val encodedNumbers = Json.encodeToString(fooNumbers)
  println(encodedNumbers)

  val decodedFooNumbers = Json.decodeFromString<FooNumbers>(encodedNumbers)
  println("$decodedFooNumbers")

  require(decodedFooNumbers == fooNumbers)
}

The require(...) fails:

FooNumbers(decimal=0.1234567890123456789012345678901234567890, integer=9876543210987654321098765432109876543210)
{"decimal":0.12345678901234568,"integer":1086983617567424234}
FooNumbers(decimal=0.12345678901234568, integer=1086983617567424234)
Exception in thread "main" java.lang.IllegalArgumentException: Failed requirement.
    at MainKt.main(asd.kt:32)
    at MainKt.main(asd.kt)

Harborage

Answer 1

Encoding raw JSON is possible in Kotlinx Serialization 1.5.0, which was released on 24th Feb 2023, and is experimental. It is not possible in earlier versions.

tl:dr: skip to 'Full example' at the bottom of this answer

Decoding using JsonDecoder

Note that it's only encoding that requires the workaround - decoding BigDecimal and BigInteger will work directly, so long as JsonDecoder is used!

private object BigDecimalSerializer : KSerializer<BigDecimal> {

  // ...

  override fun deserialize(decoder: Decoder): BigDecimal =
    when (decoder) {
      // must use decodeJsonElement() to get the value, and then convert it to a BigDecimal
      is JsonDecoder -> decoder.decodeJsonElement().jsonPrimitive.content.toBigDecimal()
      else -> decoder.decodeString().toBigDecimal()
    }
}

Encoding using JsonUnquotedLiteral

To encode, the new JsonUnquotedLiteral() function must be used when encoding JSON.

private object BigDecimalSerializer : KSerializer<BigDecimal> {

  // ...

  override fun serialize(encoder: Encoder, value: BigDecimal) =
    when (encoder) {
      // use JsonUnquotedLiteral() to encode the BigDecimal literally
      is JsonEncoder -> encoder.encodeJsonElement(JsonUnquotedLiteral(value.toPlainString()))
      else -> encoder.encodeString(value.toPlainString())
    }
}

Global config using typealias

Kotlinx Serialization uses typealias to define globally available serialization strategies. Let's do the same for BigDecimal

typealias BigDecimalJson = @Serializable(with = BigDecimalSerializer::class) BigDecimal

Example usage

After creating the serializers, the typealiases can be used in FooNumber to automatically use the KSerializers.

@Serializable
data class FooNumbers(
  val decimal: BigDecimalJson,
  val integer: BigIntegerJson,
)

The actual main function doesn't change - it's the same as before.

fun main() {
  val fooDecimal = BigDecimal("0.1234567890123456789012345678901234567890")
  val fooInteger = BigInteger("9876543210987654321098765432109876543210")

  val fooNumbers = FooNumbers(fooDecimal, fooInteger)
  println("$fooNumbers")

  val encodedNumbers = Json.encodeToString(fooNumbers)
  println(encodedNumbers)

  val decodedFooNumbers = Json.decodeFromString<FooNumbers>(encodedNumbers)
  println("$decodedFooNumbers")

  require(decodedFooNumbers == fooNumbers)
}

Now the BigDecimal and BigInteger can be encoded and decoded exactly, no loss of precision!

FooNumbers(decimal=0.1234567890123456789012345678901234567890, integer=9876543210987654321098765432109876543210)
{"decimal":0.1234567890123456789012345678901234567890,"integer":9876543210987654321098765432109876543210}
FooNumbers(decimal=0.1234567890123456789012345678901234567890, integer=9876543210987654321098765432109876543210)

Full example

Here's the full code:

import java.math.*
import kotlinx.serialization.*
import kotlinx.serialization.descriptors.*
import kotlinx.serialization.encoding.*
import kotlinx.serialization.json.*

@Serializable
data class FooNumbers(
  val decimal: BigDecimalJson,
  val integer: BigIntegerJson,
)

fun main() {
  val fooDecimal = BigDecimal("0.1234567890123456789012345678901234567890")
  val fooInteger = BigInteger("9876543210987654321098765432109876543210")

  val fooNumbers = FooNumbers(fooDecimal, fooInteger)
  println("$fooNumbers")

  val encodedNumbers = Json.encodeToString(fooNumbers)
  println(encodedNumbers)

  val decodedFooNumbers = Json.decodeFromString<FooNumbers>(encodedNumbers)
  println("$decodedFooNumbers")

  require(decodedFooNumbers == fooNumbers)
}

typealias BigDecimalJson = @Serializable(with = BigDecimalSerializer::class) BigDecimal

@OptIn(ExperimentalSerializationApi::class)
private object BigDecimalSerializer : KSerializer<BigDecimal> {

  override val descriptor = PrimitiveSerialDescriptor("java.math.BigDecimal", PrimitiveKind.DOUBLE)

  /**
   * If decoding JSON uses [JsonDecoder.decodeJsonElement] to get the raw content,
   * otherwise decodes using [Decoder.decodeString].
   */
  override fun deserialize(decoder: Decoder): BigDecimal =
    when (decoder) {
      is JsonDecoder -> decoder.decodeJsonElement().jsonPrimitive.content.toBigDecimal()
      else           -> decoder.decodeString().toBigDecimal()
    }

  /**
   * If encoding JSON uses [JsonUnquotedLiteral] to encode the exact [BigDecimal] value.
   *
   * Otherwise, [value] is encoded using encodes using [Encoder.encodeString].
   */
  override fun serialize(encoder: Encoder, value: BigDecimal) =
    when (encoder) {
      is JsonEncoder -> encoder.encodeJsonElement(JsonUnquotedLiteral(value.toPlainString()))
      else           -> encoder.encodeString(value.toPlainString())
    }
}

typealias BigIntegerJson = @Serializable(with = BigIntegerSerializer::class) BigInteger

@OptIn(ExperimentalSerializationApi::class)
private object BigIntegerSerializer : KSerializer<BigInteger> {

  override val descriptor = PrimitiveSerialDescriptor("java.math.BigInteger", PrimitiveKind.LONG)

  /**
   * If decoding JSON uses [JsonDecoder.decodeJsonElement] to get the raw content,
   * otherwise decodes using [Decoder.decodeString].
   */
  override fun deserialize(decoder: Decoder): BigInteger =
    when (decoder) {
      is JsonDecoder -> decoder.decodeJsonElement().jsonPrimitive.content.toBigInteger()
      else           -> decoder.decodeString().toBigInteger()
    }

  /**
   * If encoding JSON uses [JsonUnquotedLiteral] to encode the exact [BigInteger] value.
   *
   * Otherwise, [value] is encoded using encodes using [Encoder.encodeString].
   */
  override fun serialize(encoder: Encoder, value: BigInteger) =
    when (encoder) {
      is JsonEncoder -> encoder.encodeJsonElement(JsonUnquotedLiteral(value.toString()))
      else           -> encoder.encodeString(value.toString())
    }
}

Harborage