Disable scientific notation in python json.dumps output

Question

json.dumps outputs small float or decimal values using scientific notation, which is unacceptable to the json-rpc application this output is sent to.

>>> import json
>>> json.dumps({"x": 0.0000001})
'{"x": 1e-07}'

I want this output instead:

'{"x": 0.0000001}'

It would be ideal to avoid introducing additional dependencies.

Groundspeed

I'd hesitate to call it a "JSON-RPC" application if it doesn't actually understand JSON... – Evelyn

or floats for that matter... – Neglect

Your only choice is to convert it to a string. Will that work? – Intrigue

see https://mcmap.net/q/57161/-format-floats-with-standard-json-module/989121 – Government

Answer 1

One way to format

evil = {"x": 0.00000000001}

is to steal Decimal's "f" formatter. It's the only easy way I've found that avoids both cropping problems and exponents, but it's not space efficient.

class FancyFloat(float):
    def __repr__(self):
        return format(Decimal(self), "f")

To use it you can make an encoder that "decimalize"s the input

class JsonRpcEncoder(json.JSONEncoder):
    def decimalize(self, val):
        if isinstance(val, dict):
            return {k:self.decimalize(v) for k,v in val.items()}

        if isinstance(val, (list, tuple)):
            return type(val)(self.decimalize(v) for v in val)

        if isinstance(val, float):
            return FancyFloat(val)

        return val

    def encode(self, val):
        return super().encode(self.decimalize(val))

JsonRpcEncoder().encode(evil)
#>>> '{"x": 0.00000000000999999999999999939496969281939810930172340963650867706746794283390045166015625}'

or, of course, you could move the decimalization out into a function and call that before json.dumps.

That's how I would do it, even if it's a lame method.

---

Update

Sam Mason suggests format(Decimal(str(self)), "f") instead, which should still always round-trip, but also produces shorter outputs.

Clockwork

about using format(Decimal(str(self)), 'f') instead? I get a much more compact encoding with that, and Python already ensures that the float representation can "round-trip" back to the same float – Wideawake

@SamMason Clever, I like it! – Clockwork

Answer 2

I can't find an answer to avoid the problem that converts 0.00001 to 1e-5,so I wrote a pretty_float_json_dumps function. It works fine in my project! Hope it can help someone!!

def pretty_float_json_dumps(json_obj):
    dumps_str = ""

    if isinstance(json_obj, dict): 
        dumps_str += "{"
        for k,v in json_obj.items():
            dumps_str += json.dumps(k)+":"
            if isinstance(v, float): 
                float_tmp_str = ("%.16f" % v).rstrip("0")
                dumps_str += (float_tmp_str+'0' if float_tmp_str.endswith('.') else float_tmp_str) + ','
            elif isinstance(v, list) or isinstance(v, dict): 
                dumps_str += pretty_float_json_dumps(v)+','
            else:
                dumps_str += pretty_float_json_dumps(v)+','
        if dumps_str.endswith(','):
            dumps_str = dumps_str[:-1]
        dumps_str += "}"
    elif isinstance(json_obj, list): 
        dumps_str += "["
        for v in json_obj:
            if isinstance(v, float): 
                float_tmp_str = ("%.16f" % v).rstrip("0")
                dumps_str += (float_tmp_str+'0' if float_tmp_str.endswith('.') else float_tmp_str) + ','
            elif isinstance(v, list) or isinstance(v, dict): 
                dumps_str += pretty_float_json_dumps(v)+','
            else:
                dumps_str += pretty_float_json_dumps(v)+','
        if dumps_str.endswith(','):
            dumps_str = dumps_str[:-1]
        dumps_str += "]"
    else:
        dumps_str += json.dumps(json_obj)
    return dumps_str

Beaded

Ty. I've edited a naming error and is seems it runs okay. Do you have any idea how this performs? – Porosity

Answer 3

This is supplemental commentary, not a complete answer for avoiding scientific notation in json.dumps. With an extra round of parsing, the parse_float option on json.loads can help with some things, e.g.

$ python
Python 3.7.10 | packaged by conda-forge | (default, Feb 19 2021, 16:07:37) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import json
>>> data = {"x": 0.0000001}
>>> json.dumps(data)
'{"x": 1e-07}'
>>> data = json.loads(json.dumps(data), parse_float=lambda x: round(float(x), 6))
>>> json.dumps(data)
'{"x": 0.0}'

This only avoids scientific notation for small values that are rounded to zero. While that is not efficient for large datasets, it can help in some use-cases. This does not completely avoid scientific notation.

>>> data = {"x": 0.00001}
>>> data = json.loads(json.dumps(data), parse_float=lambda x: round(float(x), 6))
>>> json.dumps(data)
'{"x": 1e-05}'

Sultanate

Answer 4

Such requirements should be formulated only for strings, as indeed this would require a custom float-like type if we wanted to keep a "numeric" (non-string) type.

But if you are ready to accept a string representation of floats, then there are dedicated tools for such customized rounding, such as numpy.format_float_positional():

import numpy as np

my_dict = {"x": 0.0000001}

print("float:", json.dumps(my_dict))

print("float-like str:", json.dumps({k:np.format_float_positional(v, precision=7) 
                                     for k,v in my_dict.items()}))


>>> float: {"x": 1e-07} 
>>> float-like str: {"x": "0.0000001"}

Pappas

Answer 5

found somewhere on the internet

import json
from json import encoder
encoder.FLOAT_REPR = lambda o: format(o, '.2f')

after this

>>> json.dumps({'gps': [51.5, 17.5]})
'{"gps": [51.5, 17.5]}'

Lontson

That does not seem to work: json.dumps({'a': 1/2000000}) gives me '{"a": 5e-07}' while the question was how to get rid of the scientific notation. – Engineman