Decimal shift behaves bizarrely if constructed from float

Question

Consider the following Python code:

from decimal import Decimal

d = Decimal("1.23")
print(f"{d = }, {d.shift(1) = }")

When I execute it in Python 3.12.4, I get this output:

d = Decimal('1.23'), d.shift(1) = Decimal('12.30')

This is exactly the output that I expect: the shift operation, given an arg of positive 1, left shifted 1.23 by 1 decimal point, producing 12.30

Now execute this code:

from decimal import Decimal

d = Decimal(1.23)
print(f"{d = }, {d.shift(1) = }")

The sole difference is that we construct d using a float instead of a str.

When I execute it, I get this output:

d = Decimal('1.229999999999999982236431605997495353221893310546875'), d.shift(1) = Decimal('6.059974953532218933105468750E-24')

The first part of that output is what I expect: the float for 1.23 cannot perfectly represent the base 10 digits; this is expected floating point error.

But the output from shift(1) was strange. I was expecting it to be something like Decimal('12.29999999999999982236431605997495353221893310546875'). (That is, a value fairly close to 12.30.) Instead, it produced a completely unrelated result like Decimal('6.059974953532218933105468750E-24').

Why does the shift method produces such radically different results when you construct the Decimal from a float?

Jala

Weird. Testing w/ shift(0) - Decimal('1.2299999999999999822364316059') produced a similar result, while Decimal('1.229999999999999982236431605') (one digit fewer) did not. Presumably related to Is floating-point math broken? – Ea

Perhaps, @AndrewYim, but I don't see a clear sign of that. And if it is an issue related to FP representation, then it seems like that must be a bug in the implementation of Decimal, does it not? – Nonobedience

I kept trimming right-most digits and interestingly, once I'd trimmed off 24 characters, it started working. The result stepped down from -24, -23, ... to the right number. – Observance

Increase the precision to 53 or above (default 28 on my system) and it works. It looks like the calculation only keeps the last prec digits of the value. Note the correct result has 53 digits. – Shopwindow

Note: 53 digits including the shifted-in zero on the right. – Shopwindow

Interestingly, the same bug appears in both the normal C-language implementation of Decimal, and the pure-Python fallback implementation. Basically, the shift() method is broken by design - the trigger appears to be the number having more significant digits than the current context's precision value (52 vs 28 here). – Larcher

Answer 1

The decimal module implements the IBM Decimal Arithmetic Specification.

It states explicitly for rotate (but not for shift although it appears to be the same behavior):

If the coefficient of the first operand has fewer than precision digits, it is treated as though it were padded on the left with zeros to length precision before the rotation. Similarly, if the coefficient of the first operand has more than precision digits, it is truncated on the left before use.

So if there are more than precision digits, only the right-most precision digits are kept for the operation by shift and rotate.

In the following example with precision 6, 12 are dropped and 345678 are shifted or rotated:

import decimal as dec

c = dec.getcontext()
c.prec = 6

d = dec.Decimal('12345678')
print(d.shift(2))
print(f'{d.shift(-2):06}')  # formatted to show leading zeros.
print(d.rotate(2))
print(d.rotate(-2))

Output:

567800          # 34 was shifted out the left side.  Right shifted in zeros.
003456          # 78 was shifted out the right side.  Left shifted in zeros.
567834          # 34 rotated from left to right side.
783456          # 78 rotated from right to left side.

So for the example case of:

>>> dec.Decimal('1.229999999999999982236431605997495353221893310546875').shift(1)
Decimal('6.059974953532218933105468750E-24')

The default precision is 28, so only keep the last 28 digits (same exponent):

1.229999999999999982236431605997495353221893310546875
0.000000000000000000000001605997495353221893310546875  # truncated on left to 28 digits
0.000000000000000000000006059974953532218933105468750  # shift 1, drops left 1 and adds right 0

Which is ~6e-24. It follows the specification.

Increase the precision to handle the whole number:

import decimal as dec

c = dec.getcontext()
c.prec = 53
print(dec.Decimal('1.229999999999999982236431605997495353221893310546875').shift(1))

Output:

12.299999999999999822364316059974953532218933105468750

Shopwindow