58.85 displays as 58.84999999999

Question

I am building a calculator with BSP. As I tested it with various numbers, I ran into a problem that the decimal numbers don't display correctly.

For example. 58.85 -> 58.849999. But 58.84 or 58.86 work just fine. 58.8471 -> 54.84710000000001. At the end last typed digit will be saved out of nowhere.

My code following below.

method GENERATE_NUM.

  DATA: lv_digi type I.  * number of digits after the decimal point


  call METHOD me->get_decimal
    RECEIVING
      getdigits = lv_digi.

  *if it is a natural number  
  IF lv_digi = 0.
    IF thisnum < 0.
      result = thisnum * 10 - newdigit.
    ELSE.
      result = thisnum * 10 + newdigit.
    ENDIF.

   *if it is a float number
   Else.
    IF thisnum < 0.
      result = thisnum - ( newdigit / 10 ** lv_digi ).
    ELSE.
      result = thisnum + ( newdigit / 10 ** lv_digi ).
    ENDIF.

    *increase the number of decimal point by 1
    call method me->set_decimal.
  ENDif.

endmethod.

What I basically do is everytime a number is clicked, it calls the "generate_num" method. It takes THISNUM, NEWDIGIT, RESULT as parameters.
thisnum = current number (eg:58.8)
newdigit = clicked number (eg: 5)
result = generated number (expected: 58.85 but returns 58.849999).

Ommatidium

docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html – Answerable

It would be helpful if you illustrate where and how to solve this problem? – Ommatidium

It's one of those commonly asked questions that needs a very long explanation. I'll let someone else step in or find a suitable duplicate to close as. – Answerable

Answer 1

When you want decimal numbers with fixed precision, you should use the type P (packed number) instead of float.

Example:

DATA lv_fixed_point TYPE p LENGTH 16 DECIMALS 2.

This creates a fixed-point variable with two digits after the point. The exact meaning of the "length" parameter isn't that straight-forward. From the documentation:

Packed numbers - type P

Type P data allows digits after the decimal point. The number of decimal places is generic, and is determined in the program. The value range of type P data depends on its size and the number of digits after the decimal point. The valid size can be any value from 1 to 16 bytes. Two decimal digits are packed into one byte, while the last byte contains one digit and the sign. Up to 14 digits are allowed after the decimal point. The initial value is zero. When working with type P data, it is a good idea to set the program attribute Fixed point arithmetic.Otherwise, type P numbers are treated as integers.

You can use type P data for such values as distances, weights, amounts of money, and so on.

Centigram

simply changing the type to p solved my problem. Thank you very much. – Ommatidium

Answer 2

Or you can call CALL FUNCTION 'FLOATINGPOINT_COMPARE_ABSOLUTE' to factor in IEEE float epsilons. Working with floating point you should actually avoid using comparisons directly, and write your own with built-in epsilons:

 IS_EQUAL
 IS_GREATER
 IS_GREATER_EQUAL
 IS_LESS
 IS_LESS_EQUAL

are the bare minimum.

More reading about floating point here.

Learn it. Live it. Love it.

As a non-trivial exercise you can also read up on NaN +Inf and -Inf (Also sNAN if you working with other DECFLOAT types instead of f).

Weasel

But packed can be the optimal solution (but not as fast since they are still interpreted as CHAR-kinda) with the least about of hurdles. – Weasel