Lua: Rounding numbers and then truncate

Question

Which is the best efficient way to round up a number and then truncate it (remove decimal places after rounding up)?

for example if decimal is above 0.5 (that is, 0.6, 0.7, and so on), I want to round up and then truncate (case 1). Otherwise, I would like to truncate (case 2)

for example:
232.98266601563 => after rounding and truncate = 233 (case 1)
232.49445450000 => after rounding and truncate = 232 (case 2)
232.50000000000 => after rounding and truncate = 232 (case 2)

Proffer

I know this is very late, but do you remember if you intended truncate also for negative numbers, IOW round halfway towards zero? My answer should do this after FPU rounding mode is adjusted accordingly. – Exception

Answer 1

There is no build-in math.round() function in Lua, but you can do the following: print(math.floor(a+0.5)).

Kiloliter

What is a supposed to be? – Carapace

@Carapace a is the value you want to round. – Dashtilut

This returns 1 for a = 0.49999999999999994; it should return 0. – Sebi

@PedroGimeno Is that the only wrong case, or are there others? I tried 1.49999999999999988 + 0.5, which worked fine. Same for 2.49999999999999977 + 0.5. – Cat

@RolandIllig It's the only case as far as I know. The addition of 0.49999999999999994 and 0.5 returns 1 due to rounding; for bigger numbers the rounding does not cause problems because they don't have enough significant digits as for them to affect the rounding. That number is special in that 0.5 is bigger than it. – Sebi

@PedroGimeno adding 0.49999999999999994 instead of 0.5 should fix that issue and I believe it should work for all other numbers then as well (as they would round up for the same reason). – Sarette

Answer 2

A trick that is useful for rounding at decimal digits other than whole integers is to pass the value through formatted ASCII text, and use the %f format string to specify the rounding desired. For example

mils = tonumber(string.format("%.3f", exact))

will round the arbitrary value in exact to a multiple of 0.001.

A similar result can be had with scaling before and after using one of math.floor() or math.ceil(), but getting the details right according to your expectations surrounding the treatment of edge cases can be tricky. Not that this isn't an issue with string.format(), but a lot of work has gone into making it produce "expected" results.

Rounding to a multiple of something other than a power of ten will still require scaling, and still has all the tricky edge cases. One approach that is simple to express and has stable behavior is to write

function round(exact, quantum)
    local quant,frac = math.modf(exact/quantum)
    return quantum * (quant + (frac > 0.5 and 1 or 0))
end

and tweak the exact condition on frac (and possibly the sign of exact) to get the edge cases you wanted.

Oid

Unfortunately string.format() rounds differently depending on the Lua version used. See my question about LuaJIT which prefers to round up while most others round down when faced with an "exact" half. – Gabble

As for the second part, why not simply (exact + quantum*0.5) // quantum * quantum? – Ferrochromium

Answer 3

To also support negative numbers, use this:

function round(x)
  return x>=0 and math.floor(x+0.5) or math.ceil(x-0.5)
end

Kwa

This returns 1 for x = 0.49999999999999994 and -1 for -0.49999999999999994; it should return 0. See my answer for a version that properly rounds to nearest or even. – Sebi

Answer 4

If your Lua uses double precision IEC-559 (aka IEEE-754) floats, as most do, and your numbers are relatively small (the method is guaranteed to work for inputs between -2⁵¹ and 2⁵¹), the following efficient code will perform rounding using your FPU's current rounding mode, which is usually round to nearest, ties to even:

local function round(num)
  return num + (2^52 + 2^51) - (2^52 + 2^51)
end

(Note that the numbers in parentheses are calculated at compilation time; they don't affect runtime).

For example, when the FPU is set to round to nearest or even, this unit test prints "All tests passed":

local function testnum(num, expected)
  if round(num) ~= expected then
    error(("Failure rounding %.17g, expected %.17g, actual %.17g")
          :format(num+0, expected+0, round(num)+0))
  end
end

local function test(num, expected)
  testnum(num, expected)
  testnum(-num, -expected)
end

test(0, 0)
test(0.2, 0)
test(0.4, 0)
-- Most rounding algorithms you find on the net, including Ola M's answer,
-- fail this one:
test(0.49999999999999994, 0)
-- Ties are rounded to the nearest even number, rather than always up:
test(0.5, 0)
test(0.5000000000000001, 1)
test(1.4999999999999998, 1)
test(1.5, 2)
test(2.5, 2)
test(3.5, 4)
test(2^51-0.5, 2^51)
test(2^51-0.75, 2^51-1)
test(2^51-1.25, 2^51-1)
test(2^51-1.5, 2^51-2)

-- Some of these will fail with the function above,
-- but will pass with the function below.
--test(2^51 + 1, 2^51 + 1)
--test(2^51 + 1.5, 2^51 + 2)
--test(2^51 + 2, 2^51 + 2)
--test(2^51 + 2.5, 2^51 + 2)
--test(2^51 + 3, 2^51 + 3)
--test(2^51 + 3.5, 2^51 + 4)
--test(2^52 + 1, 2^52 + 1)
--test(2^52 + 2, 2^52 + 2)
--test(2^53 + 2, 2^53 + 2)

print("All tests passed")

Here's another (less efficient, of course) algorithm that performs the same FPU rounding but works for all numbers:

local function round(num)
  if math.abs(num) > 2^52 then
    return num
  end
  return num < 0 and num - 2^52 + 2^52 or num + 2^52 - 2^52
end

Sebi

Could you explain why the correction needs to be 2^52 + 2^51? I tried 2^53 - 2, and it worked equally well for your test cases. Of course I could do the math myself, and it would certainly be worth it, but maybe you can share the tricky details and edge cases with a larger audience here. – Cat

@RolandIllig My test suite is not very exhaustive; it doesn't check all corner cases. Your number fails for e.g. 2.75, which gets rounded to 2.0. Adding and subtracting 2^52 to any number between 0 and 2^52-1 rounds it by pushing it to the range 2^52..2^53, where the double precision floats lose all their decimals, before adjusting back. To cover negatives, I added 2^51 in order to ensure the intermediate result is still in the range 2^52..2^53. If you push it beyond 2^53 as your number does often, you lose even numbers too; e.g. 2^53+1 can't be exactly represented in double precision. – Sebi

"you lose even numbers too" - I meant you lose odd numbers. – Sebi

Answer 5

For bad rounding (cutting the end off):

function round(number)
  return number - (number % 1)
end

Well, if you want, you can expand this for good rounding.

function round(number)
  if (number - (number % 0.1)) - (number - (number % 1)) < 0.5 then
    number = number - (number % 1)
  else
    number = (number - (number % 1)) + 1
  end
 return number
end

print(round(3.1))
print(round(math.pi))
print(round(42))
print(round(4.5))
print(round(4.6))

Expected results:

3, 3, 42, 5, 5

Conformity

Answer 6

Here's one to round to an arbitrary number of digits (0 if not defined):

function round(x, n)
    n = math.pow(10, n or 0)
    x = x * n
    if x >= 0 then x = math.floor(x + 0.5) else x = math.ceil(x - 0.5) end
    return x / n
end

Mattox

This works for Lua, but not LuaJIT. If you let x=32.90625 and n=4 then most Luas will give you 32.9062 but LuaJIT will give you 23.0963. See my question about how to avoid this. – Gabble

Good answer, but you should have been more careful when posting the code here. The n or 0 doesn't cover the digit number not being defined (using ... as the 2nd argument in the function and referencing it via arg[2] would have achieved that), not to mention there's a possible division by 0 situation by not proper handling the returned x / n. I could mention rare -0 occurences as well, but you get the idea: the method is good, as long as the exception cases are handled correctly. – Staphyloplasty

Answer 7

I like the response above by RBerteig: mils = tonumber(string.format("%.3f", exact)). Expanded it to a function call and added a precision value.

function round(number, precision)
   local fmtStr = string.format('%%0.%sf',precision)
   number = string.format(fmtStr,number)
   return number
end

Lardner

Unfortunately string.format() rounds differently depending on the Lua version used. See my question about LuaJIT which prefers to round up while most others round down when faced with an "exact" half. – Gabble

Answer 8

if not exist math.round

function math.round(x, n)
    return tonumber(string.format("%." .. n .. "f", x))
end

Stickinthemud

Answer 9

Here is a flexible function to round to different number of places. I tested it with negative numbers, big numbers, small numbers, and all manner of edge cases, and it is useful and reliable:

function Round(num, dp)
    --[[
    round a number to so-many decimal of places, which can be negative, 
    e.g. -1 places rounds to 10's,  
    
    examples
        173.2562 rounded to 0 dps is 173.0
        173.2562 rounded to 2 dps is 173.26
        173.2562 rounded to -1 dps is 170.0
    ]]--
    local mult = 10^(dp or 0)
    return math.floor(num * mult + 0.5)/mult
end

Nemeth

API docs: (1) the divergence between dpand dps can be confusing and (2) halfway rounding is done away from zero for positive inputs and towards zero for negative ones (I think that this should be included... (or reimplemented?)) – Exception

Answer 10

For rounding to a given amount of decimals (which can also be negative), I'd suggest the following solution that is combined from the findings already presented as answers, especially the inspiring one given by Pedro Gimeno. I tested a few corner cases I'm interested in but cannot claim that this makes this function 100% reliable:

function round(number, decimals)
  local scale = 10^decimals
  local c = 2^52 + 2^51
  return ((number * scale + c ) - c) / scale
end

These cases illustrate the round-halfway-to-even property (which should be the default on most machines):

assert(round(0.5, 0) == 0)
assert(round(-0.5, 0) == 0)
assert(round(1.5, 0) == 2)
assert(round(-1.5, 0) == -2)
assert(round(0.05, 1) == 0)
assert(round(-0.05, 1) == 0)
assert(round(0.15, 1) == 0.2)
assert(round(-0.15, 1) == -0.2)

I'm aware that my answer doesn't handle the third case of the actual question, but in favor of being IEEE-754 compliant, my approach makes sense. So I'd expect that the results depend on the current rounding mode set in the FPU with FE_TONEAREST being the default. And that's why it seems high likely that after setting FE_TOWARDZERO (however you can do that in Lua) this solution would return exactly the results that were asked for in the question.

Exception

That gives inconsistent results. 0.5 is rounded down to 0 whereas 1.5 is rounded up to 2. It should give 1 and 2 as results. – Interlope

@Interlope Inconsistent in what sense? I explained that this will round halfway cases towards the next even digit. And 0, 2, 4, 6, 8 are even. 1.499 or 0.501 should be still be rounded to 1. – Exception

Oh OK, now I understand what "round-halfway-to-even" means. Sorry, English is not my first language. – Interlope

@Interlope You are welcome. The way of rounding you refer to is probably the one that's implemented in the round function of C. – Exception

Answer 11

Should be math.ceil(a-0.5) to correctly handle half-integer numbers

Amadoamador

Depends on what "correctly" means. One has to carefully decide what to do about half steps, and what to do with negative values. Both add a half and floor and your subtract a half and ceil introduce a consistent bias to the case of the exact half. And both are different from add half and truncate assuming that truncation usually rounds towards zero. Implementing round to even value is more "fair" in some sense. Rounding is fraught with subtlety. – Oid

Answer 12

Try using math.ceil(number + 0.5) This is according to this Wikipedia page. If I'm correct, this is only rounding positive integers. you need to do math.floor(number - 0.5) for negatives.

Mariannemariano

This is basically another version of @Mattox 's answer. I did not see it before I posted. – Mariannemariano

This answer is incorrect. – Flybynight

Answer 13

If it's useful to anyone, i've hash-ed out a generic version of LUA's logic, but this time for truncate() :

**emphasized text pre-apologize for not knowing lua-syntax, so this is in AWK/lua mixture, but hopefully it should be intuitive enough

-- due to lua-magic alrdy in 2^(52-to-53) zone,
-- has to use a more coarse-grained delta than
-- true IEEE754 double machineepsilon of 2^-52

function trunc_lua(x,s) {
    return \
      ((x*(s=(-1)^(x<-x)) \
           - 2^-1 + 2^-50 \  -- can also be written as
                          \  --    2^-50-5^0/2
           -  _LUAMAGIC  \   -- if u like symmetric 
                         \   --    code for fun
           +  _LUAMAGIC \
      )  *(s) };

It's essentially the same concept as rounding, but force-processing all inputs in positive-value zone, with a -1*(0.5-delta) offset. The smallest delta i could attain is 2^-52 ~ 2.222e-16.

The lua-magic values must come after all those pre-processing steps, else precision-loss may occur. And finally, restore original sign of input.

The 2 "multiplies" are simply low-overhead sign-flipping. sign-flips 4 times for originally negative values (2 manual flips and round-trip to end of mantissa), while any x >= 0, including that of -0.0, only flips twice. All tertiary function calling, float division, and integer modulus is avoided, with only 1 conditional check for x<0.

usage notes :

• (1) doesn't perform checks on input for invalid or malicious payload,
• (2) doesn't use quickly check for zero,
• (3) doesn't check for extreme inputs that may render this logic moot, and
• (4) doesn't attempt to pretty format the value

Folium

Answer 14

I found this answer on luausers.org that I used to convert positive decimal fractions to integers.

SoniEx2 Handles 0.49999999999999994 (at least in Java. no special handling of negative numbers and/or decimal places tho)

function round(n)
  return math.floor((math.floor(n*2) + 1)/2)
end

Here's a tested, working example:

local error = 0.49999999999999994
local first = 0.5
local second = 16 / 3 
local third = 17 / 3

print("0.49999999999999994 correctly rounded = "..math.floor((math.floor(error*2) + 1)/2))

print(first.." correctly rounded = "..math.floor((math.floor(first*2) + 1)/2))

print(second.." correctly rounded is "..math.floor((math.floor(second*2) + 1)/2))

print(third.." correctly rounded is "..math.floor((math.floor(third*2) + 1)/2))

Melanous

Answer 15

You should just have to add 0.5 before using math.floor()

Desmos result: https://i.sstatic.net/hUNH4.png

Some people have mentioned using math.ceil() for negative numbers but I don't think that's necessary.

Meilhac