Assert equality of floating point values to a significant figure tolerance

Question

I am trying to write a unittest to check the output of an engineering analysis. I have theoretical values which i want to check against the analysis to a certain number of significant figures. so, for example:

Ixx_ther = 0.000123
Iyy_ther = 0.0123

Ixx, Iyy = getI(*args, **kwargs)

self.assertAlmostEqual(Ixx_ther, Ixx, 6)
self.assertAlmostEqual(Iyy_ther, Iyy, 4)

In this case, i need to know the number i am trying to check as setting the tolerance to 6 in both cases would make the test too stringent and setting it to 4 would be too lax. What I need is a test for equality to the same number of significant figures. What would be ideal is to say:

Ixx_ther = 1.23E-4
Iyy_ther = 1.23E-2

Ixx, Iyy = getI(*args, **kwargs)

self.assertAlmostEqual(Ixx_ther, Ixx, 2)
self.assertAlmostEqual(Iyy_ther, Iyy, 2)

and have the assert statement drop exponent and check only the Significand for equality. I imagine this has been done before, but I have not been able to find a built-in function to assert equality in this manner. Has anyone had this problem before,

Questions

1) Has anyone had this problem before, and know of a general guide of unittests for engineering analysis

2) Is there a built-in solution. to this problem

3) Has someone already programmed a custom assert statement which works in this manner?

Chiu

Be careful about how you're defining "significant figures". It's not just the number of decimal places...the zeroes immediately after the decimal aren't counted in the usual definition, so both your examples Ixx_ther and Iyy_ther would be considered to have three significant figures. – Urbana

Answer 1

Re: is there a built-in solution for this: If you can have numpy as a dependency, have a look at numpy.testing.

Here's an example ( verbatim from assert_allclose docs):

>>> x = [1e-5, 1e-3, 1e-1]
>>> y = np.arccos(np.cos(x))
>>> assert_allclose(x, y, rtol=1e-5, atol=0)

EDIT: For completeness, here's the link to the source code: assert_allclose forwards the real work to np.allclose. Which is nearly identical to @Mark Ransom's answer (plus handling of array arguments and infinities).

Chrysostom

Thanks, this was exactly what I was looking for. I am using numpy but never saw that it has a testing module. – Chiu

@steve855: Edited in the link to the actual implementation of np.allclose – Chrysostom

Answer 2

This is a reworking of an answer I left on another question.

def AlmostEqual(a, b, digits):
    epsilon = 10 ** -digits
    return abs(a/b - 1) < epsilon

This needs a little more work if b can be zero.

Counter

Thanks, this is a very elegant solution, though i think I am going to use the numpy testing module proposed by @Zhenya – Chiu

(+1) this works almost always >>> AlmostEqual(1e320, 1e320, 2) False – Chrysostom

Answer 3

Perhaps not answering the full scope of your question, but this is how I would write such a function:

def assertAlmostEqual(arg1,arg2,tolerance=2):
    str_formatter = '{0:.' + str(tolerance) + 'e}'
    lhs = str_formatter.format(arg1).split('e')[0]
    rhs = str_formatter.format(arg2).split('e')[0]
    assert lhs == rhs

Python's string formatting mini-language can be leveraged to format your floats into a given manner. So what we can do is force them to be formatted in exponent notation such that, for i.e. inputs 0.123 and 0.000123 we have:

str_formatter.format(0.123) == '1.23e-01'
str_formatter.format(0.000123) == '1.23e-04'

And all that remains is to chop off the exponent and assert equality.

Demo:

assertAlmostEqual(0.0123,0.0001234)

assertAlmostEqual(0.123,0.0001234)

assertAlmostEqual(0.123,0.0001234,tolerance=3)
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
/home/xxx/<ipython-input-83-02fbd71b2e87> in <module>()
----> 1 assertAlmostEqual(0.123,0.0001234,tolerance=3)

/home/xxx/<ipython-input-74-ae32ed74769d> in assertAlmostEqual(arg1, arg2, tolerance)
      3     lhs = str_formatter.format(arg1).split('e')[0]
      4     rhs = str_formatter.format(arg2).split('e')[0]
----> 5     assert lhs == rhs
      6 

AssertionError: 

There might be an off-by-one issue if you don't like the way I defined tolerance. Gets the idea across though.

Chemism

Answer 4

Thanks roippi for the great idea, I modified your code somewhat:

def assertAlmostEqualSigFig(self, arg1,arg2,tolerance=2):
    if tolerance > 1: 
        tolerance -= 1
    #end

    str_formatter = '{0:.' + str(tolerance) + 'e}'
    significand_1 = float(str_formatter.format(arg1).split('e')[0])
    significand_2 = float(str_formatter.format(arg2).split('e')[0])

    exponent_1 = int(str_formatter.format(arg1).split('e')[1])
    exponent_2 = int(str_formatter.format(arg2).split('e')[1])

    self.assertEqual(significand_1, significand_2)
    self.assertEqual(exponent_1, exponent_2)

    return

I changed a few things

1) I check the exponent as well as the significand (That's a top drawer word isn't it)

2) I convert the significand and exponent to float / int respectively. This may not be necessary but i am more comfortable checking the equality of numbers as numbers rather than strings.

3) Jim Lewis noted that i need to adjust my tolerance by one since the proper format string {0:.3e} of 0.0123 is 1.230E-2 not 0.123E-1. i.e. if you want three significant figures you only want two digits after the decimal as the digit before the decimal is also significant.

Hers is an example of implementation

class testSigFigs(Parent_test_class):

    @unittest.expectedFailure
    def test_unequal_same_exp(self):
        self.assertAlmostEqualSigFig(0.123, 0.321, 3)

    @unittest.expectedFailure
    def test_unequal_diff_exp(self):
        self.assertAlmostEqualSigFig(0.123, 0.0321, 3)

    @unittest.expectedFailure
    def test_equal_diff_exp(self):
        self.assertAlmostEqualSigFig(0.0123, 0.123, 3)

    def test_equal_same_exp(self):
        self.assertAlmostEqualSigFig(0.123, 0.123, 3)

    def test_equal_within_tolerance(self):
        self.assertAlmostEqualSigFig(0.123, 0.124, 2)
    #end

And the output:

test_equal_diff_exp (__main__.testSigFigs) ... expected failure
test_equal_same_exp (__main__.testSigFigs) ... ok
test_equal_within_tolerance (__main__.testSigFigs) ... ok
test_unequal_diff_exp (__main__.testSigFigs) ... expected failure
test_unequal_same_exp (__main__.testSigFigs) ... expected failure

----------------------------------------------------------------------
Ran 5 tests in 0.081s

OK (expected failures=3)

Thank you both for your feedback.

Chiu

Answer 5

With no additional libraries, python supports a < x < b as a range expression:

def test_abserr():
    abserr=1e-4
    expect=1.3456
    result=calculate()
    assert expect-abserr <= result <= expect+abserr

or

def test_relerr():
    relerr=1e-4
    expect=1.3456
    result=calculate()
    assert expect*(1-relerr) <= result <= expect*(1+relerr)

If you use pytest, you will get the result and expected values if they are out of range in the assertion error, which helps a lot in figuring out what went wrong.

If you are using Pandas, it has a nice interval class that is cleaner to write and avoids repetition. This can be used in conjunction with pytest to get values when they are out of range.

from pandas import Interval
# plus or minus closed interval [-1,1]
pm=Interval(-1,1,'both')

def test_stuff():
    # absolute error test
    assert result in expect+pm*abserr

    # relative error test
    assert result in expect*(1+pm*relerr)

Medievalism