How to compare two lists of double in JUnit

Question

In a JUnit 4 test, I have a method getValues() that returns a List<Double> object that I want to compare with a reference list. Up to now, the best solution I've found was to use org.hamcrest.collection.IsArray.hasItems and org.hamcrest.Matchers.closeTo like this:

assertThat(getValues(), hasItems(closeTo(0.2, EPSILON), closeTo(0.3, EPSILON)));

This goes well for a test that returns only few values. But if a test returns more values, this is definitely not the best approach.

I also tried the following code. The downcast to Matcher before hasItems is required for the code to compile:

List<Matcher<Double>> doubleMatcherList = new ArrayList<Matcher<Double>>();
doubleMatcherList.add(closeTo(0.2, EPSILON));
doubleMatcherList.add(closeTo(0.3, EPSILON));
assertThat(getValues(), (Matcher) hasItems(doubleMatcherList));

The comparison failed and I don't understand why :

java.lang.AssertionError: Expected: (a collection containing <[a numeric value within <1.0E-6> of <0.2>, a numeric value within <1.0E-6> of <0.3>]>) got: <[0.2, 0.30000000000000004]>

Is there a better way to compare two large lists of doubles? The difficulty here is that a numerical tolerance is required to validate if the result of getValues() is equal to my reference list. This kind of comparison seems very easy for any lists of objects, but not with lists of Double.

Vevay

There is a solution that always works : I iterate over the values of getValues() and do a one-to-one comparison with my reference values. I was wondering if there is a simpler solution using Matchers. – Vevay

Test driven development - the implementation is driven by your tests. If something is difficult to test, you are probably not doing it right. You cannot write a test for list of doubles - don't use doubles. Why don't you use integers/longs or BigDecimals? – Bean

@Jaroslav This test occurs in an application that does intensive numerical calculations. So not working with lists of doubles is out of the question. – Vevay

@Vevay You are trying to solve the problem rather than its cause. What do these doubles represent? Why does the precision matter? Why your epsilon is 0.0001 and not 0.000001 or 0.1? – Bean

@Jaroslav In the example I gave, the values are the result of a linear interpolation between 0.1 and 0.4, with an increment of 0.1. I expect the test to return a list with 0.2 and 0.3 as missing values. This is one of the simplest case I have to test. This test fails because the list contains 0.2 and 0.300000004, unless I use an appropriate matcher. – Vevay

Do this math on integers then (or longs). Instead of doing linear interpolation between 0.1 and 0.4 with an increment of 0.1, do it between 1 and 4 with an increment of 1. Similarly as instead of representing the amount of dollars as double, you would use long that would represent cents. – Bean

@JaroslawPawlak BigDecimals are as much of a pain to test as doubles. :/ The scale has to the same as well as the number. – Quickman

@mlk You are right. I think that the best solution is to do this math on whole numbers. If epsilon is 0.001, we just give input numbers multiplied by 1000 and expect int/long/BigInteger back. – Bean

I don't agree with you guys. If the result of a test is within a numerical tolerance of the expected value, this test is a success. You believe I don't control the output, and I disagree with you. This is numerical computation. With a given input, I expect the output to be close to a given output. Numerical computation will never converge to an exact solution. Why would such test be bad? Would I have to numerically solve a set of equations or find the roots of a function using only integers? I don't think so. – Vevay

Answer 1

I think the right solution here would be a Custom Matcher. Basically something like IsIterableContainingInOrder that only works on Doubles and supports an error margin.

Quickman

Answer 2

If you are willing to convert from List<Double> to double[], assertArrayEquals allows specifying a tolerance for error:

assertArrayEquals(new double[] {1, 2}, new double[] {1.01, 2.09}, 1E-1);

In Java 8, the conversion from list to array is relatively clean (see related question). For example:

double[] toArray(List<Double> list) {
    return list.stream().mapToDouble(Number::doubleValue).toArray();
}

And the assert statement could then be as follows:

assertArrayEquals(toArray(refList), toArray(getValues()), 1E-9);

p.s. toArray can be made to work with any Number type by just changing the signature to double[] toArray(List<? extends Number> list).

Carinthia

This doesn't handle nulls, which are possible in a List of Doubles. – Bioscope

Answer 3

You need to use contains() rather than hasItems().

Casting the return of hasItems() to Matcher is hiding a type error. Your code is actually checking to see if the result of getValues() is a List that has the 2 Matchers you created, it's not evaluating those Matchers against the result of getValues().

This is because hasItems() doesn't have an overload taking a List<Matcher<? super T>> like contains() does.

This does what you want, without going through the hassle of a custom Matcher:

List<Matcher<? super Double>> doubleMatcherList = new ArrayList<>();
doubleMatcherList.add(closeTo(0.2, EPSILON));
doubleMatcherList.add(closeTo(0.3, EPSILON));
assertThat(getValues(), contains(doubleMatcherList));

Note the parameterised type of the doubleMatcherList. If it's just List<Matcher<Double>> it selects the wrong overload of contains().

Carmencita

Answer 4

If you are willing to change from Hamcrest to AssertJ assertions, here is a solution in java 8.

import static org.assertj.core.api.Assertions.assertThat;
import java.util.Arrays;
import java.util.Comparator;
import java.util.List;
import org.junit.Test;
// import java.util.function.Function;

public class DoubleComparatorTest {

  // private static final Double DELTA = 1E-4;
  // private static final Function<Double, Comparator<Double>> DOUBLE_COMPARATOR_WITH_DELTA =
  // (delta) -> (o1, o2) -> (o1 - o2 < delta) ? 0 : o1.compareTo(o2);

  private Comparator<Double> COMPARATOR = (o1, o2) -> (o1 - o2 < 0.0001) ? 0 : o1.compareTo(o2);

  @Test
  public void testScaleCalculationMaxFirstPositive() {
    List<Double> expected = Arrays.asList(1.1D, 2.2D, 3.3D);
    List<Double> actual = Arrays.asList(1.10001D, 2.2D, 3.30001D);

    assertThat(actual)
        // .usingElementComparator(DOUBLE_COMPARATOR_WITH_DELTA.apply(DELTA))
        .usingElementComparator(COMPARATOR)
        .isEqualTo(expected);
  }
}

Cheerless

Answer 5

There is no hasItems method which takes a list of matchers as argument. But you can use the one with varargs.

First convert your list of matchers to an array

@SuppressWarnings("unchecked")
private Matcher<Double>[] toDoubleMatcherArray(List<Matcher<Double>> doubleMatchers) {
    return doubleMatchers.toArray(new Matcher[0]);
}

and call it like this

assertThat(getValues(), hasItems(toDoubleMatcherArray(doubleMatcherList)));

Harald

Answer 6

I don't believe no one mentioned it. If you have the expected list, and

• want the actual list to be in the same order as expected list then use

  assertEquals(expected,actual);

• want the lists to be equal regardless of order then do assertTrue(expected.containsAll(actual) && actual.containsAll(expected));

Putsch

People did mention it, got downvoted and deleted their answers because they were nowhere near of solving OP's problem. – Bean

@JaroslawPawlak But why? – Putsch

For the same reason why 4.35d * 100 != 435.0d – Bean

Sorry am just trying to understand the concept, so if I call equals on the following lists Arrays.asList((double) 430, (double) 2) and Arrays.asList(4.3 * 100, (double) 2), should it be false? – Putsch

@RahulSharma The problem is that the two lists are different but numerically equal. In posts that were deleted, I mentioned that a list with 0.2 and 0.3 is not equal to a list with 0.2 and 0.300000004. The only way to make these two lists equal (which is what this post is about) is to introduce a numerical tolerance in the comparison. Your solution doen't solve my problem. – Vevay

@RahulSharma However, your solution would work perfectly for any other list. – Vevay

@Vevay but what type of test are you writing when you cant even produce the expected result? – Putsch

@RahulSharma These are tests for numerical algorithms. It could be interpolation, matrix operations, resolution of equations system, or finding roots of nonlinear equations, just to give you a few examples. In the example I gave in my initial post, I know that the expected result will be close to 0.2 and 0.3, and it's enough for the test to succeed. I cannot expect that a numerical method will give exact solution, otherwise there would be no need for numerical methods at all. I won't modify my code to use integers instead of doubles just because it would be easier to control the output. – Vevay

Answer 7

If the two lists have different lengths then say they are different. If they have the same lengths, then sort the lists. After sorting them compare elements at the same index.

This can be discussed in more details whether you should compare each element from list A to corresponding elements from list B, e.g. compare A[i] with B[i-d], ... B[i], ..., B[i + d].

But for starters this might help you with getting better idea.

Update: If you can't modify the lists, you can always copy them and sort them.

Cheep

Sort the lists? What? And what if the order matters? What if expected is [3.0, 1.0, 2.0] but it returns [2.0, 1.0, 3.0]? Your test is going to pass even though it shouldn't. – Bean

Well, if order matters you don't sort the lists. – Apple

I have obviously miss-understood the question. Sorry, too tired. Also, how would order matter if the name of the method is hasItems? That by default means order doesn't matter, so I don't quite understand your comment about it. The real problem would be that my answer says how to COMPARE two lists not if one contains elements of another. But still, using the same approach it can be solved in O(N log N). I don't know how hamcrest works, but I understand that the problem is complexity? – Cheep