Writing unit test for @Nonnull annotated parameter

Question

I have a method like this one:

 public void foo(@Nonnull String value) {...}

I would like to write a unit test to make sure foo() throws an NPE when value is null but I can't since the compiler refuses to compile the unit test when static null pointer flow analysis is enabled in IDE.

How do I make this test compile (in Eclipse with "Enable annotation-based null analysis" enabled):

@Test(expected = NullPointerException.class)
public void test() {
     T inst = ...
     inst.foo(null);
}

Note: In theory the static null pointer of the compiler should prevent cases like that. But there is nothing stopping someone from writing another module with the static flow analysis turned off and calling the method with null.

Common case: Big messy old project without flow analysis. I start with annotating some utility module. In that case, I'll have existing or new unit tests which check how the code behaves for all the modules which don't use flow analysis yet.

My guess is that I have to move those tests into an unchecked module and move them around as I spread flow analysis. That would work and fit well into the philosophy but it would be a lot of manual work.

To put it another way: I can't easily write a test which says "success when code doesn't compile" (I'd have to put code pieces into files, invoke the compiler from unit tests, check the output for errors ... not pretty). So how can I test easily that the code fails as it should when callers ignore @Nonnull?

Kalahari

Is it @Nonnull or @NotNull? – Coil

What happens if you pass null variable? String s = null; inst.foo(s); – Laresa

It seems to me like you're trying to test not your class, but the annotation. – Ferrigno

Seems specific to intelliJ, but this might help https://mcmap.net/q/907269/-why-nonnull-annotation-checked-at-runtime – Leighleigha

@GhostCat well I can kind of see the point of the test, and it's probably mostly a philosophical issue. But really the case here is that part of the contract is "the method works if null is not passed", and it does not make sense to test the case where the contract is broken. Or, the contract contains "if you pass null, a NullPointerException is thrown", in which case an annotation is the wrong place to implement that - after all, it's the class' responsibility to fulfill the contract, and the Annotation implementation is out of the class' control. – Ferrigno

@AdamArold That doesn't really matter. The IDEs that I know of can be configured to use different annotations for static null analysis. – Kalahari

@GhostCat I could add some details like the exact annotation which I'm using but there are many other which are supposed to achieve the same effect, so I'm leaving that out on purpose. The question works for all of them. The code doesn't compile which implies I have configured the IDE compiler to handle the annotation properly. How the instance is created is irrelevant. – Kalahari

@GhostCat Lastly, I know that the annotation doesn't change behavior, it just causes the compiler to refuse to compile the test. That what I want in production code but I want to disable it for a few tests where I want to make sure that the code breaks properly. I think code which says @Nonnull, but doesn't throw NPE, smells. – Kalahari

@Laresa The s is flagged as error "Variable can only be null at this point". – Kalahari

@AaronDigulla My answer is probably more of a comment ...well: I can't repro. Works fine in my eclipse. – Pithead

You could use more complicated case s = Boolean.TRUE ? null : "" or ` s = "TRUE'.toLowerCase().equals("true") ? null : "" ` to hide var init logic from compiler but still have null variable. – Laresa

Is @Nonnull the same as javax.annotation.Nonnull? – Disenchant

Answer 1

Hiding null within a method does the trick:

public void foo(@NonNull String bar) {
    Objects.requireNonNull(bar);
}

/** Trick the Java flow analysis to allow passing <code>null</code>
 *  for @Nonnull parameters. 
 */
@SuppressWarnings("null")
public static <T> T giveNull() {
    return null;
}

@Test(expected = NullPointerException.class)
public void testFoo() {
    foo(giveNull());
}

The above compiles fine (and yes, double-checked - when using foo(null) my IDE gives me a compile error - so "null checking" is enabled).

In contrast to the solution given via comments, the above has the nice side effect to work for any kind of parameter type (but might probably require Java8 to get the type inference correct always).

And yes, the test passes (as written above), and fails when commenting out the Objects.requireNonNull() line.

Pithead

This does not work for TestNG v7.1+. I see: org.testng.TestException: Argument for @Nonnull parameter 'xyz' must not be null. – Disenchant

@Disenchant Try to create a static AtomicReference<Object>() without a value and return that. – Kalahari

Answer 2

Why not just use plain old reflection?

try {
    YourClass.getMethod("foo", String.class).invoke(someInstance, null);
    fail("Expected InvocationException with nested NPE");
} catch(InvocationException e) {
    if (e.getCause() instanceof NullPointerException) {
        return; // success
    }
    throw e; // let the test fail
}

Note that this can break unexpectedly when refactoring (you rename the method, change the order of method parameters, move method to new type).

Sublingual

Because reflection breaks easily? – Pithead

But beyond that: one could go one step further - it might be possible to automate more of that. Meaning: if "instance creation" follows some common patterns, one could scan the class path for all custom classes, and all their public methods for the annotation. And then you not only invoke that method by reflection , but you also create the instance to invoke on. – Pithead

Well it's a test. Tests break, then you fix them. Obviously I wouldn't suggest this approach for production code (at least not unless there is a test to validate it) – Sublingual

Answer 3

Using assertThrows from Jupiter assertions I was able to test this:

public MethodName(@NonNull final param1 dao) {....

assertThrows(IllegalArgumentException.class, () -> new MethodName(null));

Footing

When null checking is enabled in the compiler, then this should not compile... – Kalahari

This will only compile for me if I call the method in the assertThrows() method. If I call the method normally it will not compile. – Footing

This looks like a compiler bug for me: It should not treat assertThrows() in a special way. Can you compile Supplier<MethodName> foo = () -> new MethodName(null);? How about Executable foo = () -> new MethodName(null);? – Kalahari

Answer 4

Here design by contract comes to picture. You can not provide null value parameter to a method annotated with notNull argument.

Chur

Hmm. What if you flip a random coin, and in 10% of all cases you assign null to a field ... which is then later used as parameter to that method? Do you think any compile time checking will tell you about that? – Pithead

And beyond that: as your input reads like now, it is much more of a comment. An answer should answer the question. Or, when giving a non-answer: make it very very clear why the question doesn't make sense. – Pithead

If you expect that the method argument(Not parameter) can be null, then why do u write the API Contract as notNull. – Chur

The point is: we both missed the point of the question initially. The OP wants to verify that any method that uses @NonNull on arguments checks these arguments. – Pithead

We can then write a client java project to call the said method and pass null value to it. The client hava project can be treated as unit test code – Chur

And the question is asking how to do write such a piece of client code. And your answer doesn't address that at all. – Pithead

My understanding here is that we are trying to test not our class method, but the annotation. Then why we will do it . – Chur

Again: read the question. He doesnt test the annotation. He wants to test the company internal contract that any method that uses such annotations also checks the parameters for being not null. – Pithead

We can write the @Test public void test(){ assertNotNull(methodargument)); inst.foo(methodargument); } } – Chur

Answer 5

You can use a field which you initialize and then set to null in a set up method:

private String nullValue = ""; // set to null in clearNullValue()
@Before
public void clearNullValue() {
    nullValue = null;
}

@Test(expected = NullPointerException.class)
public void test() {
     T inst = ...
     inst.foo(nullValue);
}

As in GhostCat's answer, the compiler is unable to know whether and when clearNullValue() is called and has to assume that the field is not null.

Kalahari