PiTest "changed conditional boundary mutation survived" without reason?

Question

I have a small Java 11 example with a JUnit 5 test that results in a pitest result of:

changed conditional boundary → SURVIVED

Main class:

public final class CheckerUtils
 {
  private CheckerUtils()
   {
    super();
   }


  public static int checkPort(final int port)
   {
    if (port < 0)
     {
      throw new IndexOutOfBoundsException("Port number out of range!");
     }
    return port;
   }

 }

Test class:

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;

import org.junit.jupiter.api.Test;

import de.powerstat.security.CheckerUtils;


public final class CheckerUtilsTests
 {
  @Test
  public void checkPortOk()
   {
    final int port = 1023;
    final int resultPort = CheckerUtils.checkPort(port);
    assertEquals(port, resultPort, "Port not as expected!");
   }


  @Test
  public void checkPortNegative1()
   {
    final int port = -1;
    assertThrows(IndexOutOfBoundsException.class, () ->
     {
      CheckerUtils.checkPort(port);
     }
    );
   }


  @Test
  public void checkPortNegative2()
   {
    final int port = -1;
    int resultPort = 0;
    try
     {
      resultPort = CheckerUtils.checkPort(port);
     }
    catch (final IndexOutOfBoundsException e)
     {
      // ignore
     }
    assertEquals(0, resultPort, "Port is not 0");
   }

 }

From my point of view the mutation should not survive, because:

1. checkPortOk() is the normal path for a legal value that is not negative
2. checkPortNegative1() is the path for the negative value when noting is mutated an the exception is thrown.
3. checkPortNegative2(): When nothing is mutated, the exception is thrown and resultPort is still 0 - so the assertion is ok here
4. checkPortNegative2(): When the < 0 is mutated to < -1 or something lower, then no exception will be thrown, so resultPort will become -1 and the assert will fail (mutation killed)
5. checkPortNegative2(): When the < 0 is mutated to < 1 or something higher, the same as under 3.

So my question is did I miss something here or is it a bug in pitest (1.4.9)?

Solution

As statet by @henry,adding the following test solves the issue:

@Test
public void checkPortOk2()
 {
  final int port = 0;
  final int resultPort = CheckerUtils.checkPort(port);
  assertEquals(port, resultPort, "Port not as expected!");
 }

Bugger

Answer 1

The conditional boundary mutation will mutate

if (port < 0)

To

if (port <= 0)

As none of the tests provide an input of 0 they are not able to distinguish the mutant from the unmutated program and the mutant will survive.

Adding a test case that describes the expected behaviour when port is 0 should kill the mutant.

Oleander

Answer 2

I've faced with similar issue. I have a trivial method for truncating a string to target length but I couldn't cover it with proper tests acceptable for pitest:

 /**
 * Truncates a String.
 * Similar to the {@code truncate} method from Apache commons-lang3.
 *
 * @param str      the String to truncate, cannot be null
 * @param maxWidth maximum length of result String, must be positive
 * @return truncated String
 */
@Nonnull
public static String truncate(@Nonnull final String str, final int maxWidth) {
    Objects.requireNonNull(str, "str cannot be null");
    if (maxWidth < 0) {
        throw new IllegalArgumentException("maxWith cannot be negative");
    }
    if (str.length() <= maxWidth) {
        return str;
    }
    return str.substring(0, maxWidth);
}

And test:

@Test
void truncateShouldWork() {
    final String target = "abcqwe";
    assertThat(StringUtils.truncate(target, 0))
            .isEmpty();
    assertThat(StringUtils.truncate(target, 1))
            .isEqualTo("a");
    assertThat(StringUtils.truncate(target, 2))
            .isEqualTo("ab");
    assertThat(StringUtils.truncate(target, 5))
            .isEqualTo("abcqw");
    assertThat(StringUtils.truncate(target, 6))
            .isEqualTo("abcqwe")
            .isSameAs(target);
    assertThat(StringUtils.truncate(target, 7))
            .isEqualTo("abcqwe")
            .isSameAs(target);
}

But I've still got an error changed conditional boundary → SURVIVED

The only idea I have is to add logging to my method:

LOGGER.trace("String {} will be truncated", str);
return str.substring(0, maxWidth);

And check this in test:

private static final String TARGET = "abcqwe";

@Test
void truncationShouldBePerformed() {
    try (LogsCaptor logsCaptor = new LogsCaptor(StringUtils.class, Level.TRACE)) {
        assertThat(StringUtils.truncate(TARGET, 0))
                .isEmpty();
        assertThat(StringUtils.truncate(TARGET, 1))
                .isEqualTo("a");
        assertThat(StringUtils.truncate(TARGET, 2))
                .isEqualTo("ab");
        assertThat(StringUtils.truncate(TARGET, 5))
                .isEqualTo("abcqw");

        assertThat(logsCaptor.getLogs())
                .hasSize(4);
    }
}

@Test
void truncationShouldNotBePerformed() {
    try (LogsCaptor logsCaptor = new LogsCaptor(StringUtils.class, Level.TRACE)) {
        assertThat(StringUtils.truncate(TARGET, 6))
                .isEqualTo("abcqwe")
                .isSameAs(TARGET);
        assertThat(StringUtils.truncate(TARGET, 7))
                .isEqualTo("abcqwe")
                .isSameAs(TARGET);

        assertThat(logsCaptor.getLogs())
                .isEmpty();
    }
}

Jumbuck