How to test methods that call System.exit()?

Question

I've got a few methods that should call System.exit() on certain inputs. Unfortunately, testing these cases causes JUnit to terminate! Putting the method calls in a new Thread doesn't seem to help, since System.exit() terminates the JVM, not just the current thread. Are there any common patterns for dealing with this? For example, can I substitute a stub for System.exit()?

The class in question is actually a command-line tool which I'm attempting to test inside JUnit. Maybe JUnit is simply not the right tool for the job? Suggestions for complementary regression testing tools are welcome (preferably something that integrates well with JUnit and EclEmma).

Math

I'm curious as to why a function would ever call System.exit()... – Antiicer

If you're calling a function that exits the application. For example, if the user tries to perform a task they are not authorized to perform more that x times in a row, you force them out of the application. – Revest

I still think that in that case, there should be a nicer way to return from the application rather than System.exit(). – Antiicer

If you're testing main(), then it makes perfect sense to call System.exit(). We have a requirement that on error a batch process should exit with 1 and on success exit with 0. – Ominous

I disagree with everyone who says System.exit() is bad - your program should fail fast. Throwing an exception merely prolongs an application in an invalid state in situations where a developer wants to exit and will give spurious errors. – Haug

@ThomasOwens I'm curious as to why you think System.exit is bad or not required. Please tell me how to return a batch with exit code 20 or 30 from a java application . – Signboard

Answer 1

Indeed, Derkeiler.com suggests:

• Why System.exit() ?

  Instead of terminating with System.exit(whateverValue), why not throw an unchecked exception? In normal use it will drift all the way out to the JVM's last-ditch catcher and shut your script down (unless you decide to catch it somewhere along the way, which might be useful someday).

  In the JUnit scenario it will be caught by the JUnit framework, which will report that such-and-such test failed and move smoothly along to the next.

• Prevent System.exit() to actually exit the JVM:

  Try modifying the TestCase to run with a security manager that prevents calling System.exit, then catch the SecurityException.

public class NoExitTestCase extends TestCase 
{

    protected static class ExitException extends SecurityException 
    {
        public final int status;
        public ExitException(int status) 
        {
            super("There is no escape!");
            this.status = status;
        }
    }

    private static class NoExitSecurityManager extends SecurityManager 
    {
        @Override
        public void checkPermission(Permission perm) 
        {
            // allow anything.
        }
        @Override
        public void checkPermission(Permission perm, Object context) 
        {
            // allow anything.
        }
        @Override
        public void checkExit(int status) 
        {
            super.checkExit(status);
            throw new ExitException(status);
        }
    }

    @Override
    protected void setUp() throws Exception 
    {
        super.setUp();
        System.setSecurityManager(new NoExitSecurityManager());
    }

    @Override
    protected void tearDown() throws Exception 
    {
        System.setSecurityManager(null); // or save and restore original
        super.tearDown();
    }

    public void testNoExit() throws Exception 
    {
        System.out.println("Printing works");
    }

    public void testExit() throws Exception 
    {
        try 
        {
            System.exit(42);
        } catch (ExitException e) 
        {
            assertEquals("Exit status", 42, e.status);
        }
    }
}

---

Update December 2012:

Will proposes in the comments using System Rules, a collection of JUnit(4.9+) rules for testing code which uses java.lang.System.
This was initially mentioned by Stefan Birkner in his answer in December 2011.

System.exit(…)

Use the ExpectedSystemExit rule to verify that System.exit(…) is called.
You could verify the exit status, too.

For instance:

public void MyTest {
    @Rule
    public final ExpectedSystemExit exit = ExpectedSystemExit.none();

    @Test
    public void noSystemExit() {
        //passes
    }

    @Test
    public void systemExitWithArbitraryStatusCode() {
        exit.expectSystemExit();
        System.exit(0);
    }

    @Test
    public void systemExitWithSelectedStatusCode0() {
        exit.expectSystemExitWithStatus(0);
        System.exit(0);
    }
}

---

2023: Emmanuel Bourg reports in the comments:

For this to work with Java 21 this system property must be set -Djava.security.manager=allow

Corriecorriedale

Didn't like the first answer, but the second is pretty cool--I hadn't messed with security managers and assumed they were more complicated than that. However, how do you test the security manager/testing mechanism. – Ursulaursulette

Make sure the tear down is executed properly, otherwise, your test will fail in a runner such as Eclipse because the JUnit application can't exit! :) – Cohette

I don't like the solution using the security manager. Seems like a hack to me just to test it. – Prophase

I like that solution but it doesn't work for me. I use Aaron M. Renn's getopt port which does a System.getProperty("gnu.posixly_correct", null) (Getopt.java, line 615). This causes a security exception to be thrown whenever the NoExitSecurityManager is installed. I also tried the more elaborate NoExitSecurityManager from the SystemRules library mentioned in another answer, but it also throws the exception. – Heathenism

@Heathenism the one from urbanophile.com/arenn/hacking/getopt then? Interesting. That would be best in a separate question (with a link to this question) in order to present what you have tried and the security exception it generates. – Corriecorriedale

@Corriecorriedale Don't agree with the first answer. Some Java programs are written to be executed by scripts which expect 0=success, 1=failure behaviour. AFAIK, the Java spec does not force a JVM to return a non-zero error code if an uncaught exception occurs. Second answer looks good though. – Nikolenikoletta

@DuncanJones re-reading docs.oracle.com/javase/specs/jvms/se7/html/… , you are right (on the lack of fixed exit status on a JVM ending because of unchecked exception) – Corriecorriedale

If you're using junit 4.7 or above, let a library deal with capturing your System.exit calls. System Rules - stefanbirkner.github.com/system-rules – Helle

@Helle interesting suggestion. I have included it in this (old) answer for more visibility. – Corriecorriedale

"Instead of terminating with System.exit(whateverValue), why not throw an unchecked exception?" -- because I'm using command line argument processing framework which calls System.exit whenever an invalid command line argument is supplied. – Ogilvy

Nothing against @Will, but this answer actually predates his comment, so I think its author should receive the credit: https://mcmap.net/q/116879/-how-to-test-methods-that-call-system-exit – Desmund

@MichaelScheper right, I have added the proper attribution in the answer. – Corriecorriedale

Voted down: don't like first point. I agree System.exit is not a good practice, however, not the answer for the question asked. don't like the second point. Messing up with java security manager is not in harmony with unit test. I like the third point, but it is just a copy & paste from Stefan Birkner answer below. His answer should be the accepted one, not this one. – Devout

I liked the solution to catch a System.exit(). I need to control the life of a web engine within my test framework, and the company security policies requires it to call exit() explicitly, so, I need to catch that. I added some extra verification in the strackTrace if the engine was the exit() caller, and it's plain good for my need. Voted up! But for testing purposes, Sytem Rules is a much more elegant approach. – Fivestar

Hi, My tests won't fail even if there is no system.exit() in the code after the exit.expectSystemExit(); line. Any ideas why that would happen? – Klaraklarika

@Klaraklarika 12 years later, I am not sure: you might want to ask a separate question, with your current 2020 Java and OS, which are certainly different from mine at the time I wrote this. – Corriecorriedale

For this to work with Java 21 this system property must be set -Djava.security.manager=allow – Nader

@EmmanuelBourg Good point, thank you for your feedback. I have included your comment in the answer for more visibility. – Corriecorriedale

Answer 2

The library System Lambda has a method catchSystemExit.With this rule you are able to test code, that calls System.exit(...):

public class MyTest {
    @Test
    public void systemExitWithArbitraryStatusCode() {
        SystemLambda.catchSystemExit(() -> {
            //the code under test, which calls System.exit(...);
        });
    }


    @Test
    public void systemExitWithSelectedStatusCode0() {
        int status = SystemLambda.catchSystemExit(() -> {
            //the code under test, which calls System.exit(0);
        });

        assertEquals(0, status);
    }
}

For Java 5 to 7 the library System Rules has a JUnit rule called ExpectedSystemExit. With this rule you are able to test code, that calls System.exit(...):

public class MyTest {
    @Rule
    public final ExpectedSystemExit exit = ExpectedSystemExit.none();

    @Test
    public void systemExitWithArbitraryStatusCode() {
        exit.expectSystemExit();
        //the code under test, which calls System.exit(...);
    }

    @Test
    public void systemExitWithSelectedStatusCode0() {
        exit.expectSystemExitWithStatus(0);
        //the code under test, which calls System.exit(0);
    }
}

Full disclosure: I'm the author of both libraries.

Kaolin

Perfect. Elegant. Didn't have to change a stitch of my original code or play around with the security manager. This should be the top answer! – Helle

This should be the top answer. Instead of endless debate on the proper use of System.exit, direct to the point. In addition, a flexible solution in adherence to JUnit itself so we don't need to reinvent the wheel or mess up with security manager. – Devout

@LeoHolanda Doesn't this solution suffer from the same "problem" you used to justify a downvote on my answer? Also, what about TestNG users? – Weisshorn

@Rogério Nope. With ExpectedSystemExit the code under test will exit when it finds the mocked System.exit call but will continue to run other tests. Furthermore, if exit is not called (with expected exit code), test will fail – Devout

@LeoHolanda You're right; looking at the implementation of the rule, I now see it uses a custom security manager which throws an exception when the call for the "system exit" check occurs, therefore ending the test. The added example test in my answer satisfies both requirements (proper verification with System.exit getting called/not called), BTW. The use of ExpectedSystemRule is nice, of course; the problem is it requires an extra 3rd-party library which provides very little in terms of real-world usefulness, and is JUnit-specific. – Weisshorn

@Stefan Birkner Unfortunately, ExpectedSystemExit doesn't work correctly in multi-threaded tests. – Archibold

SystemRules is really usefull (thanks for your code!!) . The only problem is that it seems to end the test. I was wondering if you there is any way exitRule.ignoreAndContinue(); that will allow my junit to make more verifications after. – Believe

There is exit.checkAssertionAfterwards(). – Kaolin

Does this work with Gradle and JUnit Jupiter? Gradle spawns a new JVM for tests and I always get the error: "Unexpected exception thrown. org.gradle.internal.remote.internal.MessageIOException: Could not read message from '/127.0.0.1:49215'." Providing all dependencies like junit-vintage-engine and junit:junit:4.12 and things run in general, but System.exit in the spawned JVM doesn't seem to be ignored. junit.org/junit5/docs/current/user-guide/… – Brandtr

this works flawlessly with latest junit5 release! Use this instead of System-rules which has some problems testing code for system exit. – Esurient

Answer 3

How about injecting an "ExitManager" into this Methods:

public interface ExitManager {
    void exit(int exitCode);
}

public class ExitManagerImpl implements ExitManager {
    public void exit(int exitCode) {
        System.exit(exitCode);
    }
}

public class ExitManagerMock implements ExitManager {
    public bool exitWasCalled;
    public int exitCode;
    public void exit(int exitCode) {
        exitWasCalled = true;
        this.exitCode = exitCode;
    }
}

public class MethodsCallExit {
    public void CallsExit(ExitManager exitManager) {
        // whatever
        if (foo) {
            exitManager.exit(42);
        }
        // whatever
    }
}

The production code uses the ExitManagerImpl and the test code uses ExitManagerMock and can check if exit() was called and with which exit code.

Radioman

+1 Nice solution. Also easy to implement if you are using Spring as the ExitManager becomes a simple Component. Just be aware that you need to make sure that your code doesn't carry on executing after the exitManager.exit() call. When your code is being tested with a mock ExitManager the code won't actually exit after the call to exitManager.exit. – Splice

Answer 4

You actually can mock or stub out the System.exit method, in a JUnit test.

For example, using JMockit you could write (there are other ways as well):

@Test
public void mockSystemExit(@Mocked("exit") System mockSystem)
{
    // Called by code under test:
    System.exit(); // will not exit the program
}

EDIT: Alternative test (using latest JMockit API) which does not allow any code to run after a call to System.exit(n):

@Test(expected = EOFException.class)
public void checkingForSystemExitWhileNotAllowingCodeToContinueToRun() {
    new Expectations(System.class) {{ System.exit(anyInt); result = new EOFException(); }};

    // From the code under test:
    System.exit(1);
    System.out.println("This will never run (and not exit either)");
}

Heinrik

Vote down reason: The problem with this solution is that if System.exit is not the last line in the code (i.e. inside if condition), the code will continue to run. – Devout

@LeoHolanda Added a version of the test which prevents code to run after the exit call (not that it really was a problem, IMO). – Weisshorn

Might it be more appropriate to replace the last System.out.println() with an assert statement? – Piggish

"@Mocked("exit")" doesn't seem to work anymore with JMockit 1.43: "The attribute value is undefined for the annotation type Mocked" The docs: "There are three different mocking annotations we can use when declaring mock fields and parameters: @Mocked, which will mock all methods and constructors on all existing and future instances of a mocked class (for the duration of the tests using it);" jmockit.github.io/tutorial/Mocking.html#mocked – Brandtr

Did you actually try your suggestion? I get: "java.lang.IllegalArgumentException: Class java.lang.System is not mockable" Runtime can be mocked, but doesn't stop System.exit at least in my scenario running Tests in Gradle. – Brandtr

@ThorstenSchöning That second test works with older versions of JMockit; in newer versions java.lang.System can no longer be mocked, but it can still be faked (with a MockUp). – Weisshorn

You can also use mockito-inline to mock System.exit(). The library System Stubs does this: github.com/webcompere/system-stubs – Biota

Answer 5

One trick we used in our code base was to have the call to System.exit() be encapsulated in a Runnable impl, which the method in question used by default. To unit test, we set a different mock Runnable. Something like this:

private static final Runnable DEFAULT_ACTION = new Runnable(){
  public void run(){
    System.exit(0);
  }
};

public void foo(){ 
  this.foo(DEFAULT_ACTION);
}

/* package-visible only for unit testing */
void foo(Runnable action){   
  // ...some stuff...   
  action.run(); 
}

...and the JUnit test method...

public void testFoo(){   
  final AtomicBoolean actionWasCalled = new AtomicBoolean(false);   
  fooObject.foo(new Runnable(){
    public void run(){
      actionWasCalled.set(true);
    }   
  });   
  assertTrue(actionWasCalled.get()); 
}

Homocercal

Is this what they call dependency injection? – Adenocarcinoma

This example as written is sort of half-baked dependency injection - the dependency is passed to the package-visible foo method (by either the public foo method or the unit test), but the main class still hardcodes the default Runnable implementation. – Homocercal

Answer 6

I like some of the answers already given but I wanted to demonstrate a different technique that is often useful when getting legacy code under test. Given code like:

public class Foo {
  public void bar(int i) {
    if (i < 0) {
      System.exit(i);
    }
  }
}

You can do a safe refactoring to create a method that wraps the System.exit call:

public class Foo {
  public void bar(int i) {
    if (i < 0) {
      exit(i);
    }
  }

  void exit(int i) {
    System.exit(i);
  }
}

Then you can create a fake for your test that overrides exit:

public class TestFoo extends TestCase {

  public void testShouldExitWithNegativeNumbers() {
    TestFoo foo = new TestFoo();
    foo.bar(-1);
    assertTrue(foo.exitCalled);
    assertEquals(-1, foo.exitValue);
  }

  private class TestFoo extends Foo {
    boolean exitCalled;
    int exitValue;
    void exit(int i) {
      exitCalled = true;
      exitValue = i;
    }
}

This is a generic technique for substituting behavior for test cases, and I use it all the time when refactoring legacy code. It not usually where I'm going to leave thing, but an intermediate step to get the existing code under test.

Cattier

This tecniques does not stop the conrol flow when the exit() has been called. Use an Exception instead. – Burleigh

Answer 7

For VonC's answer to run on JUnit 4, I've modified the code as follows

protected static class ExitException extends SecurityException {
    private static final long serialVersionUID = -1982617086752946683L;
    public final int status;

    public ExitException(int status) {
        super("There is no escape!");
        this.status = status;
    }
}

private static class NoExitSecurityManager extends SecurityManager {
    @Override
    public void checkPermission(Permission perm) {
        // allow anything.
    }

    @Override
    public void checkPermission(Permission perm, Object context) {
        // allow anything.
    }

    @Override
    public void checkExit(int status) {
        super.checkExit(status);
        throw new ExitException(status);
    }
}

private SecurityManager securityManager;

@Before
public void setUp() {
    securityManager = System.getSecurityManager();
    System.setSecurityManager(new NoExitSecurityManager());
}

@After
public void tearDown() {
    System.setSecurityManager(securityManager);
}

Museum

Answer 8

Create a mock-able class that wraps System.exit()

I agree with EricSchaefer. But if you use a good mocking framework like Mockito a simple concrete class is enough, no need for an interface and two implementations.

Stopping test execution on System.exit()

Problem:

// do thing1
if(someCondition) {
    System.exit(1);
}
// do thing2
System.exit(0)

A mocked Sytem.exit() will not terminate execution. This is bad if you want to test that thing2 is not executed.

Solution:

You should refactor this code as suggested by martin:

// do thing1
if(someCondition) {
    return 1;
}
// do thing2
return 0;

And do System.exit(status) in the calling function. This forces you to have all your System.exit()s in one place in or near main(). This is cleaner than calling System.exit() deep inside your logic.

Code

Wrapper:

public class SystemExit {

    public void exit(int status) {
        System.exit(status);
    }
}

Main:

public class Main {

    private final SystemExit systemExit;


    Main(SystemExit systemExit) {
        this.systemExit = systemExit;
    }


    public static void main(String[] args) {
        SystemExit aSystemExit = new SystemExit();
        Main main = new Main(aSystemExit);

        main.executeAndExit(args);
    }


    void executeAndExit(String[] args) {
        int status = execute(args);
        systemExit.exit(status);
    }


    private int execute(String[] args) {
        System.out.println("First argument:");
        if (args.length == 0) {
            return 1;
        }
        System.out.println(args[0]);
        return 0;
    }
}

Test:

public class MainTest {

    private Main       main;

    private SystemExit systemExit;


    @Before
    public void setUp() {
        systemExit = mock(SystemExit.class);
        main = new Main(systemExit);
    }


    @Test
    public void executeCallsSystemExit() {
        String[] emptyArgs = {};

        // test
        main.executeAndExit(emptyArgs);

        verify(systemExit).exit(1);
    }
}

Ramonramona

Answer 9

System Stubs - https://github.com/webcompere/system-stubs - is also able to solve this problem. It shares System Lambda's syntax for wrapping around code that we know will execute System.exit, but that can lead to odd effects when other code unexpectedly exits.

Via the JUnit 5 plugin, we can provide insurance that any exit will be converted to an exception:

@ExtendWith(SystemStubsExtension.class)
class SystemExitUseCase {
    // the presence of this in the test means System.exit becomes an exception
    @SystemStub
    private SystemExit systemExit;

    @Test
    void doSomethingThatAccidentallyCallsSystemExit() {
        // this test would have stopped the JVM, now it ends in `AbortExecutionException`
        // System.exit(1);
    }

    @Test
    void canCatchSystemExit() {
        assertThatThrownBy(() -> System.exit(1))
            .isInstanceOf(AbortExecutionException.class);

        assertThat(systemExit.getExitCode()).isEqualTo(1);
    }
}

Alternatively, the assertion-like static method can also be used:

assertThat(catchSystemExit(() -> {
   //the code under test
   System.exit(123);
})).isEqualTo(123);

Victorvictoria

@FrJeremyKrieg - as the author of that library, I can assure you that it doesn't. You can't apply mockito static mocking to System. – Victorvictoria

Sorry, I misread your blurb! I deleted my comment to avoid misleading anyone else! – Biota

@FrJeremyKrieg it's easily done... that said, the security manager is going away, so we're going to have to work out a new way of solving this problem in future. Feel free to come up with an idea and submit a PR. Always looking for new input on the library. – Victorvictoria

Any generic solution is probably going to involve code weaving or an agent, I think. Maybe JMockit (which I believe can mock System.exit()). – Biota

@FrJeremyKrieg github.com/jmockit/jmockit1 was last updated 2 years ago. I think it's not a go-forward mocking library. – Victorvictoria

Answer 10

A quick look at the api, shows that System.exit can throw an exception esp. if a securitymanager forbids the shutdown of the vm. Maybe a solution would be to install such a manager.

Admit

Answer 11

You can use the java SecurityManager to prevent the current thread from shutting down the Java VM. The following code should do what you want:

SecurityManager securityManager = new SecurityManager() {
    public void checkPermission(Permission permission) {
        if ("exitVM".equals(permission.getName())) {
            throw new SecurityException("System.exit attempted and blocked.");
        }
    }
};
System.setSecurityManager(securityManager);

Allerie

Hm. The System.exit docs say specifically that checkExit(int) will be called, not checkPermission with name="exitVM". I wonder if I should override both? – Math

The permission name actually seems to be exitVM.(statuscode), i.e. exitVM.0 - at least in my recent test on OSX. – Gossip

Answer 12

You can test System.exit(..) with replacing Runtime instance. E.g. with TestNG + Mockito:

public class ConsoleTest {
    /** Original runtime. */
    private Runtime originalRuntime;

    /** Mocked runtime. */
    private Runtime spyRuntime;

    @BeforeMethod
    public void setUp() {
        originalRuntime = Runtime.getRuntime();
        spyRuntime = spy(originalRuntime);

        // Replace original runtime with a spy (via reflection).
        Utils.setField(Runtime.class, "currentRuntime", spyRuntime);
    }

    @AfterMethod
    public void tearDown() {
        // Recover original runtime.
        Utils.setField(Runtime.class, "currentRuntime", originalRuntime);
    }

    @Test
    public void testSystemExit() {
        // Or anything you want as an answer.
        doNothing().when(spyRuntime).exit(anyInt());

        System.exit(1);

        verify(spyRuntime).exit(1);
    }
}

Carthage

Answer 13

There are environments where the returned exit code is used by the calling program (such as ERRORLEVEL in MS Batch). We have tests around the main methods that do this in our code, and our approach has been to use a similar SecurityManager override as used in other tests here.

Last night I put together a small JAR using Junit @Rule annotations to hide the security manager code, as well as add expectations based on the expected return code. http://code.google.com/p/junitsystemrules/

Mince

Answer 14

Most solutions will

• terminate the test (method, not the entire run) the moment System.exit() is called
• ignore an already installed SecurityManager
• Sometimes be quite specific to a test framework
• restrict to be used at max once per test case

Thus, most solutions are not suited for situations where:

• Verification of side-effects are to be performed after the call to System.exit()
• An existing security manager is part of the testing.
• A different test framework is used.
• You want to have multiple verifications in a single test case. This may be strictly not recommended, but can be very convenient at times, especially in combination with assertAll(), for example.

I was not happy with the restrictions imposed by the existing solutions presented in the other answers, and thus came up with something on my own.

The following class provides a method assertExits(int expectedStatus, Executable executable) which asserts that System.exit() is called with a specified status value, and the test can continue after it. It works the same way as JUnit 5 assertThrows. It also respects an existing security manager.

There is one remaining problem: When the code under test installs a new security manager which completely replaces the security manager set by the test. All other SecurityManager-based solutions known to me suffer the same problem.

import java.security.Permission;

import static java.lang.System.getSecurityManager;
import static java.lang.System.setSecurityManager;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.fail;

public enum ExitAssertions {
    ;

    public static <E extends Throwable> void assertExits(final int expectedStatus, final ThrowingExecutable<E> executable) throws E {
        final SecurityManager originalSecurityManager = getSecurityManager();
        setSecurityManager(new SecurityManager() {
            @Override
            public void checkPermission(final Permission perm) {
                if (originalSecurityManager != null)
                    originalSecurityManager.checkPermission(perm);
            }

            @Override
            public void checkPermission(final Permission perm, final Object context) {
                if (originalSecurityManager != null)
                    originalSecurityManager.checkPermission(perm, context);
            }

            @Override
            public void checkExit(final int status) {
                super.checkExit(status);
                throw new ExitException(status);
            }
        });
        try {
            executable.run();
            fail("Expected System.exit(" + expectedStatus + ") to be called, but it wasn't called.");
        } catch (final ExitException e) {
            assertEquals(expectedStatus, e.status, "Wrong System.exit() status.");
        } finally {
            setSecurityManager(originalSecurityManager);
        }
    }

    public interface ThrowingExecutable<E extends Throwable> {
        void run() throws E;
    }

    private static class ExitException extends SecurityException {
        final int status;

        private ExitException(final int status) {
            this.status = status;
        }
    }
}

You can use the class like this:

    @Test
    void example() {
        assertExits(0, () -> System.exit(0)); // succeeds
        assertExits(1, () -> System.exit(1)); // succeeds
        assertExits(2, () -> System.exit(1)); // fails
    }

The code can easily be ported to JUnit 4, TestNG, or any other framework, if necessary. The only framework-specific element is failing the test. This can easily be changed to something framework-independent (other than a Junit 4 Rule

There is room for improvement, for example, overloading assertExits() with customizable messages.

Adelaideadelaja

This is much the same implementation as both System Lambda and System Stubs use - github.com/webcompere/system-stubs#systemexit - the latter has a JUnit 5 plugin which can be used as a universal conversion of System.exit into a catchable error, thus stopping unexpected exits from stopping the test. – Victorvictoria

Answer 15

Use Runtime.exec(String command) to start JVM in a separate process.

Jampacked

How will you interact with the class under test, if it is in a separate process from the unit test? – Commotion

In 99% of the cases System.exit is inside a main method. Therefore, you interact with them by command line arguments (i.e. args). – Devout

Answer 16

There is a minor problem with the SecurityManager solution. Some methods, such as JFrame.exitOnClose, also call SecurityManager.checkExit. In my application, I didn't want that call to fail, so I used

Class[] stack = getClassContext();
if (stack[1] != JFrame.class && !okToExit) throw new ExitException();
super.checkExit(status);

Epstein

Answer 17

A generally useful approach that can be used for unit and integration testing, is to have a package private (default access) mockable runner class that provides run() and exit() methods. These methods can be overridden by Mock or Fake test classes in the test modules.

The test class (JUnit or other) provides exceptions that the exit() method can throw in place of System.exit().

package mainmocked;
class MainRunner {
    void run(final String[] args) {
        new MainMocked().run(args);    
    }
    void exit(final int status) {
        System.exit(status);
    }
}

the class with main() below, also has an altMain() to receive the mock or fake runner, when unit or integration testing:

package mainmocked;

public class MainMocked {
    private static MainRunner runner = new MainRunner();

    static void altMain(final String[] args, final MainRunner inRunner) {
        runner = inRunner;
        main(args);
    }

    public static void main(String[] args) {
        try {
          runner.run(args);
        } catch (Throwable ex) {
            // Log("error: ", ex);
            runner.exit(1);
        }
        runner.exit(0);
    } // main


    public void run(String[] args) {
        // do things ...
    }
} // class

A simple mock (with Mockito) would be:

@Test
public void testAltMain() {
    String[] args0 = {};
    MainRunner mockRunner = mock(MainRunner.class);
    MainMocked.altMain(args0, mockRunner);

    verify(mockRunner).run(args0);
    verify(mockRunner).exit(0);
  }

A more complex test class would use a Fake, in which run() could do anything, and an Exception class to replace System.exit():

private class FakeRunnerRuns extends MainRunner {
    @Override
    void run(String[] args){
        new MainMocked().run(args);
    }
    @Override
    void exit(final int status) {
        if (status == 0) {
            throw new MyMockExitExceptionOK("exit(0) success");
        }
        else {
            throw new MyMockExitExceptionFail("Unexpected Exception");
        } // ok
    } // exit
} // class

Mansour

Answer 18

Another technique here is to introduce additional code into the (hopefully small number of) places where the logic does the System.exit(). This additional code then avoids doing the System.exit() when the logic is being called as part of unit test. For example, define a package private constant like TEST_MODE which is normally false. Then have the unit test code set this true and add logic to the code under test to check for that case and bypass the System.exit call and instead throw an exception that the unit test logic can catch. By the way, in 2021 and using something like spotbugs, this problem can manifest itself in the obscure error that "java.io.IOException: An existing connection was forcibly closed by the remote host".

Yahrzeit

Answer 19

Calling System.exit() is a bad practice, unless it's done inside a main(). These methods should be throwing an exception which, ultimately, is caught by your main(), who then calls System.exit with the appropriate code.

Narcho

That doesn't answer the question, though. What if the function being tested IS ultimately the main method? So calling System.exit() might be valid and ok from a design perspective. How do you write a test case for it? – Revest

You shouldn't have to test the main method as the main method should just take any arguments, pass them to a parser method, and then kick start the application. There should be no logic in the main method to be tested. – Antiicer

@Elie: In these types of questions there are two valid answers. One answering the question posed, and one asking why the question was based. Both types of answers give a better understanding, and especially both together. – Altorelievo