Why does Jest --runInBand speed up tests?

Question

I read that the --runInBand flag speeds up Jest test duration by 50% on CI servers. I can't really find an explanation online on what that flag does except that it lets tests run in the same thread and sequentially.

Why does running the test in the same thread and sequentially make it faster? Intuitively, shouldn't that make it slower?

Swing

Answer 1

Reading your linked page and some other related sources (like this github issue) some users have found that:

...using the --runInBand helps in an environment with limited resources.

and

... --runInBand took our tests from >1.5 hours (actually I don't know how long because Jenkins timed out at 1.5 hours) to around 4 minutes. (Note: we have really poor resources for our build server)

As we can see, those users had improvements in their performances on their machines even though they had limited resources on them. If we read what does the --runInBand flag does from the docs it says:

Alias: -i. Run all tests serially in the current process, rather than creating a worker pool of child processes that run tests. This can be useful for debugging.

Therefore, taking into consideration these comments and the docs, I believe the improvement in performance is due to the fact that now the process runs in a single thread. This greatly helps a limited-resource-computer because it does not have to spend memory and time dealing and handling multiple threads in a thread pool, a task that could prove to be too expensive for its limited resources.

However, I believe this is the case only if the machine you are using also has limited resources. If you used a more "powerful" machine (i.e.: several cores, decent RAM, SSD, etc.) using multiple threads probably will be better than running a single one.

Threonine

I'm on a i5-3570 @ 3.40GHz with 16 GB RAM + SSD and I saw a dramatic performance improvement using --runInBand - It's not a state of the art computer, but I'm not sure I would call it a limited resource computer. Just doesn't make any sense to me why runInBand is faster. – Remus

Be aware that while --runInBand might also slow down runs, eg. if you have many test suites it might be better to use --maxWorkers or --maxParallel and set it to the number of cores/threads. In our Jenkins CI (4 cores, 8Gb) we had unit tests 50 mins with --runInBand which went down to 20 mins after setting it to --maxParallel=4. Without any of those settings, it just died out of memory (OOM) because it spawend to many threads (which both methods prevented). – Erotica

@Erotica thanks for the feedback and sharing your experience. The downside I still see to that is debugging... as we know, debugging threads and parallel stuff can be tedious and running in a single thread greatly helps when debugging. And, as in the case OP was facing and the sources they were referring to, ruuninband greatly helped those that couldn't afford multiple cores or had very restricted resources.... if you don't have restricted resources then by all means use all the power you can get :) – Threonine

@Threonine totally agree that run in band makes sense if you have only 1 core. Just wanted to highlight it is not the holy grail, especially not if you have 2 or more cores and point out an alternative. – Erotica

What a badly named option. --singleThread might have been clearer. Even on an 8-core i9, 32Gb MacBookPro, this option consistently brought test suite time down from 30s to 23s. – Rightward

Also consider the cost of launching multiple child worker processes. In some tightly managed environments, simply starting a new process can take a few seconds due to security and virus scans. Sometimes the cost of spawning multiple processes outweighs the cost launching one and running the tests in a serial fashion. – Tournament

For those of us using Nx or want some other advice. Nx has this nx.dev/nx-api/jest/documents/overview#performance-in-ci on their site. Seems to be our problem currently. – Maxson

Answer 2

When you run tests in multi-threads, jest creates a cache for every thread. When you run with --runInBand jest uses one cache storage for all tests.

I found it after runs 20 identical tests files, first with key --runInBand, a first test takes 25 seconds and next identical tests take 2-3s each.

When I run tests without --runInBand key, each identical test file executes in 25 seconds.

Coth

So would it be beneficial to use --runInBand when running tests locally to speed up the test run? We are using jest --coverage --runInBand --no-cache for Continuous Integration. – Weigand

You should also add the --ci flag to codeepic's command. jestjs.io/docs/en/cli#ci – Celandine

could you please kindly point to the place in source code or in documentation that confirms claim about cache per thread? – Pinkerton