How do I run the tests that are part of an installed/installing Cabal package?

Question

I have a Haskell package I've installed from Hackage, using cabal and would like to run the tests suites that are part of the package, but it isn't clear to me from the cabal documentation how to do this.

I've tried:

cabal install --reinstall --enable-tests --run-tests the-package

and its various combinations and permutations, but no tests seem to run: I get no report about the test running, and none of the output that I know the test should produce is generated.

How do I run the tests that are part of an installed cabal package, or a package that I'm in the process of installing?

Sosthenna

Maybe cabal test the-package? – Vanhomrigh

@recursion.ninja: I tried that too but that seemed way off the mark: Package has never been configured. Configuring with default flags. If this fails, please run configure manually. cabal: No cabal file found. Please create a package description file <pkgname>.cabal – Sosthenna

You're right, it's not that simple. See my answer for a more robust solution. – Vanhomrigh

There may be a known bug. – Sosthenna

Answer 1

The --run-tests flag does not appear to be working in the current version of cabal. The --enable-tests flag no longer runs tests as a new feature of cabal. Until the issue is resolved you can manually verify that a package passes it's test suite by doing the following:

1. Use cabal to download the package source
2. Use cabal to build the package in a sandbox
3. Use cabal to run the tests in the sandbox

Use this series of cabal commands to run the test for the-package:

cabal get the-package
cd the-package*
cabal sandbox init
cabal install --dependencies-only
cabal configure --enable-tests
cabal build
cabal test
cd ../
rm -r the-package*

Or use this equivalent one-liner:

cabal get the-package && cd the-package* && cabal sandbox init && cabal install --dependencies-only && cabal configure --enable-tests && cabal build && cabal test && cd ../ && rm -r the-package*

Vanhomrigh

Yes, that part I get. But is there no way to simply run the tests on an installed package in situ? What do install --enable-tests and install --run-tests even do — nothing? – Sosthenna

@raxacoricofallapatorius Are you sure the package has a test suite in its .cabal file? I cabal install --run-tests on a package (tasty) which I know has no test suite associated with it and it did not output an failure/warning message that there were no test to run. It just silently finished. – Vanhomrigh

Yes, it's there. – Sosthenna

I just tried with cabal install megaparsec --run-tests which also has a test suite and it also finished without running tests. Very strange, maybe a cabal bug? – Vanhomrigh

Yeah, maybe. Certainly nothing even remotely like what's described in the documentation is happening. – Sosthenna

My answer should give you a method to run the tests independently and verify that they are all passing, but according to this GitHub issue conversation the --run-test flag should make the independent build & run unnecessary. – Vanhomrigh

Can I do the with no side effects. Just delete the directory after? – Sosthenna

Yup, just add a cd ../ && rm -r the-package* command at the end. The sandbox is in the directory. Delete the directory and the sandbox install is gone! – Vanhomrigh

I had saw that related issue. I hope this is resolved in the next version of cabal. They should really prioritize this fix because verifying an installation is a pretty desirable function of any package installer. – Vanhomrigh

Agreed! It's too bad, really. I'm just learning Haskell, and there are a lot of things about it and its ecosystem that are pretty (surprisingly) much in disarray (compared with, say, Python). – Sosthenna

This answer is outdated since sandboxes have disappeared from cabal. I don't know a good answer myself, see github.com/haskell/cabal/issues/7267. – Position

Answer 2

The answer below no long works as noted in its comments. The following works for me with cabal 3.6.2:

cabal get the-package
cd the-package*
cabal v2-test

Patent