cabal test in a sandbox

Question

Let's say I have three of my own packages, A B & C, with dependencies on lots of additional packages in Hackage. I'm using cabal 1.18.

• C depends on A & B.
• B depends on A.
• A & B have test suites.

I set up the sandbox like this:

cd /path/to/sandbox
cabal sandbox init
cabal sandbox add-source /path/to/A
cabal sandbox add-source /path/to/B
cabal sandbox add-source /path/to/C

I want to build all the packages, run all test suites on my packages but not dependency packages, showing full test output. What's the best way to do that?

Option 1

cd /path/to/sandbox
cabal install --enable-tests A B C

Problems:

• There's no way to pass --show-details=always to cabal install.
• Test output is hidden in a log file and not shown.
• If the user happened to do cabal install A earlier, A won't get rebuilt and the tests won't get run.

Option 2

cd /path/to/A
cabal sandbox init --sandbox=/path/to/sandbox/.cabal-sandbox
cd /path/to/B
cabal sandbox init --sandbox=/path/to/sandbox/.cabal-sandbox

cd /path/to/A
cabal configure --enable-tests
cabal test --show-details=always
cd /path/to/B
cabal configure --enable-tests
cabal test --show-details=always
cabal install C

Problems:

• This causes A and B libraries to be unnecessarily rebuilt.

Option 3

In the sandbox cabal.config, add the line tests: True.

Problems:

• This will cause tests to run for all dependent packages from Hackage, which is very slow and fails in some cases.

Kerril

Your best option today is to create one sandbox per package, add each package's deps to its own sandbox and then cd in to each sandbox and run the tests with cabal test. – Temporal

That's somewhat awkward as we're actually using sandboxes as "build flavours", with optimisation and other flags turned on/off etc. – Kerril

Answer 1

Cabal is really missing functionality here. My plan is to generalize cabal so it has less (or no) concept of a "current package". Right now lots of commands assume that you're in a directory with a .cabal file and you want to do thing to that package. This is less often the case for large, multi-package projects as you've seen.

What I want is for cabal to take a list of targets for most commands, such as build, test, bench, etc. You can the run tests from several packages by

cabal test --show-details=always \
  pkg-dir1:some-test1 pkg-dir1:some-test2 pkg-dir2

(The above example shows that it should be possible to specify just some sections of a package as well.)

I realize that this doesn't help you much now, but at least you know which direction we're moving in.

Temporal