Fast Haskell rebuild+test with file watch using cabal + GHCID?

Question

my question in short is "how to get a fast save-retest workflow in a cabal-managed multi-library haskell project repository?"

I already tried a few things and did some research. Before getting into more details please have a look at the typical project repo structure and then have the question broken down into more details:

Repository Development Structure

i work on multiple Haskell projects that usually have the following form:

.
├── foo
│   ├── foo.cabal
│   ├── src
│   ├── unit-test
│   └── ...
├── bar
│   ├── bar.cabal
│   ├── src
│   ├── unit-test
│   └── ...
├── baz
│   ├── baz.cabal
│   ├── src
│   ├── unit-test
│   └── ...
├── stack.yaml
├── cabal.project
├── nix
│   └── ...
└── ...

The cabal.project file looks like this:

packages: 
  foo
  bar
  baz
  ...
tests: True
run-tests: True

The stack file contains basically the same project list and an LTS ID, so i can just just the stackProject nix function from IOHK's haskell.nix to provide myself a nix shell which has cabal etc. in place. (This question is more about cabal handling so i consider this text paragraph here only a background note that i think is not relevant for this stack overflow question.)

This setup allows me to just run cabal test all anywhere in the project, which is great. This is my simple method to see if i broke anything before closing the next git commit.

Rapid Save-Retest Workflow

Before i got to nix, i used stack build/test --watch which was nice, because i could now have a shell open which always retests and rebuilds the whole project after i changed anything anywhere.

This can be simulated with inotify:

while true; do 
  inotifywait -e modify -r ./; 
  cabal test all
done

This is not really fast but it also does the job.

After i got to know about GHCID i was amazed by how blazingly fast it is. It is also easy to use with cabal repl.

Unfortunately (this problem was also mentioned but left unanswered in a comment here How to run test suite in ghcid with cabal?), GHCID can be run on one specific unit test suite and will not detect changes on the library that the unit tests are supposed to check. (Putting all the library modules into the unit test description in the cabal file is something that i consider an ugly hack and i woul rather like to avoid that)

Also, it seems i can't run GHCID on the whole repository like cabal test all or stack test --watch do. The extreme speed of GHCID is something that i really want in my workflow.

cabal is a tool that has existed long time before stack, how do people work on their multi-lib repositories to have a fast overview of all the test cases they broke after they edited multiple files in multiple libs? If the GHCID way does not work well, what is the way to do it?

Clywd

“how do people work on their multi-lib repositories...” – I personally did recently decide to ditch all the speciality tooling, just use vanilla nvim and manually send cabal test all to its terminal window whenever I want to check. Turns out I'm working more efficient when not relying on anything to happen automatically at all – that just keeps me waiting for the tab completion to come along or whatever, and the unpredictability means I'm not getting the best out of the editor commands themselves. – Sarinasarine

I think the most effective way to get performance is to do away with all the abstraction and just treat all the modules as one big library that is loaded directly into ghcid. You might be able to do that with either a .ghci file or a dummy library that combines all of them (including the test suite). – Colour

I'm going to try these suggestions: "FAQ: I want to run arbitrary commands when arbitrary files change..." github.com/ndmitchell/… – Moderator

Answer 1

My current workaround is to nest ghcid calls:

ghcid --target=$LIBRARY_NAME --run=":! ghcid --target=$TEST_SUITE --run" 

The innermost ghcid recompiles and reruns the tests when they change, the outermost recompiles the library and restarts the innermost one when the library source changes.

Octahedral

I'll be damned. That's the closest to "perfect" I've gotten. Not pretty, but seems to work well. Thank you. – Siloam

Answer 2

I use a script with stack as follows:

ghcid -c="stack ghci <test-suite>.hs" -T="main" --warnings $@

This means:

• Run ghcid
• Use stack ghci instead of vanilla ghci
• Also load the test suite module into ghci
• Run main (from the test suite) upon loading ghci
• Run the tests even if the compile generates GHC warnings

You could easily adapt this to use, for example, cabal repl instead of stack ghci.

This has the following drawbacks:

• The name of the test module needs to be hard-coded, and is not extracted from package.yaml/the cabal file.
• It does not support multiple test suites. These could all be passed to the script, but you'd need a custom main to call them all.

I previously got around these problems by using :!stack test as the command, which runs the all test suites, but since this is issued via the command line rather than loaded into ghci, The tests ran much slower. Additionally, It did not hot-reload on modification to the tests.

The bottom line is: to get the benefits of ghcid, ghcid needs to be informed of which source files to look at. Any reloading of ghcid via a shell command will require a full reload and recompile by ghci, not expoloiting ghci's fast hot reload capability. If this information is stored in a build system config file, your build system needs to integrate with ghcid (absent a custom script.) My guess is this is too low-level for cabal, but I have opened a feature request for stack. Add a comment or reaction there if you want to see this happen!

Kunlun

Answer 3

Since cabal-install >= 3.12, the multiple components into one repl session feature is available for broad adoption with GHC >= 9.4 (hence in cabal repl and ghcid) and haskell-language-server (HLS) >= 2.6:

This is a feature which allows you to load multiple components, which may depend on each other, into a single GHC session. For instance, you can load both a library and a testsuite into a single GHCi session, and :reload commands will pick up changes across both components.

Note that, as of July 2024, HLS >= 2.8 requires the following LSP configuration:

"sessionLoading": "multiComponent"

Portsalut