Common Lisp style: multiple packages in same repo

Question

May I get recommendations or links to representative code repositories with good style for multiple related Common Lisp packages, please?

For instance, consider a high-level workflow library with accompanying lower-level API, each in its own CL package but same git repo due to synchronized releases.

Each system (*.asd file) isolates tests and may be invoked using:

(asdf:test-system foo :force t)

Separate systems may be built via make, which definitely helps isolate SBCL code-coverage reports.

Some users of the library may only want to load the lower-level API. For simplifying dependencies for those using higher-level API, it seems best to keep everything bundled in one repo. Any revision to one library would likely require updating all for the same release.

I currently have a single directory tree with a subdirectory for each CL package. There's a top-level Makefile plus one in each subdirectory that the library maintain would use. The top-level also contains symbolic links for .asd files pointing into relevant subdirectories. (It's a library deeply dependent upon POSIX calls via uiop-posix, so it's only applicable on an OS with sym-links.)

This seems to be an issue at large considering issue #1 for Quicklisp-docs [0].

Found nothing relevant in Google's CL style guide [1], State of the Common Lisp Ecosystem, 2015 [2], Edi's CL Recipes [3] or Lisp-lang [4]. Browsing repos seem to have quite a mix of styles.

• [0] https://github.com/rudolfochrist/quicklisp-docs/issues/1
• [1] https://google.github.io/styleguide/lispguide.xml
• [2] https://web.archive.org/web/20160305061250/http://eudoxia.me/article/common-lisp-sotu-2015/
• [3] http://weitz.de/cl-recipes/
• [4] http://lisp-lang.org/learn/writing-libraries but see also their section on Continuous Integration

Repo to be fixed: https://gitlab.com/dpezely/cl-mmap
(commit a23bd88d of 2018-07-14; release will be tagged when fixed)

Glaciate

In older times an application would have been a system plus optionally subsystems plus systems it uses. Say, a graphics editor GRAFED. It might have subsystems for editing, storing, printing, etc. It might use the systems COLOR and POSTSCRIPT. These systems are to organize the application/libraries into delivery parts (incl. files which are not Lisp files, like fasl files, c files, doc). Then we have any number of packages per system from 1 to n. These packages help to organize the APIs, restrict visibility or organize the language dialects used. So packages and systems had different purposes. – Parhe

Answer 1

You could consider using asdf-inferred-package. With that, you could have a mmap/high package that depends on a mmap/low package. With that setup, you can actually ask Quicklisp to load either of them directly:

(ql:quickload "mmap/high")

or

(ql:quickload "mmap/low")

You can see an example in my cl-bulk repo.

Microcopy

for quicklisp to find the kind of dependency you mention, is this constrained to only the local file system? More importantly, would it also work if the library gets into the Quicklisp distribution? – Glaciate

While cl-mmap system has only one real source file, the lower-level mmap system has several, so inferred packages wouldn't work there. I like the idea for other libraries, so thanks for highlighting that ASDF feature! – Glaciate

Inferred packages work very well when there are several files, look at the documentation! – Microcopy

This has nothing to do with the local filesystem. When ASDF or Quicklisp loads the ASDF system, they get the fact that it defines several systems. I guess including this in a Quicklisp distribution would just add all systems to its registry. – Microcopy

Answer 2

Attempting to reach a specific audience that might not have seen the question here, it was posted to Common Lisp Pro mailing list.

A summary of the various responses-- apart from great insights to various possible future directions-- there is no de facto convention, mechanism or style for addressing the combination of factors:

1. dependency synchronization across interrelated libraries/packages/systems
2. accommodating loading each individually
3. accommodating testing each individually
4. accommodating code-coverage reports of each individually

At time of adding this answer, the closest to a consistent, concrete, existing solution seems to align with what had already been implemented by the package mentioned in the original post-- or close enough. (There are of course subtle design and naming differences as indicated by the earlier answer here, but I see these variations as comparable.)

Highlights of packages and systems suggested as examples:

• An early implementation of CLIM (predating McCLIM) for its separation of API versus implementation
• Despite conventional use of ASDF systems and packages, explore how UIOP within ASDF itself is structured
• ASDF and LIL import and reexport all symbols in a directory; see Faré's full summary

Future directions and suggested reading included:

• Consider that intersection to be a software engineering question, and construct accordingly, because "In short: it depends!"
• Modules of Racket or Gerbil Scheme
• Perhaps internal updates to Google's CL style guide have added something relevant?

(Much thanks to Pascal J. Bourguignon, Ken Tilton, Scott McKay, Faré, Svante v. Erichsen, Don Morrison and Pascal Costanza for participating in the email thread.)

Glaciate