SBCL: building a standalone executable

Question

How do I build a standalone executable in SBCL? I've tried

; SLIME 2.20
CL-USER> (defun hullo ()
                     (format t "hullo"))
HULLO
CL-USER> (sb-ext:save-lisp-and-die "hullo" :toplevel #'hullo :executable t)

but that just produces the following error.

Cannot save core with multiple threads running.

Interactive thread (of current session):
  #<THREAD "main thread" RUNNING {10019563F3}>

Other threads:
  #<THREAD "Swank Sentinel" RUNNING {100329E073}>,
  #<THREAD "control-thread" RUNNING {1003423A13}>,
  #<THREAD "reader-thread" RUNNING {1003428043}>,
  #<THREAD "swank-indentation-cache-thread" RUNNING
     {1003428153}>,
  #<THREAD "auto-flush-thread" RUNNING {1004047DA3}>,
  #<THREAD "repl-thread" RUNNING {1004047FA3}>
   [Condition of type SB-IMPL::SAVE-WITH-MULTIPLE-THREADS-ERROR]

What am I doing wrong?

Aida

The error says you have multiple threads running. So you need to save without multiple threads running. Try the same directly at the repl (eg in a terminal) instead of in slime. – Padova

@DanRobertson, cool that works. But is this really the only way to compile an executable? Seems oddly cumbersome. – Aida

It is cumbersome. There are some ways to make it a bit easier. Saving images is a hard problem. There is a lot of state which one would like to save but it’s not obvious what to do about it. What happens to open files? Open sockets? What happens to your current working directory or command line arguments. What should happen to other running threads? If you have an open file and you save the core, then move the file, what should be the state of the stream when you restore the core? This wouldn’t be a problem for a running application as the inode for the file doesn’t move. Same for unlinking. – Padova

@DanRobertson, I'm not concerned about things that have been typed in slime, but I would have thought there would simply be a command-line option to produce an executable from a lisp source file. (Sorry I didn't make that very clear.) – Aida

You almost do that except instead of putting an extra command line argument, you have save-Lisp-and-die at the end of your source file. (Or put it in a function and --eval to call it) – Padova

Answer 1

What you are doing wrong is trying to save an image while multiple threads are running. Unlike many errors in Lisp the error message explains exactly what the problem is.

If you look up the function in the sbcl manual here then you find that indeed one may not save an image with multiple threads running. The extra threads come from swank (the CL half of SLIME). The manual says that you may add functions to *save-hooks* which destroy excess threads and functions to *init-hooks* to restore threads.

One way around all this is to not save the image when it is running through slime but instead to start sbcl directly at a terminal (note: no readline support), load your program and save from there.

Working with slime is different. In theory there is a SWANK-BACKEND:SAVE-IMAGE function but I’m not sure if that works. Also as saving an image kills the process you may want to fork (SB-POSIX:FORK) first, unless you are on Windows. But forking causes problems due to not being well specified and file descriptor issues (i.e. if you try fork->close swank connection->save and die then you may find that the connection in the parent process is closed (or worse, corrupted by appearing open but being closed at some lower level)). One can read about such things online. Note that due to the way sbcl threads are implemented, forking clones only the thread that forks and the other threads are not cloned. Thus forking and then saving should work but may cause problems when running the executable due to partial slime state.

You may be interested in buildapp.

If you want to be able to use slime with your saved application you can load swank and start listening on a socket or port (maybe with some command line argument) and then in Emacs you may connect to that swank backend with slime.

Padova

sbcli gives a repl with readline support (provided by cl-readline). Handy when needed. – Pylon

@Pylon I didn’t know that. That’s fantastic – Padova

Answer 2

You have to run save-lisp-and-die from a new sbcl, not from Slime. Dan Robertson explains more.

It is cumbersome the first time, but you can put it in a Makefile and re-use it. Don't forget to load your dependencies.

build:
    sbcl --load cl-torrents.asd \
         --eval '(ql:quickload :torrents)' \ 
         --eval '(use-package :torrents)' \ # not mandatory
         --eval "(sb-ext:save-lisp-and-die #p\"torrents\" :toplevel #'main :executable t)"

The quickload implies Quicklisp is already loaded, which may be the case if you installed Quicklisp on your machine, because then your ~/.sbclr contains quicklisp loading script ((load quicklisp-init)).

However sb-ext is not portable across implementations. asdf:make is the cross-platform equivalent. Add this in your .asd system definition:

:build-operation "program-op" ;; leave as is
:build-pathname "<binary-name>"
:entry-point "<my-package:main-function>"

and then call asdf:make to build the executable.

You can have a look at buildapp (mentioned above), a still popular app to do just that, for SBCL and CCL. It is in Debian. http://lisp-lang.org/wiki/article/buildapp An example usage looks like

buildapp --output myapp \
         --asdf-path . \
         --asdf-tree ~/quicklisp/dists \
         --load-system my-app \
         --entry my-app:main

But see also Roswell, a more general purpose tool, also supposed to build executables, but it is less documented. https://roswell.github.io/

If you want to build an executable on a CI system (like Gitlab CI), you may appreciate a lisp Docker image which has already SBCL, others lisps and Quicklisp installed, and if you want to parse command line arguments, see https://lispcookbook.github.io/cl-cookbook/testing.html#gitlab-ci and (my) tutorial: https://vindarel.github.io/cl-torrents/tutorial.html#org8567d07

Pylon