How to properly save Common Lisp image using SBCL?

Question

If I want to create a Lisp-image of my program, how do I do it properly? Are there any prerequisites? And doesn't it play nicely with QUICKLISP?

Right now, if I start SBCL (with just QUICKLISP pre-loaded) and save the image:

(save-lisp-and-die "core")

And then try to start SBCL again with this image

sbcl --core core

And then try to do:

(ql:quickload :cl-yaclyaml)

I get the following:

To load "cl-yaclyaml":
  Load 1 ASDF system:
    cl-yaclyaml
; Loading "cl-yaclyaml"
.......
debugger invoked on a SB-INT:EXTENSION-FAILURE in thread
#<THREAD "main thread" RUNNING {100322C613}>:
  Don't know how to REQUIRE sb-sprof.
See also:
  The SBCL Manual, Variable *MODULE-PROVIDER-FUNCTIONS*
  The SBCL Manual, Function REQUIRE

Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [RETRY                        ] Retry completing load for #<REQUIRE-SYSTEM "sb-sprof">.
  1: [ACCEPT                       ] Continue, treating completing load for #<REQUIRE-SYSTEM "sb-sprof"> as having been successful.
  2:                                 Retry ASDF operation.
  3: [CLEAR-CONFIGURATION-AND-RETRY] Retry ASDF operation after resetting the configuration.
  4: [ABORT                        ] Give up on "cl-yaclyaml"
  5:                                 Exit debugger, returning to top level.

(SB-IMPL::REQUIRE-ERROR "Don't know how to ~S ~A." REQUIRE "sb-sprof")
0] 

Alternatively, if I try:

(require 'sb-sprof)

when sbcl is started with saved core, I get the same error. If sbcl is started just as sbcl there is no error reported.

In fact, pre-loading QUICKLISP is not a problem: the same problem happens if sbcl is called initially with sbcl --no-userinit --no-sysinit.

Am I doing it wrong?

PS. If I use roswell, ros -L sbcl-bin -m core run somehow doesn't pick up the image (tested by declaring variable *A* before saving and not seeing it once restarted).

PS2. So far what it looks like is that sbcl does not provide extension modules (SB-SPROF, SB-POSIX, etc.) unless they are explicitly required prior saving the image.

Awakening

Maybe you want to describe the problem. For example by providing a reproducible test case and a copy of an actual error message. – Hypnotic

@RainerJoswig I've expanded the question. I hope it provides more information. – Awakening

One more update: it seems this is the feature of SBCL: CCL's SAVE-APPLICATION works as expected. Does SBCL do some tree-shaking when saving the image? – Awakening

What OS and what version of SBCL are you using? I can't reproduce the problem on Linux + SBCL 1.3.2. – Pindus

@Pindus Ubuntu + SBCL 1.3.2 as well. But, considering it is in /usr/local/bin/, it is not the one provided by the distribution. I think it was a binary from SBCL website. – Awakening

@Pindus @RainerJoswig I found the problem! SBCL's core is looking for contrib directory with extra modules (including SB-SPROF, SB-POSIX etc.) in the same directory where the core is. And if they are not pre-loaded at the time of saving the image, SBCL doesn't put them into the image. I wonder if this is the behaviour of a particular SBCL version or all of them? – Awakening

Do you have a SBCL_HOME environment variable set? – Pindus

Actually, I don't. And now I see (section 1.2 of installation instructions) that SBCL primarily looks for contrib in SBCL_HOME or where the image resides. Thanks! – Awakening

I found it is possible to save result of the (sb-int:sbcl-homedir-pathname) into a global variable and to restore it when image started, using sb-ext:*init-hooks* ``` (defvar home (sb-int:sbcl-homedir-pathname)) (defun restore-home () (setf sb-sys::*sbcl-homedir-pathname* home)) (pushnew 'restore-home sb-ext:*init-hooks*) ``` – Improvisatory

Oh, here is example code with load hook: gist.github.com/svetlyak40wt/00ceaa4118609555d1630e3d500050b6 – Improvisatory

Answer 1

Thanks for the help from @jkiiski here is the full explanation and solution:

1. SBCL uses extra modules (SB-SPROF, SB-POSIX and others) that are not always loaded into the image. These module reside in contrib directory located either where SBCL_HOME environment variable pointing (if it is set) or where the image resides (for example, in /usr/local/lib/sbcl/).

2. When an image is saved in another location and if SBCL_HOME is not set, SBCL won't be able to find contrib, hence the errors that I saw.

3. Setting SBCL_HOME to point to contrib location (or copying contrib to image location or new image to contrib location) solves the problem.

4. Finally, about roswell: roswell parameter -m searches for images in a specific location. For SBCL (sbcl-bin) it would be something like ~/.roswell/impls/x86-64/linux/sbcl-bin/1.3.7/dump/. Secondly, the image name for SBCL must have the form <name>.core. And to start it, use: ros -m <name> -L sbcl-bin run. (Quick edit: better use ros dump for saving images using roswell as it was pointed out to me)

Awakening

What worked for me was export SBCL_HOME=/usr/lib/sbcl/ (the one printed by (sb-int:sbcl-homedir-pathname) and not /usr/local/bin). – Arette

Answer 2

If you want to create executables, you could try the following:

(sb-ext:save-lisp-and-die 
  "core"
  :compression t
  ;; this is the main function:
  :toplevel (lambda () 
              (print "hell world")                                      
              0)
  :executable t)

With this you should be able to call QUICKLOAD as you wish. Maybe you want to checkout my extension to CL-PROJECT for creating executables: https://github.com/ritschmaster/cl-project

Seriema

:COMPRESSION doesn't work on mine (not build with zlib support). I don't want standalone executable now. And I want sbcl to drop into REPL, adding toplevel function makes it to execute and exit sbcl. – Awakening