Expand a macro form completely

Question

I'd like to learn the internals of Lisp, so I want to see how everything is implemented.

For example,

(macroexpand '(loop for i upto 10 collect i))

gives me (in SBCL)

(BLOCK NIL
  (LET ((I 0))
    (DECLARE (TYPE (AND NUMBER REAL) I))
    (SB-LOOP::WITH-LOOP-LIST-COLLECTION-HEAD (#:LOOP-LIST-HEAD-1026
                                              #:LOOP-LIST-TAIL-1027)
      (SB-LOOP::LOOP-BODY NIL
                          (NIL NIL (WHEN (> I '10) (GO SB-LOOP::END-LOOP)) NIL)
                          ((SB-LOOP::LOOP-COLLECT-RPLACD
                            (#:LOOP-LIST-HEAD-1026 #:LOOP-LIST-TAIL-1027)
                            (LIST I)))
                          (NIL (SB-LOOP::LOOP-REALLY-DESETQ I (1+ I))
                           (WHEN (> I '10) (GO SB-LOOP::END-LOOP)) NIL)
                          ((RETURN-FROM NIL
                             (SB-LOOP::LOOP-COLLECT-ANSWER
                              #:LOOP-LIST-HEAD-1026)))))))

But LOOP-BODY, WITH-LOOP-LIST-COLLECTION-HEAD, etc. are still macros. How can I expand a macro form completely?

Armandinaarmando

Answer 1

To see the full expansion one needs to walk the Lisp form on all levels and expand them. For this it is necessary that this so-called code walker understands Lisp syntax (and not just s-expression syntax). For example in (lambda (a b) (setf a b)), the list (a b) is a parameter list and should not be macro expanded.

Various Common Lisp implementations provide such a tool. The answer of 6502 mentions MACROEXPAND-ALL which is provided by SBCL.

If you use a development environment, it is usually provided as a command:

• SLIME: M-x slime-macroexpand-all with C-c M-m

• LispWorks: menu Expression > Walk or M-x Walk Form, shorter M-Sh-m.

Wall

When portability is a concern, try John Fremlin's macroexpand-dammit, it works outside SBCL and does the same thing (though the macrolet definitions are apparently replaced with progn as unneeded). – Celestine

Answer 2

The other answers are excellent for you question but you say you want to see how everything is implemented.

Many macros (as you know already) are implemented using macros and whilst macroexpand-all is very useful but you can lose the context of what macro was responsible for what change.

One nice middle ground (if you are using slime) is to use slime-expand-1 (C-c Enter) which shows the expansion is another buffer. You can then useslime-expand-1 inside this new buffer to expand macros in-place. This allows you to walk the tree expanding as you read and also to use undo to close the expansions again.

For me this has been a god-send in understanding other people's macros. Hope this helps you too, have fun!

Sedgemoor

This is an excellent method. Slimv has this workflow too, just fyi for the vim+lisp users – Cantrell

Answer 3

You can try to use MACROEXPAND-ALL but what you may get is not necessarily useful.

In something like LOOP the real meat is the macro itself, not the generated code.

Baggott

Answer 4

(Note: If you're not interested in portability, SBCL provides macroexpand-all, which will do what you're after. If you're after a portable solution, read on...)

The quick-and-dirty solution would be to macroexpand the form itself, then recursively macroexpand all but the first element of the resulting list. This is an imperfect solution; it will fail completely the moment it attempts to process a let's bindings (the first argument to let, the list of bindings, is not meant to be macroexpanded, but this code will do it anyway).

;;; Quick-and-dirty macroexpand-all
(defun macroexpand* (form)
  (let ((form (macroexpand form)))
    (cons (car form) (mapcar #'macroexpand (cdr form)))))

A more complete solution would consider special forms specially, not macroexpanding their unevaluated arguments. I could update with such a solution, if desired.

Us