Capturing Macros in Scheme
Asked Answered
C

3

7

What's the simplest way to define a capturing macro using define-syntax or define-syntax-rule in Racket?

As a concrete example, here's the trivial aif in a CL-style macro system.

(defmacro aif (test if-true &optional if-false)
    `(let ((it ,test))
        (if it ,if-true ,if-false)))

The idea is that it will be bound to the result of test in the if-true and if-false clauses. The naive transliteration (minus optional alternative) is

(define-syntax-rule (aif test if-true if-false)
    (let ((it test))
       (if it if-true if-false)))

which evaluates without complaint, but errors if you try to use it in the clauses:

> (aif "Something" (displayln it) (displayln "Nope")))
reference to undefined identifier: it

The anaphora egg implements aif as

(define-syntax aif
  (ir-macro-transformer
   (lambda (form inject compare?)
     (let ((it (inject 'it)))
       (let ((test (cadr form))
         (consequent (caddr form))
         (alternative (cdddr form)))
     (if (null? alternative)
         `(let ((,it ,test))
        (if ,it ,consequent))
         `(let ((,it ,test))
        (if ,it ,consequent ,(car alternative)))))))))

but Racket doesn't seem to have ir-macro-transformer defined or documented.

Cataldo answered 18/11, 2013 at 17:44 Comment(0)
T
11

Racket macros are designed to avoid capture by default. When you use define-syntax-rule it will respect lexical scope.

When you want to "break hygiene" intentionally, traditionally in Scheme you have to use syntax-case and (carefully) use datum->syntax.

But in Racket the easiest and safest way to do "anaphoric" macros is with a syntax parameter and the simple define-syntax-rule.

For example:

(require racket/stxparam)

(define-syntax-parameter it
  (lambda (stx)
    (raise-syntax-error (syntax-e stx) "can only be used inside aif")))

(define-syntax-rule (aif condition true-expr false-expr)
  (let ([tmp condition])
    (if tmp
        (syntax-parameterize ([it (make-rename-transformer #'tmp)])
          true-expr)
        false-expr)))

I wrote about syntax parameters here and also you should read Eli Barzilay's Dirty Looking Hygiene blog post and Keeping it Clean with Syntax Parameters paper (PDF).

Tori answered 18/11, 2013 at 18:27 Comment(9)
TBH, the blog post is not really relevant since the paper covers it more nicely and with more examples...Protuberate
@EliBarzilay Although I wanted to include the blog post for anyone who might be "OMG PDF!" or "OMG Research Paper!", I agree the paper is best.Tori
Yeah, I know that some people have that reaction -- it's just that the beginning parts in that paper are very accessible... But yeah, it does make a little value in having just a blog post...Protuberate
It's broken: (let ((it 'BAD)) (aif 'show-it (displayln it) 'other-thing)) displays BAD instead of show-it. The macro fails to capture it. What you're doing instead is relying on the equivalent of a global variable.Canaster
@ThrowawayAccount3Million As the last paragraph here explains: "The resulting macro does not break hygiene. For example, (let ([it 3]) (if #t it)) evaluates to 3, because it shadows the global it that if changes. This is a change from a real unhygienic macro — but that's the whole point: we (the macro author) do not interfere with scopes in the user code."Tori
@GregHendershott So you and Eli are already aware that it's broken. That doesn't mean it's not broken. Sometimes we want to interfere with scope. That's the whole point of capturing macros. As it happens, since my previous comment I've found a way to write a real capturing macro in Racket that involves walking down the syntax tree and stripping the scope (via datum->syntax composed with syntax->datum) from any identifier that has the same name as the one you want to capture.Canaster
When you truly need capture, or want capture for the sake of capture, you can use datum->syntax as I'd mentioned. With it, all things are possible, good or bad. But to back up: I wouldn't use aif for real code. I think a better idea would be something like if-let, where you supply the id. Usually it's clearer and more-dependable when users of the macro supply the identifier. (Obviously there are exceptions like struct, but at least that introduces names prefixed by a user-supplied id.)Tori
Example if-let.Tori
I agree that it's usually clearer and less bug-prone not to inject bindings into the scope of a user's code. In fact, that's easily 99% of the time. But I don't agree with the extremes to which R6RS has gone in this regard, where even the word else in a cond or case form is a bound identifier that cond can't see if there's a local binding named else. Every now and again you run into a situation where it's more convenient to be able to inject an identifier, and IMO aif is one of those cases. I'd have no qualms with using it in production code.Canaster
W
4

See Greg Hendershott's macro tutorial. This section uses anaphoric if as example:

http://www.greghendershott.com/fear-of-macros/Syntax_parameters.html

Winded answered 18/11, 2013 at 18:12 Comment(2)
Or better yet, see Greg Hendershott's answer. :)Bluma
In which Greg Hendershott ultimately refers to Eli Barzilay's blog post and paper... thank goodness we have tail elimination... :)Tori
C
1

Although the answer above is the accepted way to implement aif in the Racket community, it has severe flaws. Specifically, you can shadow it by defining a local variable named it.

(let ((it 'gets-in-the-way))
     (aif 'what-i-intended
          (display it)))

The above would display gets-in-the-way instead of what-i-intended, even though aif is defining its own variable named it. The outer let form renders aif's inner let definition invisible. This is what the Scheme community wants to happen. In fact, they want you to write code that behaves like this so badly, that they voted to have my original answer deleted when I wouldn't concede that their way was better.

There is no bug-free way to write capturing macros in Scheme. The closest you can come is to walk down the syntax tree that may contain variables you want to capture and explicitly strip the scoping information that they contain, replacing it with new scoping information that forces them to refer to your local versions of those variables. I wrote three "for-syntax" functions and a macro to help with this:

(begin-for-syntax
 (define (contains? atom stx-list)
   (syntax-case stx-list ()
     (() #f)
     ((var . rest-vars)
      (if (eq? (syntax->datum #'var)
               (syntax->datum atom))
          #t
          (contains? atom #'rest-vars)))))

 (define (strip stx vars hd)
   (if (contains? hd vars)
       (datum->syntax stx
                      (syntax->datum hd))
       hd))

 (define (capture stx vars body)
   (syntax-case body ()
     (() #'())
     (((subform . tl) . rest)
      #`(#,(capture stx vars #'(subform . tl)) . #,(capture stx vars #'rest)))
     ((hd . tl)
      #`(#,(strip stx vars #'hd) . #,(capture stx vars #'tl)))
     (tl (strip stx vars #'tl)))))

(define-syntax capture-vars
  (λ (stx)
     (syntax-case stx ()
         ((_ (vars ...) . body)
          #`(begin . #,(capture #'(vars ...) #'(vars ...) #'body))))))

That gives you the capture-vars macro, which allows you to explicitly name the variables from the body you'd like to capture. aif can then be written like this:

(define-syntax aif
  (syntax-rules ()
       ((_ something true false)
        (capture-vars (it)
           (let ((it something))
            (if it true false))))
       ((_ something true)
        (aif something true (void)))))

Note that the aif I have defined works like regular Scheme's if in that the else-clause is optional.

Unlike the answer above, it is truly captured. It's not merely a global variable:

 (let ((it 'gets-in-the-way))
     (aif 'what-i-intended
          (display it)))

The inadequacy of just using a single call to datum->syntax

Some people think that all you have to do to create a capturing macro is use datum->syntax on one of the top forms passed to your macro, like this:

(define-syntax aif
  (λ (stx)
     (syntax-case stx ()
       ((_ expr true-expr false-expr)
        (with-syntax
            ((it (datum->syntax #'expr 'it)))
            #'(let ((it expr))
                (if it true-expr false-expr))))
       ((_ expr true-expr)
        #'(aif expr true-expr (void))))))

Just using datum->syntax is only a 90% solution to writing capturing macros. It will work in most cases, but break in some cases, specifically if you incorporate a capturing macro written this way in another macro. The above macro will only capture it if the expr comes from the same scope as the true-expr. If they come from different scopes (this can happen merely by wrapping the user's expr in a form generated by your macro), then the it in true-expr will not be captured and you'll be left asking yourself "WTF won't it capture?"

You may be tempted to quick-fix this by using (datum->syntax #'true-expr 'it) instead of (datum->syntax #'expr 'it). In fact this makes the problem worse, since now you won't be able to use aif to define acond:

(define-syntax acond
    (syntax-rules (else)
        ((_) (void))
        ((_ (condition . body) (else . else-body))
         (aif condition (begin . body) (begin . else-body)))
        ((_ (condition . body) . rest)
         (aif condition (begin . body) (acond . rest)))))

If aif is defined using the capture-vars macro, the above will work as expected. But if it's defined by using datum->syntax on the true-expr, the the addition of begin to the bodies will result in it being visible in the scope of acond's macro definition instead of the code that invoked acond.

The impossibility of really writing a capturing macro in Racket

This example was brought to my attention, and demonstrates why you just can't write a real capturing macro in Scheme:

(define-syntax alias-it
  (syntax-rules ()
     ((_ new-it . body)
      (let ((it new-it)) . body))))

(aif (+ 1 2) (alias-it foo ...))

capture-vars cannot capture the it in alias-it's macroexpansion, because it won't be on the AST until after aif is finished expanding.

It is not possible at all to fix this problem, because the macro definition of alias-it is most probably not visible from the scope of aif's macro definition. So when you attempt to expand it within aif, perhaps by using expand, alias-it will be treated as a function. Testing shows that the lexical information attached to alias-it does not cause it to be recognized as a macro for a macro definition written out of scope from alias-it.

Some would argue that this shows why the syntax-parameter solution is the superior solution, but perhaps what it really shows is why writing your code in Common Lisp is the superior solution.

Canaster answered 2/5, 2015 at 22:52 Comment(7)
Downvoting not because of the silly flamewar you're trying to get into, but because it is just not an answer to the original question. There are three fundamental problems in your answer (and maybe in your understanding). This will be long so I'll split it into several comments.Protuberate
(A) Yes, using a syntax parameter means that the binding can be shadowed -- the reason that this is desirable is that the it syntax parameter is viewed not as a global, but as part of the aif macro -- so shadowing it is equivalent to shadowing aif, and that will (obviously) break it. (Note btw that it should be defined wherever aif is defined -- which means that it's as global as aif which, in racket, usually means that it's defined in some module (ie, it's still not "global").) This means that your criticism of syntax parameters is misplaced.Protuberate
(B) Your "writing a real capturing macro is impossible" conclusion is wrong. You can probably fix it by first expanding the input forms (eg, use local-expand). Doing that won't be new though: see the text on syntax parameters which mentions stripping all lexical information (and CL not keeping it in the first place). Also see Racket's define-macro which makes the simple CL-like aif definition work fine, and it should also work when used in acond and even with alias-it. (define-macro is doing what you're essentially trying to do, only in a more organized and methodical way.Protuberate
(C) Finally, the main problem is, again, that it's really not answering the question. If you really insist on this direction, point people at define-macro and be done with it. As for your long post, you should look into this more, phrase it better, and post it somewhere where such discussions are proper. The Racket mailing list is one such place -- and if you stick to the details instead of being offended, the result is surely going to be better. SO is just not a good place for such discussions, by design. This is also why these "comment-discussions" are hard to do, so I'll stop here.Protuberate
I'll look into it more, but I've had problems getting define-macro to do this kind of thing in the past, which is why I didn't mention it. The same kind of problems I mentioned in the answer, where the macro works only as long as it's not part of another macro. I gave up on it back when lists were still mutable.Canaster
Regarding the appropriateness if this site as a forum, I don't see anybody talking about capturing macros anywhere but here, and I'm only writing about them because the subject came up.Canaster
One possible problem wrt these macros is if you want to combine them with regular ones. My guess is that in case of macros-using-macros you'll want the whole chain done with define-macro. As for the discussion, there have been countless such discussions, on newsgroups, mailing lists, and blogs. The whole anaphoric macros thing is a very well chewed-up subject. That also applies to the realization that it is much better considered part of the anaphoric macro syntax rather than a new name that the macro injects, which makes syntax parameters work so well.Protuberate

© 2022 - 2024 — McMap. All rights reserved.