Binding a self reference via macros

Question

The project I am working on defines some complex structures that receive messages and run within their own thread. The structures are user-defined and transformed via macros to threads and runtime stuff. Roughly speaking we can say that a complex structure consists of some behaviour that implements the logic, and a procedure for spawning an instance of the behaviour. In the code below I have vastly simplified the situation, where a behaviour defined by the create-thread-behaviour macro is a simple thunk that can be spawned via the spawn macro. I'd like to implement the ability for (an instance of) a behaviour to send messages to itself via a self parameter that would be bound to (current-thread) (~ the thread that is running the behaviour).

I've tried to rig something up using syntax-parameterize but for some reason cannot get it to work. The code below implements a simple application that should clarify what I want to achieve - the special point of interest is the (unimplemented) <self> reference towards the bottom.

#lang racket
(require (for-syntax syntax/parse))

(define-syntax (create-thread-behaviour stx)
  (syntax-parse stx
    [(_ body:expr ...+)
     #'(λ () body ...)]))

(define-syntax (spawn stx)
  (syntax-parse stx
    [(_ behaviour:id)
     #'(thread behaviour)]))


(define behaviour
  (create-thread-behaviour
   (let loop ()
     (define message (thread-receive))
     (printf "message: ~a~n" message)
     (thread-send <self> "And this is crazy.")
     (loop))))

(define instance (spawn behaviour))
(thread-send instance "Hey I just met you")

So the thing with syntax parameters that I tried is the following, which raises the self-defined "can only be used in a behaviour" error. I know I have correctly used syntax parameters before, but perhaps I have just been looking at the problem for too long.

(require racket/stxparam)

(define-syntax-parameter self
  (lambda (stx) (raise-syntax-error (syntax-e stx) "can only be used in a behaviour")))

(define-syntax (spawn stx)
  (syntax-parse stx
    [(_ behaviour:id)
     #'(thread
        (lambda ()
          (syntax-parameterize ([self #'(current-thread)])
            (behaviour))))]))

Treytri

Could you post a (non-working) attempt that uses syntax-parameterize? That’s definitely the right tool for the job here, and it would probably be more helpful to point out what you’re missing than to reimplement all of it. – Bron

You're right! Give me a minute :) – Treytri

Answer 1

You’re right that syntax parameters seem like the right tool for the job here. There are two issues here around your use of them, however, that are causing the issue. Let’s take them one at a time.

First of all, syntax parameters are semantically just syntax transformers, as you can see by your initial use of define-syntax-parameter, which binds the syntax parameter to a function. Your use of syntax-parameterize, in contrast, binds the syntax parameter to a piece of syntax, which is wrong. Instead, you need to bind it to a syntax transformer as well.

An easy way to achieve the behavior you’re looking for is to use the make-variable-like-transformer function from syntax/transformer, which makes a syntax transformer that, as the name would imply, behaves like a variable. More generally, though, it actually produces a transformer that behaves like an expression, which (current-thread) is. For that reason, your use of syntax-parameterize should actually look like this:

(require (for-syntax syntax/transformer))

(syntax-parameterize ([self (make-variable-like-transformer #'(current-thread))])
  (behaviour))

This will avoid the “bad syntax” errors when attempting to use self after it has been parameterized.

However, there’s another problem in your code, which is that it appears to use a syntax parameter like a normal, non-syntax parameter, when syntax parameters don’t work like that. Normal parameters are effectively dynamically scoped, so the use of syntax-parameterize to wrap (behavior) would adjust self within the dynamic extent of the call to behavior.

Syntax parameters, however, don’t work like that. In fact, they can’t: Racket is syntactically a lexically scoped language, so you really can’t have a dynamic syntax binding: all syntax transformers are expanded at compile time, so adjusting a binding during the dynamic extent of a call is impossible. Syntax parameters are entirely lexically scoped, they simply hygienically adjust a binding within a particular scope. In that sense, they are really just like let, except that they adjust an existing binding rather than produce a new one.

With this consideration in mind, it becomes clear that putting the syntax-parameterize form in spawn can’t really work, because behavior is lexically defined outside of spawn. You could just move the use of syntax-parameterize to create-thread-behavior, but now there’s another problem, which is that this wouldn’t work:

(define (behavior-impl)
  (define message (thread-receive))
  (printf "message: ~a~n" message)
  (thread-send self "And this is crazy.")
  (behavior-impl))

(define behaviour
  (create-thread-behavior
   (behavior-impl)))

Now, once again, self is used outside of the lexical extent of syntax-parameterize, so it won’t be bound.

You’ve mentioned that this is a simplified example of what you’re actually doing, so maybe your real example requires a more complicated solution. If so, you may just need to require that self is only bound within the lexical extent of create-thread-behavior. However, your current use of self is remarkably simple, and in fact, it never changes: it’s always (current-thread). For that reason, you could actually just ditch syntax parameters entirely and define self directly:

(define-syntax self (make-variable-like-transformer #'(current-thread)))

Now self will work everywhere as a variable-looking reference to the value of a parameter, current-thread. This might be what you actually want, since it allows the value of self to be truly dynamically scoped (since it uses runtime parameters, not syntax parameters), but it still makes it look like a variable instead of a function.

Bron

This is an amazing answer! It's not the first time you have given me an amazing answer either! Thank you very much! The use of self in the real code is almost as simple as the example, except that more logic is involved, including some contexts where self cannot be defined. Is it a difficult task to disallow using self outside of the lexical context of create-thread-behaviour? A solution could be to use a regular parameter to determine whether self is used in the right context (if so, return current thread, otherwise raise error). I imagine it can be enforced at compile-time as well? – Treytri

@Treytri If you want to disallow self outside of the lexical extent of create-thread-behavior, you can use a syntax parameter and wrap syntax-parameterize around the body of the expansion of create-thread-behavior, quite similar to what you were already doing. If you want to disallow self outside the dynamic extent of create-thread-behavior, you’ll have to use a normal parameter and do a runtime check, since the dynamic extent inherently cannot be known at compile time. – Bron