Module meta-language in Racket

Question

I'm trying to write in Racket a module meta-language mylang, which accepts a second language to which is passes the modified body, such that:

(module foo mylang typed/racket body)

is equivalent to:

(module foo typed/racket transformed-body)

where the typed/racket part can be replaced with any other module language, of course.

I attempted a simple version which leaves the body unchanged. It works fine on the command-line, but gives the following error when run in DrRacket:

/usr/share/racket/pkgs/typed-racket-lib/typed-racket/typecheck/tc-toplevel.rkt:479:30: require: namespace mismatch;
 reference to a module that is not available
  reference phase: 1
  referenced module: "/usr/share/racket/pkgs/typed-racket-lib/typed-racket/env/env-req.rkt"
  referenced phase level: 0 in: add-mod!

Here's the whole code:

#lang racket

(module mylang racket
  (provide (rename-out [-#%module-begin #%module-begin]))
  (require (for-syntax syntax/strip-context))
  (define-syntax (-#%module-begin stx)
    (syntax-case stx ()
      [(_ lng . rest)
       (let ([lng-sym (syntax-e #'lng)])
         (namespace-require `(for-meta -1 ,lng-sym))
         (with-syntax ([mb (namespace-symbol->identifier '#%module-begin)])
           #`(mb . #,(replace-context #'mb #'rest))))])))

(module foo (submod ".." mylang) typed/racket/base
  (ann (+ 1) Number))

(require 'foo)

---

Requirements (i.e. solutions I'd rather avoid):

• Adding a (require (only-in typed/racket)) inside the mylang module makes this work, but I'm interested in a general solution, where mylang does not need to know about typed/racket at al (i.e. if somebody adds a new language foo, then mylang should work with it out of the box).

• Also, I'm not interested in tricks which declare a submodule and immediately require and re-provide it, as is done here, because this changes the path to the actual module (so main and test loose their special behaviour, for example).

  It is also slower at compile-time, as submodules get visited and/or instantiated more times (this can be seen by writing (begin-for-syntax (displayln 'here)), and has a noticeable impact for large typed/racket programs.

• Bonus points if the arrows in DrRacket work for built-ins provided by the delegated-to language, e.g. have arrows from ann, + and Number to typed/racket/base, in the example above.

Murther

I am inclined to think this is a bug in Typed Racket -- but I don't know enough about the internals of Typed Racket to know for sure. @sam-tobin-hochstadt – Vivi

Note: I am basing this on the odd error location alone. – Vivi

Answer 1

One thing you can do, which I don't think violates your requirements, is put it in a module, fully expand that module, and then match on the #%plain-module-begin to insert a require.

#lang racket

(module mylang racket
  (provide (rename-out [-#%module-begin #%module-begin]))
  (define-syntax (-#%module-begin stx)
    (syntax-case stx ()
      [(_ lng . rest)
       (with-syntax ([#%module-begin (datum->syntax #f '#%module-begin)])
         ;; put the code in a module form, and fully expand that module
         (define mod-stx
           (local-expand
            #'(module ignored lng (#%module-begin . rest))
            'top-level
            (list)))
         ;; pattern-match on the #%plain-module-begin form to insert a require
         (syntax-case mod-stx (module #%plain-module-begin)
           [(module _ lng (#%plain-module-begin . mod-body))
            #'(#%plain-module-begin
                (#%require lng)
                .
                mod-body)]))])))

;; Yay the check syntax arrows work!
(module foo (submod ".." mylang) typed/racket/base
  (ann (+ 1) Number))

(require 'foo)

And if you wanted to transform the body in some way, you could do that either before or after expansion.

The pattern-matching to insert the extra (#%require lng) is necessary because expanding the module body in a context where lng is available isn't enough. Taking the mod-body code back out of the module form means that the bindings will refer to lng, but lng won't be available at run-time. That's why I get the require: namespace mismatch; reference to a module that is not available error without it, and that's why it needs to be added after expansion.

Update from comments

However, as @GeorgesDupéron pointed out in a comment, this introduces another problem. If lng provides an identifier x and the module where it is used imports a different x, there will be an import conflict where there shouldn't be. Require lines should be in a "nested scope" with respect to the module language so that they can shadow identifiers like x here.

@GeorgesDupéron found a solution to this problem in this email on the racket users list, using (make-syntax-introducer) on the mod-body to produce the nested scope.

(module mylang racket
  (provide (rename-out [-#%module-begin #%module-begin]))
  (define-syntax (-#%module-begin stx)
    (syntax-case stx ()
      [(_ lng . rest)
       (with-syntax ([#%module-begin (datum->syntax #f '#%module-begin)])
         ;; put the code in a module form, and fully expand that module
         (define mod-stx
           (local-expand
            #'(module ignored lng (#%module-begin . rest))
            'top-level
            (list)))
         ;; pattern-match on the #%plain-module-begin form to insert a require
         (syntax-case mod-stx (module #%plain-module-begin)
           [(module _ lng (#%plain-module-begin . mod-body))
            #`(#%plain-module-begin
                (#%require lng)
                .
                #,((make-syntax-introducer) #'mod-body))]))])))

Phosphorus

That's a nice solution! – Vivi

Thanks again! A subtle corner case I encountered: if foo requires a module (e.g. type-expander) which shadows some of the bindings exported by typed/racket/base, it will conflict with the inserted (#%require lng) (because module-language bindings can be shadowed by the (require type-expander), but required bindings cannot be shadowed by another require). Changing the inserted (#%require lng) to (#%require (only lng)) seems to be enough to have the module available, without actually importing any bindings which might clash with other required modules. – Murther

And I've changed it again based on your email on the list – Phosphorus