Sun Jul 19 14:28:13 CEST 2009

Re: About 'eval at expansion time

Reply from Matthew Flatt[1].  Also, see at the end of this post.

The trick is to expand to `let-syntax'.  I feel I understand the basic
idea, but I have to think a bit more about this though.

I read the reply, slept over it, now I'm going to try to explain it.
The macro that does the magic is:

 (define-syntax (show stx)
   (syntax-case stx ()
     [(_ e)
      #'(let-syntax ([exp
                      (lambda (stx)
                        (with-syntax ([v e]) ;; (1)
                          #`(printf "~s yields ~s\n" 'e 'v)))]) ;; (2)

The `let-syntax' form[2] is quite straightforward.  It binds an
identifier to syntax.  The `with-syntax' form[3] is like `syntax-case'
but matches all patterns.  No magic there.

The magic happens by using `e' in two different positions.  One that
makes it verbatim into the eventual expansion of the macro (2) and one
that will be evaluated at expansion time (1).  Or: (1) is behind one
`syntax' and (2) is behind two, where one level of syntax quoting is
removed by the expansion, which serves the role of `eval' at expansion
time of `show'.

A neat trick indeed!

[1] http://list.cs.brown.edu/pipermail/plt-scheme/2009-July/034623.html
[2] http://docs.plt-scheme.org/reference/let.html#%28form._%28%28lib._scheme/private/letstx-scheme..ss%29._let-syntax%29%29
[3] http://docs.plt-scheme.org/reference/stx-patterns.html#%28form._%28%28lib._scheme/private/stxcase-scheme..ss%29._with-syntax%29%29

Matthew's full reply:

If I understand, then the following little program (in two modules) is
analogous to yours. It defines a `show' form that takes an expression
`e' to evaluate at compile time, and it prints the quoted form of `e'
followed by is compile-time result. The compile-time form can use a
`plus' binding, which stands for any sort of binding that you might
like to use during compile-time evaluation:


 ;; "arith-ct.ss"
 #lang scheme/base

 (define (plus a b)
   (+ a b))

 (define-namespace-anchor a)

 (define (show* e)
   (eval e (namespace-anchor->namespace a)))

 (provide show*)

 ;; use-arith.ss:
 #lang scheme

 (require (for-syntax "arith-ct.ss"))

 (define-syntax (show stx)
   (syntax-case stx ()
     [(_ e)
      (with-syntax ([v (show* #'e)])
        #`(printf "~s yields ~s\n" 'e 'v))]))

 (show (plus 1 2))
 ;; expands to (printf "~s yields ~s\n" '(plus 1 2) '3)


Here's how I'd implement the `show' form, instead:


 ;; "arith.ss"
 #lang scheme

 (define-for-syntax (plus a b)
   (+ a b))

 (define-syntax (show stx)
   (syntax-case stx ()
     [(_ e)
      #'(let-syntax ([exp
                      (lambda (stx)
                        (with-syntax ([v e])
                          #`(printf "~s yields ~s\n" 'e 'v)))])

 (show (plus 1 2))
 ;; expands to (printf "~s yields ~s\n" '(plus 1 2) '3)

 ;; To use `show' outside this module, we need to
 ;;  export `for-syntax' any bindings intended
 ;;  to be used within `show' expressions:
 (provide show (for-syntax plus))


This second implementation expands

 (show e)


 (let-syntax ([(exp) .... e ....])

where `e' is in an expression position in the right-hand size of the
`let-syntax', so it gets evaluated at expansion time. The main trick is
to invent a local name (in this case `exp') to house the expand-time
expression and trigger its evaluation in an expression position.

As you suggest, this approach requires a macro-generating `show', which
is in some sense a "higher order macro".

For a problem like this, I'd avoid `eval' because the other approach
composes better. Consider a module that imports `show', but also adds
its own compile-time extension `times':

 #lang scheme
 (require "arith.ss")

 (define-for-syntax (times a b) (* a b))

 (show (plus 1 (times 2 3)))

This wouldn't work if "arith.ss" were replaced by "use-arith.ss",
because the `eval' in "arith-ct.ss" wouldn't know where to find the
module that has the `times' binding. The problem is that module is in
the process of being compiled, and so it hasn't been registered for
reflective operations like `eval'.

Your example doesn't seem to involve any bindings like `plus' or
`times', and, offhand, I can't think of another concrete reason that
"arith.ss" is better than "arith-ct.ss" plus "use-arith.ss". Less
concretely, though, the reflection in the latter seems to me more
difficult to reason about. (As it turns out, I correctly predicted that
the `times' example would not work with "use-arith.ss", but I
mispredicted the specific reason.)

I hope I've understood your actual problem well enough that the above
examples are relevant.