Upload
margaretmargaret-barnett
View
214
Download
0
Embed Size (px)
Citation preview
A Theory ofHygienic Macros
David Herman, Mitchell WandNortheastern University
2
A Theory of Hygienic Macros
Introduction to macros and hygiene
The challenge of formalizing hygiene
m and its properties
3
In the beginning MACDEF SUM A,B,C LOAD A ADD B STORE CENDDEF
4
The essence of macros
Derived (user-defined) syntactic constructs Defined by rewriting to existing constructs Translated at compile-time (“macro expansion”)
repeat s until e
begin s ; while (not e) s end end
5
The essence of macros
Derived (user-defined) syntactic constructs Defined by rewriting to existing constructs Translated at compile-time (“macro expansion”)
(repeat s until e)
(begin s (while (not e) s))
6
Example: short-circuit or
(or (foo "bar") (baz 42))
(define-macro (or e1 e2) (with (t e1) (if t t e2)))
evaluate only if (foo "bar") produces #f
evaluate first; evaluate only once
pattern; matches subexpressions
e1 and e2
template; rewrites expression using
e1 and e2
7
The power of macros
Abstractions for control, scope, program structure Powerful meta-programming idioms Simplifies language core (“extensible languages”)
functions, recursion, iteration, primitive datatypes
local bindings, custom loop forms, test harnesses, data
conditionals,constructors,
regularexpressions,
languagedesigner
library writer
little lang-uages, …
short-circuit operators,
8
A brief history of hygiene
9
Naïve macro expansion
(define-macro (or e1 e2) (with (t e1) (if t t e2)))
(with (x 1) (or #f x))(with (x 1) (with (t #f) (if t t x)))
(with (t 1) (or #f t))(with (t 1) (with (t #f) (if t t t)))
10
Naïve macros are leaky abstractions(define-macro (or e1 e2) (with (t e1) (if t t e2)))
A leaky abstraction is a brittle abstraction.
Please don’t refer to any variablesnamed t in expression e2!
Oh, and don’t use or in a contextthat shadows the names with or if!
11
For reasons of hygiene it will always be assumed that the bound variables that occur in a certain expression are different from the free ones. This can be fulfilled by renaming bound variables.
—H. P. Barendregt
Kohlbecker: This calls for hygiene
12
Hygienic macro expansion
(define-macro (or e1 e2) (with (t e1) (if t t e2)))
(with (x 1) (or #f x))(with (x 1) (with (t #f) (if t t x)))
(with (t 1) (or #f t))(with (t 1) (with (t′ #f) (if t′ t′ t)))
13
Hygiene and syntactic abstraction
functions, recursion, iteration, primitive datatypes
libraries
functions, recursion, iteration, primitive datatypes
local bindings, custom loop forms, test harnesses, data
conditionals,constructors,
regularexpressions,little lang-uages, …
short-circuit operators,
language
14
Hygiene and syntactic abstraction
functions, recursion, iteration, primitive datatypes
custom loop forms, derived datatypes, first-class module systems, object-orientation, test harnesses, data
constructors, laziness, short-circuit boolean operators, iteration protocols,
static analyses, parser generators, partial evaluation, debuggers,
coroutines, exceptions, threads, type checkers,
introspection, etc…
local bindings, software contracts,
language
libraries
15
What do we know about hygiene?
16
Hygiene by example
(define-macro (or e1 e2) (with (t e1) (if t t e2)))
(with (x 1) (or #f x))(with (x 1) (with (t #f) (if t t x)))
(with (t 1) (or #f t))(with (t 1) (with (t #f) (if t t t)))
17
Hygiene by informal rules
Kohlbecker et al, ’86:
Generated identifiers that become binding instances in the completely expanded program must only bind variables that are generated at the same transcription step.
Clinger and Rees, ’91:
1. It is impossible to write a macro that inserts a binding that can capture references other than those inserted by the macro.
2. It is impossible to write a macro that inserts a reference that can be captured by bindings other than those inserted by the macro.
SRFI 72:
A binding for an identifier can only capture a reference to another if both were present in the source or introduced during a single evaluation of a syntax or quasisyntax form, with the understanding that the evaluation of any nested, unquoted syntax or quasisyntax forms counts as part of the evaluation of an enclosing quasisyntax.
18
Hygiene by implementation
19
But which implementation?
?
Kohlbecker et al, ’86 Clinger & Rees ’91 Dybvig et al, ’93 Van Tonder, ’05
20
Insight #1: Lexical scope
(define-macro (or e1 e2) (with (t e1) (if t t e2)))
(with (x 1) (or #f x))(with (x 1) (with (t #f) (if t t x)))
(with (t 1) (or #f t))(with (t 1) (with (t′ #f) (if t′ t′ t)))
=
=
21
Hygienic macro expansionpreserves -equivalence.
The essence of hygienic macros
22
What variables are bound by with?(define-macro (with (a e1) e2) )
(with (x 1) (or #f x))
(with (t 1) (or #f t))
=
…
23
What is the binding structure?
(define-macro (murky a e) (begin (set! a e) (lambda (a) e)))
(lambda (foo) (murky foo (+ foo 1)))
24
What is the binding structure?
(define-macro (turing-machine e)
)
25
What is the binding structure?
(define-macro (indecisive a e) … (set! a e) … (lambda (a) e))
(lambda (foo) (indecisive foo (+ foo 1)))
26
What variables are bound by with?(define-macro (with (a e1) e2) …) )
Standard answer: look at results of expansion
(with (x 1) (or #f x))((lambda (x) ((lambda (t) (if t t x)) #f) 1)
(with (t 1) (or #f t))((lambda (t) ((lambda (t′) (if t′ t′ t)) #f) 1)
…
27
Using expansion to define scope
To specify hygienic expansion, we need to know the binding structure (scope) of programs.
To specify the binding structure of programs, we need to know the results ofhygienic expansion.
Oops.
28
Inside every large language is a small language struggling to get out…
—Igarashi, Pierce, and Wadler
29
Insight #2: Specifying lexical scope
30
What variables are bound by with?(define-macro (with (a e1) e2) …) )
In practice, programmers write binding specifications:
with takes the form (with (a e1) e2) and produces an expression, where:
a is an identifier e1 and e2 are expressions a is bound in e2
…
31
A shape type for with
(with (<a> expr) expra) → expr
binder
expression in same
environment
expression in scope of a
32
Explicitly type-annotated macros
let val x = 1 in … x …end
(with (x 1) … x …)
(define-macro (with (a e1) e2) : expr ((a : var) (e1 : expr) (e2 : expr a)) )
…
…
33
Shape-directed -conversion
(with (<a> expr) expra) → expr
(with (x 1) (or #f x))
(with (t 1) (or #f t))
=
34
Shape-directed -conversion
(with (<a> expr) expra) → expr
(with (z 1) (or #f z))
(with (t 1) (or #f t))
=
35
Shape-directed -conversion
(with (<a> expr) expra) → expr
(with (z 1) (or #f z))
(with (z 1) (or #f z))
=
36
Quick recap
Macro shape types binding structure Binding structure -equivalence -equivalence correctness of hygienic
expansion
So, what of hygienic expansion?
37
m: a model of hygienic macros
(with (x 1) x)
(x.x) 1
S-expressions
compiler’s core language
38
m: a model of hygienic macros
e ::= var | var.e | e e| let syntax var = m in e end| (op s …)
core language
source
39
Hygienic macro expansion
let syntax x = m in e end → e[m/x]
if BV(e) ∩ FV(m) =
((macro p => e) s …) → (e)
if (p) = s for some substitution and BV(s) ∩ FV(e) = and BV(e) # s
40
Properties of m
41
Confluence
(with (x 1) (or #f x))
(x.(with (t #f) (if t t x))) 1
(x.(or #f x)) 1(with (x 1) (with (t #f) (if t t x)))
42
Hygiene
For (well-typed) e0 = e′0 and fully-expanded e and e′, if e0 →
* e and e′0 →
* e′, then e = e′.
e0
e′0
e
e′
*
*
43
Hygiene…from confluence
* *
s
s1 s′1
s′
44
Hygiene…from confluence
* *
s0
s1 s′1
s2
s′0
s′2
45
Contributions
Semantics for hygienic expansion, accommodating: Lexically scoped macros Binding forms of arbitrary shape
Formal definitions and proofs -equivalence in presence of macros Shape-directed -conversion Confluence and hygiene
46
What we’ve gained
Formalized programmers’ mental model of the binding structure of macros.
Showed the model has good theoretical properties.
Validated intuition: hygienic macro expansion frees the programmer to -convert with impunity.
But—more features yet to model…
47
Thank you.
Hygienic macro expansionpreserves -equivalence.
Moral: