Evolving the ML Module SystemEvolving the ML Module System
Derek Dreyer
Toyota Technological Institute at Chicago
April 15, 2004
2
Data AbstractionData Abstraction
• Should be able to restrict how much clients of a program module know about its implementation
• Enforcement of program invariants
• Protect clients from implementation changes
3
Object-Oriented Approach (Java)Object-Oriented Approach (Java)
• Classes/objects encapsulate code with data
• Private fields/methods are inaccessible to clients
• Semantics of data abstraction tied up with other features of OOP– Inheritance, subtyping, dynamic dispatch
4
Module-Oriented Approach (ML)Module-Oriented Approach (ML)
• Modules are units of “core-language” code
• Interface of a module describes what other modules get to know about it
• Implementor-side data abstraction via sealing
• Client-side data abstraction via functors
5
Evolving the ML Module System Evolving the ML Module System
• Want to make the ML module system even better:– e.g. Add support for recursive modules
6
Evolving the ML Module System Evolving the ML Module System
• Want to make the ML module system even better:– e.g. Add support for recursive modules
• But where do we start?– Several variants of the ML module system:
• Standard ML, Objective Caml, Moscow ML, etc.
– Relationships/tradeoffs between them are unclear
7
The Goal of This WorkThe Goal of This Work
• Develop a unifying account of existing variations on the ML module system
• Build upon this foundation with support for new features (e.g. recursive modules)
8
OverviewOverview
• Concrete examples of data abstraction in ML• High-level analysis of data abstraction in ML• Extending ML with recursive modules• Future work
9
IntSet ModuleIntSet Module
• Module implementing sets of integers:
module IntSet =
mod
type set = int list
val emptyset : set = []
fun insert (x:int,S:set) : set = x::S
fun member (x:int,S:set) : bool = ...
...
end
10
Using the IntSet ModuleUsing the IntSet Module
• Clients use “dot notation” to refer to components of the IntSet module:
module IntSet = mod ... end
val S : IntSet.set =
IntSet.insert(3, IntSet.emptyset)
11
Using the IntSet ModuleUsing the IntSet Module
• Clients use “dot notation” to refer to components of the IntSet module:
module IntSet = mod ... end
val S : int list =
IntSet.insert(3, IntSet.emptyset)
12
Abstract IntSet InterfaceAbstract IntSet Interface
• Hide definition of set type in interface of IntSet:
interface INT_SET =
iface
type set
val emptyset : set
fun insert : int * set -> set
fun member : int * set -> bool
... end
13
Data Abstraction via SealingData Abstraction via Sealing
• Seal the implementation with the interface:
module IntSet =
(mod ... end) :> INT_SET
• Clients of IntSet can’t see definition of IntSet.set
• IntSet.set is an abstract type
14
Modules Can Have EffectsModules Can Have Effects
• IntSet module was purely functional– Body just defined values and functions
• But modules can have side effects– E.g. creation of mutable state
15
Symbol Table ModuleSymbol Table Module
• When evaluated, generates a new symbol table:
module SymbolTable =
mod
val table = HashTable.create(...)
type symbol = int
fun string_to_symbol (x:string) = ...
fun symbol_to_string (n:symbol) = ...
end
16
Sealing the Symbol TableSealing the Symbol Table
module SymbolTable =
(mod ... end) :>
iface
type symbol
fun string_to_symbol : string -> symbol
fun symbol_to_string : symbol -> string
...
end
17
Making Sets More GenericMaking Sets More Generic
• IntSet module only supports integer sets
• Implementation of sets basically the sameregardless of what the type of items in the set is
• Functors allow you to implement generic sets– Can be instantiated with different item types
– Similar to templates in C++, but more powerful
18
FunctorsFunctors
• A functor is a function from modules to modules:
Some Comparable
Item Type
Set Functor
Sets of
That Item
input output
19
Interface of Comparable ItemsInterface of Comparable Items
interface COMPARABLE =
iface
type item
fun compare : item * item -> bool
end
20
The Set FunctorThe Set Functor
module Set =
functor (Item : COMPARABLE) ->
mod
type set = Item.item list
fun member (x,S) =
...Item.compare(x,y)...
...
end
21
Applying the Set FunctorApplying the Set Functor
module IntItem =
mod
type item = int
fun compare(x,y) = Int.compare(x,y)
end
module IntSet = Set(IntItem)
22
Applying the Set FunctorApplying the Set Functor
module StringItem =
mod
type item = string
fun compare(x,y) = String.compare(x,y)
end
module StringSet = Set(StringItem)
23
Two Forms of Data AbstractionTwo Forms of Data Abstraction
• Can seal a module with an abstract interface– Implementor-side abstraction
• Can use functors to make a module more generic– Client-side abstraction
24
OverviewOverview
• Concrete examples of data abstraction in ML• High-level analysis of data abstraction in ML• Extending ML with recursive modules• Future work
25
Type Components of ModulesType Components of Modules
• Modules in ML have type components
• Can “project out” type components of modules:– e.g. IntSet.set, SymbolTable.symbol.
• In all the examples so far, we’ve only projected types out of module variables (or names)
26
Module ExpressionsModule Expressions
• Examples of module expressions:
mod ... end
(mod ... end) :> (iface ... end)
Set(IntItem)
27
QuestionQuestion
• Why not be able to project types from arbitrary module expressions?
mod ... end
(mod ... end) :> (iface ... end)
Set(IntItem)
28
ExampleExample
interface I =
iface type t ... end
module A =
(mod type t = int ... end) :> I
module B =
(mod type t = float ... end) :> I
29
Non-Projectible ModuleNon-Projectible Module
• Suppose M = if button_is_selected() then A else B
30
Non-Projectible ModuleNon-Projectible Module
• Suppose M = if button_is_selected() then A else B
module C = M
module D = M
31
Non-Projectible ModuleNon-Projectible Module
• Suppose M = if button_is_selected() then A else B
module C = M
module D = M
• If M.t is a valid type, then C.t = M.t = D.t • But C.t might be int and D.t might be float!!• Unsound for M to be projectible
32
Projectible ModuleProjectible Module
• Suppose M = mod type t = int; val x = 3 end
module C = M
module D = M
• Fine if M is projectible, since C.t = M.t = D.t = int
33
PurityPurity
• “Impure” module expression: if button_is_selected() then A else B
• “Pure” module expression: mod type t = int; val x = 3 end
34
PurityPurity
• “Impure” module expression: if button_is_selected() then A else B
• “Pure” module expression: mod type t = int; val x = ref 3 end
35
PurityPurity
• “Impure” module expression: if button_is_selected() then A else B
• “Pure” module expression: mod type t = int; val x = ref 3 end
• Sound for M to be projectible , M is pure (w.r.t. type components)
36
PlanPlan
• Consider how purity and projectibility relate to:– Sealing– Functors– The way that sealing and functors interact
37
SealingSealing
• Suppose M = (mod ... end) :> (iface type t ... end)
module C = M
module D = M
• Sealing has no run-time effect, so M is pure• But if M is projectible, then C.t = M.t = D.t• This violates abstraction!
38
Big PictureBig Picture
if button_is_selected() then A else B
Impure Modules (all non-projectible)
Pure Modules
Projectible Non-projectible
A mod
type t = int val x = ref 3
end
B
M :> I
39
FunctorsFunctors
• To track purity in the presence of functors:– Need to know whether applying a functor will
unleash an effect or not
• Distinguish types of total and partial functors:
– F : I1 ! I2 , body of F is pure
– F : I1 ! I2 , body of F is impure
tot
par
40
Total Total ,, Applicative Applicative
• F : I1 ! I2, M : 1, F and M are pure
module C = F(M)
module D = F(M)
• F(M) is pure ) projectible
C.t = F(M).t = D.t
tot
41
Partial Partial ,, Generative Generative
• F : I1 ! I2, M : 1, F and M are pure
module C = F(M)
module D = F(M)
• F(M) is impure ) non-projectible
C.t D.t
par
42
Set Functor ExampleSet Functor Example
module Set = functor (Item : COMPARABLE) ->
(mod ... end :>
iface
type set
fun member : Item.item * set -> set
...
end)
• Body is pure, so Set functor is total
43
Set Functor ExampleSet Functor Example
module Set = functor (Item : COMPARABLE) ->
(mod ... end :>
iface
type set
fun member : Item.item * set -> set
...
end)
module IntSet1 = Set(IntItem)
module IntSet2 = Set(IntItem)
• IntSet1.set and IntSet2.set are compatible
44
SymbolTable ModuleSymbolTable Module
module SymbolTable =
mod ... end :>
iface
type symbol
fun string_to_symbol : string -> symbol
fun symbol_to_string : symbol -> string
end
45
SymbolTable FunctorSymbolTable Functor
module SymbolTable = functor () ->
(mod ... end :>
iface
type symbol
fun string_to_symbol : string -> symbol
fun symbol_to_string : symbol -> string
end)
• Body is pure, so SymbolTable functor is total
46
SymbolTable FunctorSymbolTable Functor
module SymbolTable = functor () -> (mod ... end :> iface type symbol fun string_to_symbol : string -> symbol fun symbol_to_string : symbol -> string end)
module ST1 = SymbolTable()module ST2 = SymbolTable()
• But ST1.symbol and ST2.symbol are not compatible!
47
Pure vs. Impure SealingPure vs. Impure Sealing
if button_is_selected() then A else B
Impure Modules (all non-projectible)
Pure Modules
Projectible Non-projectible
A mod
type t = int val x = ref 3
end
B
M :> I
M :>> I Two forms
of sealing
48
SymbolTable Functor RevisitedSymbolTable Functor Revisited
module SymbolTable = functor () ->
(mod ... end :>>
iface
type symbol
fun string_to_symbol : string -> symbol
fun symbol_to_string : symbol -> string
end)
• Body is impure, so SymbolTable functor is partial
49
SummarySummary
• Analysis in terms of purity and projectibility
• Previous systems make “applicative” or “generative” a design choice, but we support both
• Previous systems just employ one or the other form of sealing and call it “sealing”
50
Unifying Previous SystemsUnifying Previous Systems
• Standard ML ‘97– Only has impure sealing, all functors are partial/generative
• Objective Caml / Leroy (1995)– Only has pure sealing, all functors are total/applicative
• Shao (1999)– Supports both total and partial functors
– Only has impure sealing, can’t write applicative Set functor
• Russo (2000)– Two languages, one like SML and one like O’Caml
– Moscow ML combines them, but language is unsound
51
OverviewOverview
• Concrete examples of data abstraction in ML• High-level analysis of data abstraction in ML• Extending ML with recursive modules• Future work
52
Recursive ModulesRecursive Modules
• Existing proposals fall into two categories:– “Units” or “Mixin modules” meant to replace ML modules
(Flatt-Felleisen 98, Ancona-Zucca 96, Duggan-Sourelis 98)
– Extend ML module system with recursive module construct(Crary et al. 99, Russo 01)
• Issues provoked by recursive module construct:– Recursion over general expressions with effects (see
[Dreyer 04] for details)
– Interaction of recursion and data abstraction
53
Recursive Module ExampleRecursive Module Example
rec (X : iface
module A : IA
module B : IB
end.
mod
module A = MA :> IA
module B = MB :> IB
end
)
54
Abstract InterfaceAbstract Interface
iface
module A : iface
type t
...
end
module B : iface
type u
fun g : A.t -> u
end
end
X :
55
Recursive Module BodyRecursive Module Body
mod
module A = mod
type t = int
fun f(x) = ... X.B.g(3) ...
end :> IA
module B = ...
end
56
The Two The Two A.tA.t’s’s
mod
module A = mod
type t = int
fun f(x) = ... X.B.g(3) ...
end :> IA
module B = ...
end
But X.B.g’s argument type is X.A.t,which is an abstract type!
57
Moscow ML SolutionMoscow ML Solution
iface
module A : iface
type t = int
...
end
module B : iface
type u
fun g : A.t -> u
end
end
X :
58
Our SolutionOur Solution
mod
module A = mod
type t = int
fun f(x) = ... X.B.g(3) ...
end :> IA
module B = ...
end
At this point, we know that A.t = int,so we know that X.A.t = int as well.
59
Our SolutionOur Solution
mod
module A = mod
type t = int
fun f(x) = ... X.B.g(3) ...
end :> IA
module B = mod ... end :> IB
end
At this point, A.t is abstract, so all we know is that X.A.t = A.t.
60
Current and Future WorkCurrent and Future Work
• Designed an ML dialect based on our analysis, following [Harper-Stone 97]– Currently implementing it in TILT compiler for SML
• Future Work:– Further extensions to ML module system
• E.g. type classes
– Generalizations of ML-style data abstraction
Thank you!Thank you!