Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
1
Layered Indexed EffectsFoundations and Applications of Effectful Dependently Typed Programming
ASEEM RASTOGI,Microsoft Research, India
GUIDO MARTÍNEZ, CIFASIS-CONICET, ArgentinaAYMERIC FROMHERZ,Microsoft Research & CMU, USA
TAHINA RAMANANANDRO,Microsoft Research, USA
NIKHIL SWAMY,Microsoft Research, USA
Programming and reasoning about effects in a functional programming language is a long-standing research
problem. While monads have been a widely adopted common basis for effects in general purpose languages
like Haskell, to improve modularity, compositionality, and precision of static reasoning, many refinements
and alternatives have been developed, including algebraic effects, graded monads, parameterized monads,
Hoare monads, Dijkstra monads, productors, polymonads, and combinations thereof, to only name a few.
To benefit from all these approaches, we propose a new language feature, layered indexed effects, andimplement it as an extension to the F
★programming language. Using our feature, programmers can extend
F★with user-defined effect disciplines in all the aforementioned styles. Layered indexed effects provide a
type-and-effect directed elaboration algorithm which allows programs to be developed in applicative syntax
and to be reasoned about in a programmer-chosen abstraction, hiding the underlying semantic representations
that justify a given effect-typing discipline. Being dependently typed, F★supports developing both the semantic
models and reasoning about client programs within the same framework.
We evaluate our work on four case studies: a generic framework for operation-indexed algebraic effects and
handlers, together with the derivation of a Hoare-style program logic to reason about their stateful behavior;
a new graded monad for stateful information flow control; a library of verified low-level message formatters
structured as a parameterized monad upon a Hoare monad; and a hierarchy of effects mixing graded monads,
Hoare monads, polymonads and continuations, to structure programs and proofs in a concurrent separation
logic. Additionally, layered indexed effects have also simplified the core of F★, allowing us to derive basic
Dijkstra monad constructions in F★that were previously primitive. Given our positive experience, we speculate
that other dependently typed languages might benefit from layered indexed effects too.
1 INTRODUCTIONMonads have been a remarkably effective way to structure and reason about programs with
computational effects [Moggi 1989; Wadler 1992]. Over the years, a great many computational
effects have been shown to be expressible as monads, including state, exceptions, continuations,
parsing, printing, asynchrony, and many others besides. Still, monads are far from the final word
on effectful functional programming, particularly when using multiple effects.
Aiming to improve modularity, compositionality and precision, various refinements and alter-
natives to monads have been proposed. For instance, Plotkin and Power [2003] propose algebraic
effects as a generic semantic basis for effects, and many others [Bauer and Pretnar 2015; Brady
2013; Leijen 2017; Lindley et al. 2017; Plotkin and Pretnar 2009] have developed programming
languages and libraries to program compositionally with algebraic effects and handlers. To support
formal reasoning about effectful computations, others have proposed refining monads with various
indexing structures, including parameterized monads [Atkey 2009], graded monads [Katsumata
Authors’ addresses: Aseem Rastogi, Microsoft Research, India; Guido Martínez, CIFASIS-CONICET, Argentina; Aymeric
Fromherz, Microsoft Research & CMU, USA; Tahina Ramananandro, Microsoft Research, USA; Nikhil Swamy, Microsoft
Research, USA.
2018. 2475-1421/2018/1-ART1 $15.00
https://doi.org/
Proc. ACM Program. Lang., Vol. 1, No. CONF, Article 1. Publication date: January 2018.
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
1:2 Aseem Rastogi, Guido Martínez, Aymeric Fromherz, Tahina Ramananandro, and Nikhil Swamy
2014], Hoare monads [Nanevski et al. 2008], Dijkstra monads [Swamy et al. 2013], productors [Tate
2013], and polymonads [Hicks et al. 2014].
While each of these frameworks offer attractive benefits over vanilla monads for both program-
ming and proving, and several languages support various subsets of them, no single framework
supports them all. What is missing is a shared structure which captures the commonalities of all
these constructions, e.g., something akin to Wadler’s classic monad interface and accompanying
“do-notation” that makes programming with monads practical. The main contribution of this paper,
layered indexed effects, is such a structure: it allows defining a wide range of indexed monads to
encode a variety of effect disciplines, while offering a direct programming style with automatic
inference and elaboration into custom effectful semantics. By embedding layered indexed effects
inside F★[Swamy et al. 2016], a dependently typed programming language and proof assistant,
user-defined effect disciplines can also be used to structure formal proofs.
1.1 Layered Indexed Effects: A First ExampleFor a first example, consider the indexed monad gst (a:Type) (t:tag), with tag = R | RW shown below,
defining either a reader or a read-write state monad, with actions to read and write the state.
let state = int
type tag = R | RW
let (⊕) t0 t1 = match (t0, t1) with | (R, R) → R | _ → RW
let gst (a:Type) (t:tag) = match t with R→ (state→ a) | RW→ (state→ a ∗ state)
let return (x:a) : gst a R = _s → x
let bind t0 t1 (f:gst a t0) (g: a → gst b t1) : gst b (t0 ⊕ t1) = ...
let read : gst state R = _s→ s
let write (s:state) : gst unit RW = __→ (), s
This structure is an instance of Katsumata’s (2014) graded monad—a monad-like structure indexed
by a monoid𝑀 whose return and bind are specified in the indices by the identity and operator of the
monoid𝑀 . In this case, the monoid in question is (tag, ⊕, R). The tag index on gst provides useful
static reasoning about stateful programs, notably, gst a R computations do not modify the state.
However, programming with gst isn’t trivial. Consider incrementing the state: one would write
it as bind read (_x → write (x + 1)), while hoping for type inference to instantiate many missing
implicit arguments. If one were interested in graded monads only, one could develop libraries to
support elaboration and inference for them—but, the diversity of effect typing disciplines would
require a proliferation of ad hoc approaches to cater to their specific needs.
Automated elaboration, subsumption, and abstraction. Using layered indexed effects, we can turn
the gst monad into a user-defined effect, as shown below. Doing so introduces a new indexed
computation type GST corresponding to gst. The computation type GST classifies a family of effects
represented by gst—in this case, read and read-write stateful computations. Actions in gst can be
reflected as actions in GST, in the spirit of Filinski’s (1999) monadic reflection and layered monads.
effect { GST (a:Type) (t:tag) = gst a t with return; bind }
let read () : GST state R = GST?.reflect (_ s→ s)
let write (s:state) : GST unit RW = GST?.reflect (_ _ → (), s)
Having introduced the GST effect, F★ can automatically typecheck effectful terms written in direct
style, and internally elaborate them into the more verbose monadic syntax. For instance, the term
write (read () + 1) is inferred to have the computation type GST unit RW and internally elaborated into
the explicit monadic form shown earlier. Optionally, programmers can also specify a subsumption
relation for an effect, to enable transparently reuse of, say, reader computations (with tag R) in
Proc. ACM Program. Lang., Vol. 1, No. CONF, Article 1. Publication date: January 2018.
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
1:3
contexts expecting read-write computations; branching computations with different indices can
also be implicitly lifted and composed. Further, effect definitions can be layered on top of each
other, e.g., we could add an additional layer to represent exceptions on top of the GST effect. Finally,
being embedded in F★’s dependent type theory, parametric polymorphism over the effects within
an effect family is also easily supported, as is refining GST with Hoare-style specifications.
1.2 ContributionsF★, prior to our work, provided only various flavors of Dijkstra monads (a monad-indexed monad)
to reason about effectful programs. While Dijkstra monads, like other Hoare-style specification
mechanisms chosen in other dependently typed languages [Charguéraud 2011; Chlipala et al. 2009;
Letan and Régis-Gianas 2020; Nanevski et al. 2008], are attractive for their generality, one size
doesn’t always fit all: it can be easier to work with domain-specific abstractions rather than general-
purpose ones. Layered indexed effects allow programmers to build abstractions of their choice.
But that’s not all – they also simplify the foundations of F★. Indeed, one of our core contributions
is to show that Dijkstra monads need no longer be a primitive notion in F★and can instead be a
user-defined monad-indexed layered effect.
Elaborating effects into a core calculus of dependent refinement types (§3). Explicitly Monadic
F★, or EMF
★, a core calculus for F
★defined by Ahman et al. [2017], no longer needs its monadic
parts. Instead, we simplify the core of F★to a subset of EMF
★, which we call TotalF
★, a pure type
system with refinement types. Based on this smaller core, we design LIF★, a surface language with
user-defined layered indexed effects, and a simple type-and-effect directed elaboration of LIF★
programs into TotalF★. Theorem 3.1 proves that the elaboration of LIF
★to TotalF
★is well-typed.
We have also implemented layered indexed effects in F★and used them to express several new
effect typing disciplines enabling new applications—we present four case studies.1
Algebraic effect handlers and Dijkstra monads (§2.1, §2.2). We develop a library of generic algebraic
effects and handlers, using a graded monad to track the operations a computation may invoke. The
library supports generic handling and interpreting into a semantics of the user’s choosing. We
show how to interpret heap-manipulating programs into both a computational semantics and a
predicate transformer semantics, deriving a Dijkstra monad (via a construction due to Maillard
et al. [2019]) for stateful algebraic effect programs, which seamlessly integrate with F★’s SMT-based
automated proving engine.
Stateful information flow control (§2.3). We present a new graded monad for information flow
control (IFC) in stateful computations. Our approach is more precise than prior approaches to
monadic static information flow control and internalizes the proof of noninterference. Using the
abstraction of layered indexed effects, client programs can be verified for noninterference while
reasoning only on IFC label orderings, instead of relational proofs on the underlying semantics.
Binary message formatting (§4). We improve support for low-level message formatting in Ev-
erParse [Ramananandro et al. 2019], a library of monadic parser and formatter combinators. By
layering a parameterized monad on top of EverParse’s existing use of a Hoare monad for C programs,
we obtain more concise programs and better proof automation, while still supporting extraction to
C code, with the abstraction of layered indexed effects imposing no runtime cost.
Concurrent separation logic (§5). Finally, we develop a hierarchy of effect layers including two
graded Hoare polymonads, two non-graded Hoare polymonads, and two further effect layers for
1We submit our formal development and code for all the case studies as anonymous supplementary material.
Proc. ACM Program. Lang., Vol. 1, No. CONF, Article 1. Publication date: January 2018.
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
1:4 Aseem Rastogi, Guido Martínez, Aymeric Fromherz, Tahina Ramananandro, and Nikhil Swamy
capturing continuations, to structure and simplify programs and proofs in Steel [Swamy et al. 2020],
an embedding of concurrent separation logic in F★.
The diversity of our experience encourages us to conclude that layered indexed effects are a
simple but powerful language mechanism which allows programmers to easily define layered
effect-typing disciplines in abstractions of their choosing, and enjoy a direct programming style
with automatic inference and elaboration into custom effectful semantics.
2 ENCODING EFFECT TYPING DISCIPLINESThis section introduces layered indexed effects and F
★with three main examples. First, we review
Dijkstra monads and show how they no longer need to be primitive in F★. Next, we develop a library
of generic algebraic effects and handlers and provide two typing disciplines for them: first, a graded
monad recording unhandled actions and then, a Dijkstra monad for verifying programs using a
mutable heap. Finally, we present an effect discipline for stateful information flow control. In all
cases, the abstraction and elaboration mechanisms offered by layered indexed effects allow client
programs to be written in native, direct-style syntax, and verified in a user-chosen abstraction,
hiding details of the underlying semantic representations.
As such, layered indexed effects generalizes F★both “upwards” and “downwards”: upwards, by
allowing more interesting indexing structures that were previously out of reach; and downwards,
by removing Dijkstra monads from its core, a welcome simplification for a theorem prover.
2.1 Introducing F★, Dijkstra Monads, and Layered Indexed EffectsF★is a program verifier and a proof assistant based on a dependent type theory with a hierarchy
of predicative universes (like Coq or Agda). F★also has a dependently typed metaprogramming
system inspired by Lean and Idris (called Meta-F★[Martínez et al. 2019]) that allows using F
★itself
to build and run tactics for constructing programs or proofs. More specific to F★is its effectful type
system, extensible with user-defined effects, and its use of SMT solving to automate some proofs.
To date, effects in F★have been tied to Dijkstra monads [Ahman et al. 2017; Maillard et al. 2019;
Swamy et al. 2013]—no longer, as we will soon see.
Basic Syntax. F★ syntax is roughly modeled on OCaml (val, let, match, etc.) though with differ-
ences due to the additional typing features. Binding occurrences b of variables take the form x:t,
declaring a variable x at type t; or #x:t indicating that the binding is for an implicit argument. The
syntax _(b1) ... (b𝑛) → t introduces a lambda abstraction, whereas b1 → ...→ b𝑛 → c is the shape of a
curried function type. Refinement types are written b{t}, e.g., nat = x:int{x≥ 0} represents natural
numbers. We define squash t as the unit refinement _:unit{t}, which can be seen as the type of
(computationally irrelevant) proofs of t. As usual we omit the type in a binding when it can be
inferred; and for non-dependent function types, we omit the variable name. For example, the type
of the append function on vectors is #a:Type → #m:nat→ #n:nat→ vec a m → vec a n→ vec a (m + n),
where the two explicit arguments and the return type depend on the three implicit arguments
marked with ‘#’. We mostly omit implicit binders, except when needed for clarity, treating all
unbound variables in types as prenex quantified. The type of pairs in F★is represented by a & b; in
contrast, dependent tuple types are written as x:a & b with x bound in b. A dependent pair value is
written (| e, f |) and we use x.1 and x.2 for the first and second dependent projection maps. We elide
universe annotations throughout this paper.
A Dijkstra monad is a monad-like structure M indexed by a separate monad of specifications W,
forming types of the shape M a w, where w : W a. Dijkstra monads in F★have traditionally taken W
to be a weakest-precondition monad [Ahman et al. 2017; Swamy et al. 2016, 2013] in the style of
Dijkstra (hence their name), though this is not a fundamental requirement [Maillard et al. 2019].
Proc. ACM Program. Lang., Vol. 1, No. CONF, Article 1. Publication date: January 2018.
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
1:5
A simple Dijkstra monad for pure computations. Consider the identity monad id t = t, whose
return is _x→ x and whose bind is the (reverse) function application _x f → f x. A simple Dijkstra
monad for id is an indexed monad, pure (t:Type) (w:wp t), where wp t = (t → prop) → prop is the type
of a predicate transformer for id t computations, transforming a postcondition (a predicate on
results, t → prop) to a precondition (a proposition, prop). Notice that wp t is itself a continuation
monad (with answer type prop), with wreturn x = _p→ p x and wbind w1w2= _p → w1(_x → w2x p). As
we will see shortly, the return and bind on pure computations manifest in the indices via wreturn
and wbind. This makes it natural to compute wp specifications for pure computations, and to use
those specifications for Hoare-style program proofs. For example, integer division can be given a
type such as x:int → y:int→ pure int (_p → y ≠0 ∧ p (x / y)), indicating a pure function on integers
x and y, with a precondition that y is non-zero and returning a value x/y.
Computation types: Tot, PURE, and user-defined Dijkstra monads. Recognizing its generality and
correspondence to weakest precondition calculi for Hoare logics, F★baked in notions of Dijkstra
monads. Towards that end, F★contains a notion of a computation type, distinguishing computations
from values in a manner similar (though not identical) to Levy [2004]. In their most general form,
arrow types in F★are written x:t → C where C is a computation type, including Tot t, for total
computations2(corresponding to the identity monad), and a variety of other Dijkstra monads all
with the shape M t wp, where wp is a monad at the level of specifications. Here M ranges over PURE,
the primitive Dijkstra monad for pure computations, as well as other user-defined Dijkstra monads,
defined in terms of PURE. Two core typing rules (shown slightly simplified) internalize the use of
Dijkstra monads, allowing values to be returned in any computation type M (corresponding to a
return in M’s index) and for M-computations to be sequenced (with a bind at the level of indices).
T-Return
Γ ⊢ 𝑣 : Tot 𝑡
Γ ⊢ 𝑣 : 𝑀 𝑡 (𝑀.𝑟𝑒𝑡𝑢𝑟𝑛 𝑣)
T-Bind
Γ ⊢ 𝑒 : 𝑀 𝑎 𝑤1 Γ, 𝑥 : 𝑎 ⊢ 𝑓 : 𝑀 𝑏 (𝑤2 𝑥)Γ ⊢ let 𝑥 = 𝑒 in 𝑓 : 𝑀 𝑏 (𝑀.𝑏𝑖𝑛𝑑 𝑤1 𝑤2)
The shared Dijkstra monad structure makes these typing rules uniform for all computation types.
Layered indexed effects, in a nutshell. The main contribution of this paper is to generalize these
typing rules so that they no longer require all computation types to be Dijkstra monads (§3). Instead,
computation types in F★are now of the form Tot t (as before) or are indexed in arbitrary ways as
in M t 𝑖, where the new rule for T-Bind allows the (variable number of) indices 𝑖 to be composed in
arbitrary, user-controlled ways—all that’s required to introduce a layered indexed effect M t 𝑖 is for
it to have the following:
(1) A representation type, M.repr t 𝑖
(2) A return, with shape a:Type → x:a → 𝑖𝑚𝑝𝑙𝑖𝑐𝑖𝑡𝑠 → M.repr a 𝑡
(3) A bind, with shape a:Type→ b:Type → 𝑖𝑚𝑝𝑙𝑖𝑐𝑖𝑡𝑠 → M.repr a 𝑝 → (x:a → M.repr b 𝑞) → M.repr b 𝑟
The T-Return and T-Bind rules for a given M correspond to applications of the return and bind
combinators, with the 𝑖𝑚𝑝𝑙𝑖𝑐𝑖𝑡𝑠 computed by F★’s existing higher-order unification engine. With no
restrictions on the indexing structure, layered indexed effects can be used with a diversity of indexed
monad-like structures, and programmers benefit from F★’s inference and elaboration algorithm
for their user-defined effects. One can choose to prove basic laws for user-defined structures (and
we often do), but F★neither requires nor depends on any particular laws (§3). Particularly when
augmented with other features, e.g. rules for subsumption and lifting between computations types,
layered indexed effects make programming with a variety of custom indexed effects in F★easy.
2F★also includes a primitive computation type Div t for potentially non-terminating computations, isolated from its logical
core—we discuss this further in §3.2. Additionally, we write x:t → t' as sugar for x:t→ Tot t'.
Proc. ACM Program. Lang., Vol. 1, No. CONF, Article 1. Publication date: January 2018.
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
1:6 Aseem Rastogi, Guido Martínez, Aymeric Fromherz, Tahina Ramananandro, and Nikhil Swamy
Encoding PURE as a layered effect. To turn PURE into a user-defined computation type, all that’s
required is to define the pure type shown earlier, a return and bind for it, and to ask F★to turn it
into an effect.
let pure (a:Type) (w:wp a) : Type = p:(a → prop)→ squash (wp p) → Tot (v:a{p v})
let return (a:Type) (x:a) : pure a (wreturn x) = __ _→ x
let bind (a b:Type) (w1:wp a) (w2:(a → wp b)) (f: pure a w1) (g: (x:a → pure b (w2 x))) : pure b (wbind w1 w2)
= _p _→ let x = f (_ x → w2 x p) () in g x p ()
effect { PURE (a:Type) (w:wp a) = pure with return; bind }
The representation we chose for pure is a form of continuation-passing, where the first argument
p is the postcondition to prove; the second squash (wp p) is a hypothesis for the corresponding
precondition; and the result v:a{p v} is refined to satisfy the postcondition. In bind, we first run f
into a value, by calling it with the (_x → w2x p) postcondition, whose precondition according to
w1 must hold: it is exactly wbind w1w2p. Then, we simply call g with it at postcondition p. The unit
values take the place of the squashed proofs of preconditions, discharged by the SMT solver.
In essence, using F★’s existing support for dependent types and refinement types, we can give
a semantic model for pure, while the effect definition turns pure into a computation type PURE so
that client programs are agnostic to the choice of representation (several other representations for
pure are also possible) and can simply work with respect to the abstraction provided.
Beyond the bare minimum of providing a return and bind, effect definitions can be augmented
with rules for subsumption and for composing conditional computations. For instance, subcomp
below allowsweakening a specification (by strengthening its precondition pointwise)—the definition
of subcomp is its own soundness proof. Meanwhile, cond shows that case analyzing a boolean b
and running either f or g produces a pure computation whose wp-index reflects the case analysis
too—F★checks that if b then f else g can be given the type claimed by cond _ _ _ f g b, ensuring its
soundness.
let subcomp (a:Type) (w1 w2 : wp a) (u : squash (∀ p. w2 p =⇒ w1 p)) (f : pure a w1) : pure a w2 = f
let cond (a : Type) (w1 w2 : wp a) (f : pure a w1) (g : pure a w2) (b : bool) : Type =
pure a (_ p → (b =⇒ w1 p) ∧ ((¬b) =⇒ w2 p))
effect { PURE (a:Type) (w:wp a) = pure with return; bind; subcomp; cond }
Without these two additional combinators, F★would do no better than forcing the branches of a
case analysis to match precisely (i.e. via provable equality), and there would be no way to, e.g.,
branch into PURE computations with non-equal indices.
With our layered indexed effect in place, we can use primitive syntax for computations in PURE
and leave it to F★’s inference to compute WPs. As is common in F
★, we also define an alias Pure
so we can write pre- and postconditions instead of WPs when needed, decorating them with
the tags requires and ensures to help with readability. Below is a small example of a partial map
function over a list: the function f being mapped has an argument-dependent precondition, which
is propagated into the precondition of map via a quantification over the elements of the list.
effect Pure (a:Type) (pre:prop) (post:a → prop) = PURE a (_ p → pre ∧ (∀ x. post x =⇒ p x))
let rec map #a #b #pre (f : (x:a → Pure b (requires (pre x)) (ensures (_ _ →⊤)))) (l : list a)
: Pure (list b) (requires (∀ x. x ∈ l =⇒ pre x)) (ensures (_ _ →⊤))= match l with | []→ [] | x::xs→ f x :: map f xs
Of course, Dijkstra monads may seem overkill for specifying pure computations—one could also
have written the type for map with refinement types alone. However, all other F★effects are built on
top of PURE, and making it a derived notion paves the way to removing it from the core, simplifying
Proc. ACM Program. Lang., Vol. 1, No. CONF, Article 1. Publication date: January 2018.
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
1:7
F★’s foundations. Further, being liberated from Dijkstra monads alone, we can develop other effect
typing disciplines, as we see will see next.
2.2 Graded Monads and Dijkstra Monads for Algebraic Effects and HandlersAlgebraic effects and handlers are a framework for modeling effects in an extensible, composable,
re-interpretable manner, with strong semantic foundations [Plotkin and Power 2003] and several
new languages and libraries emerging to support them [Bauer and Pretnar 2015; Brady 2013; Leijen
2017; Lindley et al. 2017; Plotkin and Pretnar 2009]. In this section, we show that algebraic effects
and handlers can be encoded generically using dependent types in F★, and exposed to programmers
as a layered indexed effect supporting programming in an abstract, high-level style. Further, we
show that operation labels can be conveniently tracked as an index, much like in existing effect
systems for algebraic effects. Also, we reconcile them with WPs for the particular case of state,
proving that functional specifications can be strengthened from the intensional information of its
operations, employing a unique combination of graded and Dijkstra monads.
Roadmap. Our starting point is a canonical free monad representation tree0 a of computa-
tions with generic actions producing a-typed results. Next, we refine tree0 to a graded monad
tree a (l:list op), where l over-approximates the actions in the tree. We then define dependently
typed handlers, that allow handling actions in a computation in a custom manner. Packaging tree
as an indexed effect Alg a l allows programming with effects and handlers in a direct syntax, with
types recording the effectful actions that are yet to be handled; e.g. an Alg a [] is a pure computa-
tion that can be reduced to a result. Further, to verify programs, we give a more refined variant
AlgWP a l wp introducing an additional WP index, which describes the behavior of the computation
with respect to a standard, state-passing implementation. We prove the WP index to be sound, and
provide an operation to strengthen the WP of write-free programs, as well as provide some simple
examples. Being agnostic to the choice of indexing, layered indexed effects supports equally well
effectful programming with graded monads and program verification with Dijkstra monads—both
on top of the same algebraic effects framework.
A free monad over generic actions. We begin by defining a type for the operations and giving their
input and output types.3We include stateful operations (Read and Write) and exceptions (Raise), but
other operations can be easily added.4Then, we define the type for computations that sequentially
compose them according to their arity, i.e. the free monad over them.
type op = | Read | Write | Raise
let op_inp (o : op) : Type = match o with | Read→ unit | Write → state | Raise→ exn
let op_out (o : op) : Type = match o with | Read→ state | Write → unit | Raise→ empty
type tree0 (a : Type) =
| Return : a → tree0 a
| Op : op:op→ i:(op_inp op)→ k:(op_out op → tree0 a) → tree0 a
Refining the free monad with a set of labels. The tree0 type contains all possible combinations of
the operations. To be more precise, we will limit the operations that may appear in a computation.
We say a computation “abides” by an operation set when all of its operations are also in the set.
Our representation type tree, indexed by a set of operations, is defined simply as the trees that
abide by their annotated set. We use a list for the index, but only interpret it via membership,
3An alternative is to index op with input and output types. This is essentially equivalent, but notationally heavier.
4Our implementation in the supplementary material contains an additional uninterpreted Other : int → op, and never
relies on knowing the full set of operations.
Proc. ACM Program. Lang., Vol. 1, No. CONF, Article 1. Publication date: January 2018.
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
1:8 Aseem Rastogi, Guido Martínez, Aymeric Fromherz, Tahina Ramananandro, and Nikhil Swamy
so order and multiplicity are irrelevant. This makes tree a graded monad, where the monoid in
question takes the union of sets of operations.
type ops = list op
let rec abides #a (labs:ops) (c : tree0 a) : prop = match c with
| Return _→⊤| Op a i k→ a ∈ labs ∧ (∀ o. abides labs (k o))
type tree (a:Type) (labs:ops) = c:(tree0 a){abides labs c}
Packaging the refined free monad as an effect. Elevating tree to an indexed effect is straightforward,only requiring to define the basic combinators. With most of the heavy lifting done by F
★’s SMT
backend, we do not need to explicitly prove the abides refinements, provided some theorems about
list membership are in scope. We also provide a subsumption rule to grow the labels where needed,
and also reorder and deduplicate them, since they are essentially sets. The branching combinator
joins the labels of each branch. It can be easily verified that these last two satisfy the expected
relation (and F★proves that automatically as well).
let return a (x:a) : tree a [] = Return x
let bind a b labs1 labs2 (c : tree a labs1) (f : a → tree b labs2) : tree b (labs1 ∪ labs2) = (∗ elided ∗)
let subcomp a labs1 labs2 (f : tree a labs1) : Pure (tree a labs2) (requires (labs1 ⊆ labs2)) = f
let cond (a:Type) (labs1 labs2 : ops) (f:tree a labs1) (g:tree a labs2) (p: bool) = tree a (labs1 ∪ labs2)
effect { Alg (a:Type) (labs:ops) = tree with return; bind; subcomp; cond }
Reflecting operations. An effect like Alg is an abstract alias for its underlying representation type
tree. Based on an existing feature of F★, itself inspired by Filinski [1999], we can reify Alg to its
representation and conversely reflect tree as an Alg, revealing the abstraction only when necessary.
Using reflection, algebraic operations can be uniformly turned into computations via geneff below.
For raise, we take an extra step of matching on its (empty) result to make it polymorphic. Simple
programs can already be written in direct style, with automatic inference of the labels—as a matter
of best practice, F★requires documenting the type of top-level definitions, otherwise even those
could be left out and inferred.
let geneff (o : op) (i : op_inp o) : Alg (op_out o) [o] = Alg?.reflect (Op o i Return)
let get () : Alg state [Read] = geneff Read ()
let put (s:state) : Alg unit [Write] = geneff Write s
let raise (e:exn) : Alg 𝛼 [Raise] = match geneff Raise e with (∗ empty match ∗)
exception Failure of string
let add_st (x y : int) : Alg int [Read; Raise] =
let s = get () in (∗ assuming integer state ∗)if s < 0 then raise (Failure "error") else x + y + s
Implementing handlers generically. Programming dependently typed generic handlers for treesis relatively smooth. The type handler_tree_op o b labs below represents a handler for the single
operation o into a computation of type b and operations in labs. The handler will be called with
the parameter to the operation and its continuation that has already been handled recursively.
A handler_tree labs0 b labs1 is a dependent product of operation handlers over domain labs0;
in other words, one handler for each operation in labs0, all of them coinciding in the type and
operations they handle to. The handle_tree function implements generic handling of a computation
tree c given a value case v and a generic handler h for every operation in c.
Proc. ACM Program. Lang., Vol. 1, No. CONF, Article 1. Publication date: January 2018.
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
1:9
type handler_tree_op (o:op) (b:Type) (labs:ops) = op_inp o→ (op_out o → tree b labs) → tree b labs
type handler_tree (labs0:ops) (b:Type) (labs1:ops) = o:op{o ∈ labs0}→ handler_tree_op o b labs1
let rec handle_tree (c : tree a labs0) (v : a → tree b labs1) (h : handler_tree labs0 b labs1)
: tree b labs1 = match c with
| Return x → v x
| Op act i k → h act i (_ o → handle_tree (k o) v h)
While general, this handling construct over trees is uncomfortable as one needs to manually bind
and return computations, which may become unwieldy as trees nest, e.g., the usual handler for
Read/Write into the state monad involves a (tree (state → tree (a & state) labs) labs). It is much
more convenient to define an effectful handling construct directly usable in Alg. To do so, we can
simply lift handle_tree into Alg via reflection (and reification in negative positions). Handling
via handle_with can be done in direct style, with full support for label inference and automatic
elaboration into monadic form.
type handler (labs0 : ops) (b:Type) (labs1 : ops) =
o:op{o ∈ labs0}→ op_inp o → (op_out o → Alg b labs1)→ Alg b labs1
let handle_with #a #b #labs0 #labs1
(f : unit→ Alg a labs0) (v : a → Alg b labs1) (h : handler labs0 b labs1) : Alg b labs1
= (∗ elided, essentially a wrapper of handle_tree, translating h into a handler_tree, etc. ∗)
We show below the handler of Raise into the option monad, and that of Read/Write into the state
monad, both of which are written in direct style and leave other operations unhandled. To only
handle part of the operations within a computation, we add a default case to h where needed. This
can be done generically via defh below: it defines a handler that leaves every operation unhandled
by executing it and calling the continuation. Accordingly, its type states that it does not modify the
label set. Thanks to unification, all of its arguments are usually inferred, even the operation; in our
implementation we make also o implicit and elide the underscore argument. However, catchST does
requires a small annotation: F★needs the information about the nesting of effectful computations,
and about the set of internal labels. Barring that, everything else (lifting, binding, weakening, etc)
is handled seamlessly by the effect system. The applicative syntax in the handler is similar to that
of other languages specifically designed for algebraic effects [Bauer and Pretnar 2015; Leijen 2014].
let defh #b #labs : handler labs b labs = _o i k → k (geneff o i) (∗ essentially rebuilding an Op node ∗)
let catchE #a #labs (f : unit → Alg a (Raise::labs)) : Alg (option a) labs =
handle_with f Some (function Raise → _i k→ None
| _ → defh _)
let catchST #a #labs (f: unit→ Alg a (Read::Write::labs)) (s0:state) : Alg (a & state) labs =
handle_with #_ #(state→ Alg _ labs) #_ #labs
f (_ x s→ (x, s)) (function Read → __ k s → k s s
| Write → _s k _→ k () s
| _ → defh _) s0
Running programs. To run computations, we can handle all their operations and extract the final
result. As usual, the choice of how to interpret operations is given by the handlers. In the example
below we show how programs combining state and exceptions can be interpreted into the two
most common monadic layouts, according to the stacking of the relevant monad transformers. The
order of the labels of f is not important at all, even for inference: in both examples unification will
work inwards from the fact that run takes a label-free computation and directly infer a list of labels
for f. It will then use subcomp to check that it abides by them, allowing re-orderings.
Proc. ACM Program. Lang., Vol. 1, No. CONF, Article 1. Publication date: January 2018.
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
1:10 Aseem Rastogi, Guido Martínez, Aymeric Fromherz, Tahina Ramananandro, and Nikhil Swamy
let run #a (f : unit→ Alg a []) : Tot a = match reify (f ()) with | Return x→ x
let run_STExn #a (f : unit→ Alg a [Write; Raise; Read]) (s0:state) : option (a & state) =
run (_ () → catchE (_ ()→ catchST f s0))
let run_ExnST #a (f : unit→ Alg a [Write; Raise; Read]) (s0:state) : (option a) & state =
run (_ () → catchST (_ ()→ catchE f) s0)
AlgWP: A graded Dijkstra monad for stateful Alg programs. While Alg above bounds the operations
each computation may invoke, there is no way to specify their order, or any property about the
final value and state. To do so, we can bring back the idea of using a WP calculus to the algebraic
setting by adding a new index to the effect. We limit ourselves to stateful operations and use WPs
to specify the behavior according to the state monad: without fixing an interpretation, there is not
much to be said in terms of verification unless equations are added. That is the topic of future work
in the area, but we believe the effectful representation would not be heavily affected by it.
Our new effect will refine tree even further. The main idea is to begin with a function that takes
a computation tree and computes a “default” stateful WP from it. Then, we take the representation
treewp a l wp to be the computation trees with operations in l whose default WP is pointwise
weaker than its annotated WP (allowing underspecification). This construction is due to Maillard
et al. [2019] but our setting is more general due to the additional grading, not supported in Dijkstra
monads. We begin by defining a predicate transformer monad for stateful programs.
type post_t (a:Type) = a → state→ prop type st_wp (a:Type) = state → post_t a → prop
let read_wp : st_wp state = _s0 p → p s0 s0
let write_wp : state → st_wp unit = _s _ p → p () s
let return_wp (#a : Type) (x:a) : st_wp a = _s0 p → p x s0
let bind_wp (#a #b : Type) (w1 : st_wp a) (w2 : a → st_wp b) : st_wp b = _s0 p → w1 s0 (_ y s1 → w2 y s1 p)
The st_wp a type captures functions transforming postconditions on results and states to precondi-
tions on states. The return_wp, bind_wp, read_wp and write_wp combinators can be understood as
the CPS transform of the corresponding computational constructs [Ahman et al. 2017]. Next, we
interpret Read-Write trees into st_wp via interp_as_wp. We refine the tree type by both limiting its
operations and adding a refinement comparing its WP via (≼), the strengthening relation on statefulWPs. We also prove the WP calculus sound by interpreting AlgWP programs into state-passing pure
programs, the correctness of which we argued for in §2.1. A soundness theorem for Read only
programs can also be easily provided.
let rec interp_as_wp #a (t : tree a [Read; Write]) : st_wp a = match t with
| Return x→ return_wp x
| Op Read _ k→ bind_wp read_wp (_ s → interp_as_wp (k s))
| Op Write s k→ bind_wp (write_wp s) (_ ()→ interp_as_wp (k ()))
type rwops = l:ops{l ⊆ [Read; Write]}
let treewp (a : Type) (l:rwops) (w: st_wp a) = t:(tree a l){ w ≼ interp_as_wp t }
effect { AlgWP (a:Type) (l:rwops) (w:st_wp a) = treewp with ...}
let get () : AlgWP state [Read] read_wp = AlgWP?.reflect (Op Read () Return)
let put (s:state) : AlgWP unit [Write] (write_wp s) = AlgWP?.reflect (Op Write s Return)
let soundness #a #l #wp (t : unit→ AlgWP a l wp) : s0:state→ PURE (a & state) (wp s0) = ...
Using this graded Dijkstra monad, we can verify functional correctness properties, which a
graded monad alone cannot capture. For instance, when the state is instantiated to a heap (mapping
locations to values), we can prove that the program below correctly swaps two references in the
heap—AlgPP is simply a pre-/postcondition alias to AlgWP, in the style of Pure in §2.1.
Proc. ACM Program. Lang., Vol. 1, No. CONF, Article 1. Publication date: January 2018.
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
1:11
let swap (l1 l2 : loc) : AlgPP unit [Write; Read] (requires __ → l1 ≠l2)
(ensures _h0 _ h1 → h0\{l1;l2} == h1\{l1;l2} ∧ h1.[l1] == h0.[l2] ∧ h1.[l2] == h0.[l1])
= let r = !l2 in l2 := !l1; l1 := r
Of course, this is a simple example, and verification of stateful programs is a discipline in itself, but
it illustrates how different layers and styles of verification can interact within a single language.
More interestingly, the static information in the label index can be exploited by the WP. The
quotient function below strengthens the postcondition of a write-free AlgWP program into addi-
tionally ensuring that the state does not change. Operationally, quotient just runs f (), so it can be
seen as a proof that f does not change the state. On the other hand, ignore_writes below handlesall Write operations in f by ignoring them. In this case the postcondition becomes possibly weaker:
it says nothing about the result, but still ensures that the state does not change.
val quotient #a #pre #post (f : unit → AlgPP a [Read] pre post)
: AlgPP a [Read] pre (_ h0 x h1 → post h0 x h1 ∧ h0 = h1)
val ignore_writes #a (#labs:rwops{Write ∉ labs)}) #wp (f : unit→ AlgWP a (Write::labs) wp)
: AlgPP a labs pre (_ h0 _ h1 → h0 = h1)
We have barely scratched the surface of algebraic effects and handlers in F★, but we see it as a
very promising area for further work. That these disciplines can be prototyped so straightforwardly
is a testament to the usability of layered indexed effects.
2.3 A Graded Monad for Stateful Information Flow ControlIn this section, we present an indexed effect to track information flows [Denning 1976] in higher-
order, stateful programs. Our general approach is to adopt the techniques of monadic information
flow control (IFC), in the tradition of Abadi et al.’s (1999) dependency core calculus and others [Crary
et al. 2005; Devriese and Piessens 2011; Hicks et al. 2014; Russo et al. 2008], though expressed as a
layered indexed effect based on a graded monad. We enhance prior approaches in three notable
ways. First, rather than tracking flows among a given lattice of information flow control labels, our
approach tracks flows between arbitrarily fine-grained sets of memory references—working in a
dependently typed setting, dynamic IFC labels [Zheng and Myers 2007] are very natural. Second, we
develop a novel graded monad in which to separately track read and write effects as well as a flowsrelation, which records dependences among the reads and writes. Finally, unlike prior approaches
which have relied on a separate metatheory for security, we prove a noninterference theorem
entirely within the system. Yet, the semantic model does not pollute client reasoning: by packaging
our graded monad as an effect layer, client F★programs can be verified for noninterference by
analysis of the flows relation, as one might expect in a custom information-flow analysis, hiding all
details of the underlying semantic model.
A two-step plan. We start by presenting a relational model of read and write effects, in the
spirit of Benton et al. [2006]. We develop a graded monad rwst a r w, describing the type of an
a-returning computation that may read the memory locations r and write the locations w. This
model is already sufficient for many purposes (e.g., to justify the soundness of various program
optimizations), as well as to track information flow. Indeed, one can prove the information flows
in an rwst a r w program are contained in the flows between the read set r and the write set w.
However, as we will see, this model is also overly conservative. Addressing this imprecision, we
present the main construction of this section, an indexed effect IFC a r w flows, a refinement of rwst
which additionally tracks a subset of the reads that can influence a subset of the writes.
Tracking read and write effects. We consider computations in a state monad st a = store →a & store where store = Map.t loc int, maps abstract memory locations loc to integers—we do not
Proc. ACM Program. Lang., Vol. 1, No. CONF, Article 1. Publication date: January 2018.
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
1:12 Aseem Rastogi, Guido Martínez, Aymeric Fromherz, Tahina Ramananandro, and Nikhil Swamy
model dynamic allocation nor typed references, as they are orthogonal. Our first graded monad,
rwst is the refinement of st shown below.
let rwst (a : Type) (r : label) (w : label) = f:(st a){ reads_bounded f r ∧ writes_bounded f w }
𝑤ℎ𝑒𝑟𝑒
let label = set loc
let writes_bounded f w = ∀l. l ∉ w =⇒ (∀ s. sel (snd (f s)) l == sel s l) (∗ locs not in w do not change ∗)let reads_bounded f r = ∀l. l ∉ r =⇒ (∀ s0 i.
let s0' = upd s0 l i in (∗ havoc s at l ∗)let (x, s1), (x', s1') = f s0, f s0' in (∗ run the computation twice ∗)x == x' ∧ (∗ results are the same ∗)(∀ l'. l' ≠l =⇒ sel s1 l' == sel s1' l') ∧ (∗ every location not equal to l is the same ∗)(sel s1 l == sel s1' l ∨ (∗ either l is written and it is the same in both runs ∗)(sel s1 l == sel s0 l ∧ sel s1' l == sel s0' l))) (∗ or its value in unchanged in both runs ∗)
Constraining the write effects in rwst is relatively straightforward: writes_bounded f w simply
states that the contents of all locations l not in the write set w are unchanged when running f
in any state. Specifying read effects is bit more subtle—reads_bounded f r captures the essence of
noninterference. It states, roughly, that for any location l not in the read set r, the result and final
state of the computation do not depend on the contents of l. That is, when running f in any pair
of states s0 and s0' that differ only on the contents of l, the final results x and x' are identical; the
final states s1 and s1' are identical on all locations different from l; and l itself is either written (in
which case its contents are the same in both final states), or it is unchanged.
It is relatively straightforward to define the following monadic combinators and actions for rwst,
and to package it as a layered effect.
let return #a (x : a) : rwst a {} {} = _s → x,s
let bind #a #b (f : rwst a r0 w0) (g : a → rwst b r1 w1) : rwst b (r0 ∪ r1) (w0 ∪ w1) = . . .
effect { RWST (a : Type) (r : label) (w : label) = rwst with return; bind }
let read (l:loc) : RWST int {l} {} = RWST?.reflect (_ s → sel s l, s)
let write (l:loc) (x:int) : RWST unit {} {l} = RWST?.reflect (_ s → (), upd s l x)
One can now write effectful programs as shown below, and have F★compute read and write sets
at a fine granularity—the abstraction imposed by the effect layer ensures that one has to reason
only about labels rather than about the extensional behavior of the underlying computations.
let read_write (l0 l1 : loc) : RWST unit {l0} {l1} = write l1 (read l0)
However, the abstraction of rwst is fairly imprecise inasmuch as the indexing structure cannot
distinguish read_write above from write_read below, even though the latter clearly does not leak
the contents of l0 to l1.
let write_read (l0 l1 : loc) : RWST int {l0} {l1} = write l1 0; read l0
Refining reads and writes with a flows relation. Our second step involves refining rwst to the ifc
graded monad shown below, which includes an additional index fs:flows describing the information
flows in a computation as a list of edges, where (from, to) ∈ fs indicates a potential information flow
from the set of memory locations in the “source label” from to the set of locations in the “sink label”
to. The respects f fs refinement states that if (from, to) is not in the flows relation, then running f
twice in states that differ only on the contents of from produces states that agree on the contents of
to.
let ifc a (r:label) (w:label) (fs:flows) = f:(rwst a r w){ f `respects` fs }
𝑤ℎ𝑒𝑟𝑒
Proc. ACM Program. Lang., Vol. 1, No. CONF, Article 1. Publication date: January 2018.
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
1:13
let flow = label & label let flows = list flow
let has_flow1 from to f = from ∈ fst f ∧ to ∈ snd f
let has_flow from to fs = ∃f. f ∈ fs ∧ has_flow1 from to f
let no_leak (f:st a) (from to:loc) = ∀s0 k. sel (snd (f s0)) to == sel (snd (f (upd s0 from k))) to
let respects (f:st a) (fs:flows) = ∀from to. ¬(has_flow from to fs) ∧ from≠to =⇒ no_leak f from to
With this refinement, we can define a more precise graded monad structure, as shown below.
let return (x:a) : ifc a {} {} [] = _s → x, s
let bind (f:ifc a r0 w0 fs0) (g:a → ifc b r1 w1 fs1)
: ifc b (r0 ∪ r1) (w0 ∪ w1) (fs0 @ add_src r0 (({}, w1)::fs1)) = . . .
𝑤ℎ𝑒𝑟𝑒 add_src r fs = List.map (_ (src, sink)→ (src ∪ r, sink)) fs
The proof of the correctness of bind is non-trivial and requires about 100 lines of auxiliary
lemmas. However, once sealed into the effect layer IFC (as shown below), we can write and verify
source programs reasoning only about labels and flow relations—the proof relating labels and flow
relations to the underlying semantic model is done once and for all.
Subsumption. Further, we can enhance our model with a subsumption relation based on a partial
order on labels and flow relations.
let included_in (fs0 fs1 : flows) =
∀f0. f0 ∈ fs0 =⇒ (∀ from to. has_flow1 from to f0 =⇒ (∃ f1. f1 ∈ fs1 ∧ has_flow1 from to f1))
let ifc_sub (f : ifc a r0 w0 fs0{ r0 ⊆ r1 ∧ w0 ⊆ w1 ∧ fs0 `included_in` fs1 }) : ifc a r1 w1 fs1 = f
effect { IFC (a:Type) (r:label) (w:label) (fs:flows) = ifc with return; bind; ifc_sub }
With these structures in place, our sample programs which could not be distinguished by RWST,
are correctly distinguished by the flows relation of IFC, with proofs entirely automated by reasoning
using the abstraction of labels and the flows relation only.
let read_write (l0 l1:loc) : IFC unit {l0} {l1} [{l0}, {l1}] = write l1 (read l0)
let write_read (l0 l1:loc) : IFC int {l0} {l1} [] = write l1 0; read l0
Of course, label-based IFC remains inherently imprecise. For example, the following typing
suggests a flow from l0 to l1, even though the result of read l0 is discarded.
let noop (l0 l1:loc) : IFC unit {l0,l1} {l1} [{l0,l1}, {l1}] = let _ = read l0 in write l1 (read l1)
In principle, one could rely on a semantic proof in F★of noop to show that it does not actually leak
any information and refine its type—we leave mingling label-based noninterference with semantic
proofs to future work.
Having computed the flows in a computation, one can simply check whether the flows exhibited
are permissible or not. For example, one could build a traditional lattice of security labels as sets
classifying memory locations and check the following:
val high : label let low = all\high let lref = l:loc{l ∈ low} let href = l:loc{l ∈ high}
let read_write (l:lref) (h:href) : IFC unit low high [low, high] = write h (read l)
Is ifc really a graded monad? Some of the prevailing wisdom is that static enforcement of stateful
information-flow control does not fit into the framework of graded monads. Indeed, both Tate
[2013] and Hicks et al. [2014] propose more unusual indexing structures based on effectors or
polymonads for IFC. However, our ifc construction shows that it can indeed be done as a graded
monad, with a level of precision greater than Hicks et al.’s polymonadic information flow control.
To see that ifc is a graded monad, we need to prove that its indexing structure is a monoid with
return indexed by the unit of the monoid and bind’s corresponding to composition of indices in the
Proc. ACM Program. Lang., Vol. 1, No. CONF, Article 1. Publication date: January 2018.
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
1:14 Aseem Rastogi, Guido Martínez, Aymeric Fromherz, Tahina Ramananandro, and Nikhil Swamy
monoid. That rwst is a graded monad should be obvious, since its indices are merely sets composed
with ∪. What remains is a monoid structure for the flows relation, but composing the flows indices
depends on the read and write sets. As such, one can combine the three indices of ifc into a single
ix structure, and define a candidate monoid with unit 𝜖 and composition ⊕.
let ix = label & label & flows
let (𝜖) = {}, {}, []
let (⊕) (r0, w0, f0) (r1, w1, f1) = r0 ∪ r1, w0 ∪ w1, f0 @ add_src r0 (({}, w1)::f1)
This structure forms a monoid under a suitable equivalence relation for ix, which corresponds
to equivalence on the label sets and equivalence of flows, as shown below. Pleasantly, flows_equiv
also satisfies an intuitive algebraic structure that distributes with ∪ on labels.
let flows_equiv : erel flows = _fs0 fs1 → fs0 `included_in` fs1 ∧ fs1 `included_in` fs0
let included_in_union_distr (a b c : label)
: Lemma (flows_equiv [(a, b ∪ c)] [(a, b); (a, c)] ∧ flows_equiv [(a ∪ b, c)] [(a, c); (b, c)]) = ()
3 FORMALIZATION AND IMPLEMENTATION OF LAYERED INDEXED EFFECTSAhman et al. [2017] present EMF
★, a pure type system extended with refinement types and Dijkstra
monads, where effectful terms are represented in explicitly monadic form. To model layered indexed
effects, we have no need for EMF★’s effectful features nor its support for Dijkstra monads, as it can
be encoded. We refer to the effectless fragment of EMF★, including total functions and refinement
types, as TotalF★and use it as the target language for elaborating LIF
★, a language with layered
indexed effects intended as a model of F★’s surface language. The main result of this section is that
elaboration is type-preserving, implying that typing derivations in LIF★can be soundly interpreted
in TotalF★. We also discuss several implementation aspects of layered indexed effects in the F
★
typechecker.
3.1 LIF*: Elaborating User-defined EffectsLIF
★models the surface language of F
★with support for user-defined layered indexed effects.
Figure 1 shows the syntax and selected typing rules.
LIF★adds effectful constructs to TotalF
★. These include the monadic computation types 𝐹 𝑡 𝑒
(where 𝐹 is the effect label, 𝑡 is the return type, and 𝑒 are the effect indices) in 𝐶 , monadic let-bindings, and explicit reify and reflect coercions to go back-and-forth between computation types
and their underlying representations. An effect definition 𝐷 defines a layered indexed effect 𝐹
with indices types 𝑥 : 𝑡 and combinators 𝑒𝑟𝑒𝑝𝑟 , 𝑒𝑟𝑒𝑡𝑢𝑟𝑛, 𝑒𝑏𝑖𝑛𝑑 , and 𝑒𝑠𝑢𝑏𝑐𝑜𝑚𝑝 , while lift𝑀2
𝑀1
defines a
combinator to lift𝑀1 computations to𝑀2 (𝑀 ranges over Tot and 𝐹 ). LIF★ inherits from TotalF★
proof-irrelevant refinement types, dependent functions, standard dependent pattern matching
with case (with explicitly annotated return computation type 𝐶), and a non-cumulative predicative
universe hierarchy (Type𝑢 ). We elide universe annotations in the rest of the section. The ascription
form 𝑒:𝐶 ascribes 𝑒 with computation type 𝐶 . The implicit monadic structure in the terms is
elaborated into explicit binds and lifts by the typing judgment.
Type-directed elaboration. The main typechecking judgment in LIF★has the form Δ ⊢ 𝑒 : 𝐶 ⇝ 𝑒 ′
stating that under a typing context Δ, expression 𝑒 has computation type 𝐶 and elaborates to
expression 𝑒 ′ in TotalF★. Before we explain the selected typing rules shown in Figure 1, we discuss
typechecking the effect definitions and lifts.
While LIF★does not impose any constraints on the layering or indexing structure, the types of
the combinators in an effect definition 𝐷 are constrained to have specific shapes. For example, 𝑒𝑏𝑖𝑛𝑑
Proc. ACM Program. Lang., Vol. 1, No. CONF, Article 1. Publication date: January 2018.
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
1:15
constant 𝑇 ::= unit | Type𝑢 | sum | inl | inrterm 𝑒, 𝑡 ::= 𝑥 | 𝑇 | 𝑥 :𝑡1{𝑡2} | 𝑥 :𝑡 → 𝐶 | _𝑥 :𝑡 . 𝑒 | 𝑒1 𝑒2 | let 𝑥 = 𝑒1 in 𝑒2
| case𝐶 (𝑒 as 𝑦) 𝑥 .𝑒1 𝑥 .𝑒2 | reify 𝑒 | 𝐹 .reflect 𝑒 | 𝑒:𝐶
computation type 𝐶 ::= Tot 𝑡 | 𝐹 𝑡 𝑒 effect label 𝑀 ::= Tot | 𝐹effect definition 𝐷 ::= 𝐹 (𝑎 : Type) (𝑥 : 𝑡) {𝑒𝑟𝑒𝑝𝑟 ; 𝑒𝑟𝑒𝑡𝑢𝑟𝑛 ; 𝑒𝑏𝑖𝑛𝑑 ; 𝑒𝑠𝑢𝑏𝑐𝑜𝑚𝑝 }lift definition 𝐿 ::= lift𝑀2
𝑀1
= 𝑒 signature 𝑆 ::= · | 𝐷, 𝑆 | 𝐿, 𝑆type environment Γ ::= ·|𝑥 : 𝑡, Γ typing context Δ ::= 𝑆 ; Γ
T-Var
𝑥 : 𝑡 ∈ Γ
Δ ⊢ 𝑥 : 𝑡 ⇝ 𝑥
T-Let
Δ ⊢ 𝑒1 : 𝐶1 ⇝ 𝑒 ′1
Δ ⊢ _𝑥 . 𝑒2 : 𝑥 :𝐶𝑡1→ 𝐶2 ⇝ 𝑒 ′
2𝑥 ∉ 𝐹𝑉 (𝐶𝑡
2)
Δ, 𝑓 : 𝐶1, 𝑔 : 𝑥 :𝐶𝑡1→ 𝐶2 ⊢ 𝑀.bind (lift𝐶
𝐶1
𝑓 ) (_𝑥 . lift𝐶𝐶2
(𝑔 𝑥)) : 𝐶 ⇝ 𝑒 ′
Δ ⊢ let 𝑥 = 𝑒1 in 𝑒2 : 𝐶 ⇝ 𝑒 ′[𝑒 ′1/𝑓 ] [𝑒 ′
2/𝑔]
T-Reify
Δ ⊢ 𝑒 : 𝐶 ⇝ 𝑒 ′
Δ ⊢ reify 𝑒 : 𝐶 ⇝ 𝑒 ′
T-Reflect
Δ ⊢ 𝑒 : 𝐶 ⇝ 𝑒 ′
Δ ⊢ 𝐹 .reflect 𝑒 : 𝐶 ⇝ 𝑒 ′
T-As
Δ ⊢ 𝑒 : 𝐶 ′⇝ 𝑒 ′ Δ ⊢ 𝐶 ′ <: 𝐶
Δ ⊢ 𝑒:𝐶 : 𝐶 ⇝ 𝑒 ′
T-Case
Δ ⊢ 𝑒 : sum 𝑡1 𝑡2 ⇝ 𝑒 ′ Δ, 𝑦 : sum 𝑡1 𝑡2 ⊢ 𝐶 : Type⇝ 𝑡 ′ 𝑖 ∈ 1, 2 𝑇 = inl if 𝑖 = 1, inr o/wΔ, 𝑥 : 𝑡𝑖 ⊢ 𝑒𝑖 : 𝐶𝑖 [𝑇 𝑥/𝑦] ⇝ 𝑒 ′𝑖 Δ, 𝑥 : 𝑡𝑖 , 𝑓𝑖 : 𝐶𝑖 [𝑇 𝑥/𝑦] ⊢ lift𝐶
𝐶𝑖𝑓𝑖 : 𝐶 [𝑇 𝑥/𝑦] ⇝ 𝑒 ′′𝑖
Δ ⊢ case𝐶 (𝑒 as 𝑦) 𝑥 .𝑒1 𝑥 .𝑒2 : 𝐶 [𝑒/𝑦] ⇝ case𝑡 ′ (𝑒 ′ as 𝑦) 𝑥 .𝑒 ′′1[𝑒 ′
1/𝑓1] 𝑥 .𝑒 ′′2
[𝑒 ′2/𝑓2]
C-M
Δ ⊢ 𝐶 : Type⇝ Tot 𝑡 ′
Δ ⊢ 𝐶 : Type⇝ Tot 𝑡 ′
SC-M
Δ ⊢ 𝐶1 : Type⇝ _
Δ ⊢ 𝐶 <: 𝐶1
Δ ⊢ 𝐶 <: 𝐶1
S-Conv
Δ ⊢ 𝑡 : Type⇝ 𝑡 ′ Δ ⊢ 𝑡1 : Type⇝ 𝑡 ′1
𝑡 ′ →∗ 𝑡 ′1∨ 𝑡 ′
1→∗ 𝑡 ′
Δ ⊢ 𝑡 <: 𝑡1
Fig. 1. Syntax of LIF★ and selected typing judgments (primed symbols are TotalF★ syntax)
for an effect 𝐹 is typechecked to have a monadic bind shape as (abbreviating 𝑡1 : Type → 𝑡2 : Typeas 𝑡1 𝑡2 : Type):𝑆 ; · ⊢ 𝑒𝑏𝑖𝑛𝑑 : 𝑡1 𝑡2 : Type → 𝑥 : 𝑡 → 𝑒𝑟𝑒𝑝𝑟 𝑡1 𝑒𝑓 → (𝑥 :𝑡1 → 𝑒𝑟𝑒𝑝𝑟 𝑡2 𝑒𝑔) → 𝑒𝑟𝑒𝑝𝑟 𝑡2 𝑒 ⇝ _
Here 𝑥 : 𝑡 are arbitrary binders that may appear in the indices 𝑒𝑓 , 𝑒𝑔, and 𝑒 . Similarly, a lift𝐹 ′
𝐹 .𝑒 is
typechecked as a coercion that takes 𝐹 computations to 𝐹 ′:
𝑆 ; · ⊢ lift𝐹 ′
𝐹 .𝑒 : 𝑡 : Type → 𝑥 : 𝑡 → 𝐹 .𝑒𝑟𝑒𝑝𝑟 𝑡 𝑒𝑓 → 𝐹 ′.𝑒𝑟𝑒𝑝𝑟 𝑡 𝑒 ⇝ _
Every user-defined effect in LIF★gets an automatic lift from Tot: lift𝐹
Tot = 𝐹 .𝑒𝑟𝑒𝑡𝑢𝑟𝑛 .
We now explain the typing rules from Figure 1. We adopt some notational conventions for brevity.
We use 𝐶𝑡to project the return type component from a 𝐶 . 𝐶 is the underlying representation of 𝐶
defined as 𝑡 when 𝐶 = Tot 𝑡 and 𝐹 .𝑒𝑟𝑒𝑝𝑟 𝑡 𝑒 when 𝐶 = 𝐹 𝑡 𝑒 . We also elide the type 𝑡 from _𝑥 :𝑡 . 𝑒
when it is clear from the context.𝑀.bind projects the bind combinator for the effect𝑀 (for𝑀 = Tot,bind desugars to function application). Finally we write lift𝐶2
𝐶1
to mean lift𝑀2
𝑀1
where𝑀𝑖 is the effect
label of 𝐶𝑖 . For applications of the effect combinators, we elide the non-repr arguments; in the full
typing rules they are picked non-deterministically in the declarative style while, as we discuss later,
our implementation infers them using higher-order unification.
Proc. ACM Program. Lang., Vol. 1, No. CONF, Article 1. Publication date: January 2018.
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
1:16 Aseem Rastogi, Guido Martínez, Aymeric Fromherz, Tahina Ramananandro, and Nikhil Swamy
Rule T-Var is the standard variable typing rule with identity compilation to TotalF★. Rule T-Let
is the let-binding rule. Whereas EMF* had explicit monadic binds and lifts in the syntax, in LIF★,
monadic elaboration is type-directed and more accurately describes our implementation. The rule
first typechecks 𝑒1:𝐶1 and 𝑒2:𝐶2. Since𝐶1 and𝐶2 could have different effect labels, the rule lifts them
to a common effect𝑀 , and binds the resulting𝑀 computations. The rule is reminiscent of Swamy
et al.’s (2011) and Hicks et al.’s (2014) monadic elaboration rules, though both their calculi are
non-dependent. Concretely, the rule introduces two fresh variables 𝑓 : 𝐶1 and 𝑔 : 𝑥 :𝐶𝑡1→ 𝐶2,
applies the lift combinators to 𝑓 and 𝑔, and then applies the resulting computations to𝑀.bind. Thelet-binding is assigned the computation type 𝐶 and the compiled TotalF
★term is 𝑒 ′ with 𝑒 ′
1and 𝑒 ′
2
substituted for 𝑓 and 𝑔.
Rule T-Case is similar. It first typechecks the scrutinee 𝑒 and the two branches 𝑒1 and 𝑒2 under
appropriate assumptions. The rule then lifts the two branches to an effect𝑀 by applying the lift
combinators to fresh variables 𝑓1 and 𝑓2, and constructs the final TotalF★term with appropriate
substitutions as in T-Let. T-Reify and T-Reflect move back-and-forth between a computation
type and its representation. Interestingly, the elaboration of reify 𝑒 (resp. M.reflect 𝑒) is just theelaboration of 𝑒 ; reify and reflect are just identity coercions in LIF
★with no counterpart necessary
in TotalF★. In contrast, Filinski [1999] uses monadic reflection to structure the compilation of
monadic computations using state and continuations—we leave exploring this possibility to the
future, which may allow for more efficient compilation of user-defined effects.
We do not have the standard declarative subsumption rule for expression typing; instead we rely
on the rule T-As to explicitly trigger the subtyping of the computation types. With the placement of
lift coercions also directed by the syntax (rules T-Let and T-Case), this makes our typing judgment
purely syntax-directed. We discuss the coherence of multiple lifts and least upper bound of effects
in §3.2.
Rule C-M typechecks a computation type𝐶 by typechecking𝐶 . The computation type subtyping
rule SC-M delegates to subtyping on the underlying representations. Rule S-Conv shows the
subtyping rule based on type conversion using reduction; it compiles the two types and reduces
them in TotalF★. Similarly, the refinement subtyping rule (elided for space reasons) appeals to the
logical validity judgment in TotalF★after compiling the two types. We also elide the standard arrow
subtyping, reflexivity, and transitivity rules.
Our main theorem states that the LIF★translation to TotalF
★is well-typed.
Theorem 3.1. If 𝑆 ; Γ ⊢ 𝑒 : 𝐶 ⇝ 𝑒 ′, then 𝑆 ; Γ ⊢ 𝐶 : Type⇝ 𝑡 ′ and [| Γ |]𝑆 ⊢ 𝑒 ′ : 𝑡 ′.
Here, [| Γ |]𝑆 is the pointwise translation of the typing environment, and [| Γ |]𝑆 ⊢ 𝑒 ′ : 𝑡 ′ isthe typing judgment in TotalF
★. Using the theorem, a typing derivation in LIF
★can be soundly
interpreted in TotalF★. Ahman et al. [2017] prove EMF* normalizing, type-preserving, and a con-
sistency property for its refinement logic—these results also apply to its TotalF★fragment. Still,
TotalF★is not yet sufficient to model all of the F
★implementation, lacking a few important features,
notably, partial correctness for Swamy et al.’s (2016) divergence effect, universe polymorphism,
type conversion using provable equality, and inductive types. We plan to study these enhancements
in the future, grateful to no longer have to consider their interaction with Dijkstra monads.
The proof of the theorem is by mutual induction on the typing derivation with the following
lemmas:
Lemma 3.2 (Commutation of Subtyping).
(a) If 𝑆 ; Γ ⊢ 𝑡 <: 𝑡1 and 𝑆 ; Γ ⊢ 𝑡 : Type⇝ 𝑡 ′ then 𝑆 ; Γ ⊢ 𝑡1 : Type⇝ 𝑡 ′1and [| Γ |]𝑆 ⊢ 𝑡 ′ <: 𝑡 ′
1.
(b) If 𝑆 ; Γ ⊢ 𝐶 <: 𝐶1 and 𝑆 ; Γ ⊢ 𝐶 : Type⇝ 𝑡 ′ then 𝑆 ; Γ ⊢ 𝐶1 : Type⇝ 𝑡 ′1and [| Γ |]𝑆 ⊢ 𝑡 ′ <: 𝑡 ′
1.
Proc. ACM Program. Lang., Vol. 1, No. CONF, Article 1. Publication date: January 2018.
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
1:17
The essence of effect abstraction: Admissibility of using 𝑒𝑠𝑢𝑏𝑐𝑜𝑚𝑝 for subtyping. The reader may have
noticed that the rule SC-M breaks the effect abstraction by peeking into the effect representation
for checking subtyping. However, this is not necessary: we show the admissibility of checking
subtyping using the 𝑒𝑠𝑢𝑏𝑐𝑜𝑚𝑝 effect combinator.
The 𝑒𝑠𝑢𝑏𝑐𝑜𝑚𝑝 combinator for a user-defined effect 𝐹 is typechecked as follows:
𝑆 ; · ⊢ 𝑒𝑠𝑢𝑏𝑐𝑜𝑚𝑝 : 𝑡 : Type → 𝑥 : 𝑡 → 𝑓 : 𝑒𝑟𝑒𝑝𝑟 𝑡 𝑒{𝑡1} → 𝑒𝑟𝑒𝑝𝑟 𝑡 𝑒1 ⇝ _
and
𝑒𝑠𝑢𝑏𝑐𝑜𝑚𝑝 = _ 𝑡 𝑥 𝑓 . 𝑓
where 𝑡1 is a refinement formula. The intuition is that the combinator is a coercion that can be
used to coerce a computation from 𝐹 𝑡 𝑒 to 𝐹 𝑡 𝑒1, provided that the refinement formula is valid. The
implementation of 𝑒𝑠𝑢𝑏𝑐𝑜𝑚𝑝 is required to be an identity function. This is because our subtyping
relation is currently non-coercive. To prove 𝐹 𝑡 𝑒 <: 𝐹 𝑡 𝑒1, we check that:
𝑆 ; Γ, 𝑓 : 𝐹 .𝑒𝑟𝑒𝑝𝑟 𝑡 𝑒 ⊢ 𝐹 .𝑒𝑠𝑢𝑏𝑐𝑜𝑚𝑝 𝑓 : 𝐹 .𝑒𝑟𝑒𝑝𝑟 𝑡 𝑒1 ⇝ _
Behind the scenes, this typechecking judgment proves the refinement formula in the type of the
𝑒𝑠𝑢𝑏𝑐𝑜𝑚𝑝 combinator. The following lemma establishes the soundness of 𝑒𝑠𝑢𝑏𝑐𝑜𝑚𝑝 :
Lemma 3.3 (Soundness of 𝑒𝑠𝑢𝑏𝑐𝑜𝑚𝑝 ). If Δ ⊢ 𝐹 𝑡 𝑒 : Type ⇝ _, Δ ⊢ 𝐹 𝑡 𝑒1 : Type ⇝ _, andΔ, 𝑓 : 𝐹 .𝑒𝑟𝑒𝑝𝑟 𝑡 𝑒 ⊢ 𝐹 .𝑒𝑠𝑢𝑏𝑐𝑜𝑚𝑝 𝑓 : 𝐹 .𝑒𝑟𝑒𝑝𝑟 𝑡 𝑒1 ⇝ _, then Δ ⊢ 𝐹 𝑡 𝑒 <: 𝐹 𝑡 𝑒1.
The proof of the lemma unfolds the applications of 𝑒𝑠𝑢𝑏𝑐𝑜𝑚𝑝 to prove the subtyping of the
representations, after which an application of SC-M gives us the conclusion. In our implementation,
we use the 𝑒𝑠𝑢𝑏𝑐𝑜𝑚𝑝 combinator rather than implementing the SC-M rule as is, ensuring that effect
abstractions are preserved.
3.2 Implementation of Layered Indexed EffectsWe have implemented layered indexed effects in the F
★typechecker and all the examples presented
in the paper are supported by our implementation. Below we discuss some implementation aspects.
Coherence of lifts and effect upper bounds. When adding a lift𝑀1
𝑀to the signature, our implemen-
tation computes all the new lift edges that it induces via transitive closure. For example, if lift𝑀𝑀2
and lift𝑀3
𝑀1
already exist, this new lift induces a lift𝑀3
𝑀2
via composition. For all such new edges, if the
effects involved already have an edge between them, F★ignores the new edge and emits a warning
as such. Further, F★also ensures that for all effects𝑀 and𝑀1, either they cannot be composed or
they have a unique least upper bound. This ensures that the final effect𝑀 is unique in the rules
T-Let and T-Case. Finally, F★ensures that there are no cycles in the effects ordering.
Effect combinator for composing branches of a conditional. While in LIF★we have formalized a
dependent pattern matching case, our implementation allows for specifying an optional custom
effect combinator for combining branches. The shape of the combinator is as follows:
𝑆 ; · ⊢ 𝑒𝑐𝑜𝑛𝑑 : 𝑡 : Type → 𝑥 : 𝑡 → 𝑓 : 𝑒𝑟𝑒𝑝𝑟 𝑡 𝑒𝑡ℎ𝑒𝑛 → 𝑔 : 𝑒𝑟𝑒𝑝𝑟 𝑡 𝑒𝑒𝑙𝑠𝑒 → 𝑏 : 𝑏𝑜𝑜𝑙 → Type⇝ _
and
𝑒𝑐𝑜𝑛𝑑 = _ 𝑡 𝑥 𝑓 𝑔 𝑏. 𝑒𝑟𝑒𝑝𝑟 𝑡 𝑒𝑐𝑜𝑚𝑝𝑜𝑠𝑒𝑑
F★ensures the soundness of the combinator by checking that under the assumption 𝑏, the type
of 𝑓 is a subtype – as per the effect subcomp combinator – of the composed type, similarly for 𝑔
under the corresponding assumption 𝑛𝑜𝑡 𝑏. When the combinator is omitted, F★chooses a default
one that forces the computation type indices for the branches to be provably equal.
Inferring effect indices using higher-order unification. Our implementation relies on the higher-
order unifier of F★to infer effect indices and arguments of the effect combinators. For example,
suppose we have a computation type 𝐹 𝑡1 𝑒1 and we want to apply the lift𝐹 ′
𝐹 combinator, where:
Proc. ACM Program. Lang., Vol. 1, No. CONF, Article 1. Publication date: January 2018.
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
1:18 Aseem Rastogi, Guido Martínez, Aymeric Fromherz, Tahina Ramananandro, and Nikhil Swamy
𝑆 ; · ⊢ lift𝐹 ′
𝐹 .𝑒 : 𝑡 : Type → 𝑥 : 𝑡 → 𝐹 .𝑒𝑟𝑒𝑝𝑟 𝑡 𝑒𝑓 → 𝐹 ′.𝑒𝑟𝑒𝑝𝑟 𝑡 𝑒 ⇝ _
To apply this combinator, we create fresh unification variables for the binders 𝑡 and 𝑥 , and
substitute them with the unification variables in 𝐹 .𝑒𝑟𝑒𝑝𝑟 𝑡 𝑒𝑓 and 𝐹 ′.𝑒𝑟𝑒𝑝𝑟 𝑡 𝑒 , without unfoldingthe 𝑒𝑟𝑒𝑝𝑟 . We then unify 𝑡1 with the unification variable for 𝑡 , 𝑒1 with substituted 𝑒𝑓 , and return
(substituted) 𝐹 ′ 𝑡 𝑒 as the lifted computation type. This allow us to compute instantiations of the
combinators without reifying 𝐹 𝑡1 𝑒1 or reflecting the result type of lift. We follow this recipe for all
the effect combinators.
Support for divergence. Our implementation also supports the existing Div effect in F★for classi-
fying divergent computations. To ensure consistency, the logical core of F★is restricted to the pure
fragment, separated from Div using the effect type system. When defining layered indexed effects,
the repr types may encapsulate Div computations. A layered indexed effect𝑀 may optionally be
marked divergent. When so, the semantic termination checker of F★is disabled for 𝑀 computa-
tions. However, reification of such effects results in a Div computation to capture the fact that this
computation may diverge.
Polymonads. Polymonads are closely related to layered indexed effects. Hicks et al. [2014] define
a polymonad as a collection of unary type constructors¯M and a family of bind-like operators
with signature M𝑖 a → (a → M𝑗 b)→ M𝑘 b, together with a distinguished element of¯M called Id, cor-
responding to an identity monad. In our dependently typed setting, we consider all the M𝑖 to
be instances of the same indexed effect constructor, with the indices varying arbitrarily in bind,
and where Id corresponds to Tot. Sometimes, however, it can be convenient to also allow the
effect constructors to vary in bind (see §5 for an example). As such, our implementation also
supports writing polymonadic_bind (F1, F2) ▷ F = e, where 𝑒 is typechecked and applied similarly to
the single-monadic bind shown earlier, i.e.,
𝑆 ; · ⊢ 𝑒 : 𝑡1 𝑡2 : Type → 𝑥 : 𝑡 → 𝐹1.𝑒𝑟𝑒𝑝𝑟 𝑡1 𝑒𝑓 → (𝑥 :𝑡1 → 𝐹2.𝑒𝑟𝑒𝑝𝑟 𝑡2 𝑒𝑔) → 𝐹 .𝑒𝑟𝑒𝑝𝑟 𝑡2 𝑒 ⇝ _
Our implementation also allows writing polymonadic_subcomp F1 <: F2 = e, a construct that allows
lifting and subsuming F1 to F2 in a single step. In a way, polymonadic binds and subcomps allow
programmers to construct a polymonad openly and incrementally, with parallels to, e.g., the
instances of a typeclass. Both these constructs offer programmers finer control when combining
and relating computations in multiple effects, while the typechecker makes sure that the effect
ordering is still coherent.
4 CASE STUDY: MESSAGE FORMATTING FOR TLSLayered indexed effects are not just for defining new effect typing disciplines—effect layers stacked
upon existing effects can make client programs and proofs more abstract, without any additional
runtime overhead.We demonstrate this at work by layering an effect over EverParse [Ramananandro
et al. 2019], an existing library in F★for verified low-level binary message parsing and formatting.
Background: EverParse and Low★. EverParse is a parser generator for low-level binary message
formats, built upon a verified library of monadic parsing and formatting combinators. It produces
parsers and formatters verified for memory-safety (no buffer overruns, etc.) and functional correct-
ness (the parser is an inverse of the formatter). EverParse is programmed in Low★, a DSL in F
★
for C-like programming [Protzenko et al. 2017]. Low★’s central construct is the Stack effect which
models programming with mutable locations on the stack and heap, with explicit memory layout
and lifetimes. Its signature, given below, shows a Hoare monad.
effect Stack (a:Type) (pre:mem→ prop) (post:mem→ a → mem→ prop)
Programs in Stack may only allocate on the stack, while reading and writing both the stack and
the heap, with pre- and postconditions referring to mem, a region-based memory encapsulating both
Proc. ACM Program. Lang., Vol. 1, No. CONF, Article 1. Publication date: January 2018.
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
1:19
stack and heap. Low★provides fine-grained control for general-purpose low-level programming,
at the expense of low-level proof obligations related to spatial and temporal memory safety and
framing. When programming specialized code such as binary message formatters, layered indexed
effects offer a way to build domain-specific abstractions, liberating programs from memory safety
proofs and many details of low-level message formats.
The problem: Existing code mired in low-level details. Consider, for instance, formatting a struct of
two 32-bit integer fields into a mutable array of bytes, a buffer U8.t in Low★parlance.
type example = { left: U32.t; right: U32.t }
EverParse generates a lemma stating that if the output buffer contains two binary representations
of integers back to back, then it contains a valid binary representation of an example struct value:
val example_intro mem (output: buffer U8.t) (offset_from: U32.t) : Lemma
(requires valid_from parse_u32 mem output offset_from ∧valid_from parse_u32 mem output (offset_to parse_u32 mem output offset_from))
(ensures valid_from parse_example mem output offset_from)
To format a value of this type, one must write code like this:
let write_example (output: buffer U8.t) (len: U32.t) (left right: U32.t) : Stack bool
(requires _m0 → live m0 output ∧ len == length output) (* memory safety *)(ensures _m0 success m1 → modifies output m0 m1 ∧ (* memory safety & framing *)(success =⇒ valid_from parse_example m1 output 0)) (* output correctness wrt. parser *)
= if len < 8 then false (∗ output buffer too small ∗)else let off1 = write_u32 output 0 left in let _ = write_u32 output off1 right in
let mem = get () in example_intro mem output 0; true
In this example, the user needs to reason about the concrete byte offsets: they need to provide
the positions where values should be written, relying on write_u32 returning the position just past
the 32-bit integer it wrote in memory. Then, they have to apply the validity lemma: satisfying
its precondition involves (crucially) proving that the writing of the second field write does not
overlap the first one, through Low★memory framing. These proofs are here implicit but still incur
verification cost to the SMT solver; as the complexity of the structs increases, it has a significant
impact on the SMT proof automation. Moreover, the user also needs to worry about the size of the
output buffer being large enough to store the two integer fields.
4.1 The Write Layered EffectTo abstract low-level byte layout and error handling, we define a Write effect layered over Stack,
where an effectful computation f : Write t pbefore pafter returns a value of type t while working
on a hidden underlying mutable buffer. Each such computation requires upon being called that the
buffer contains binary data valid according to the pbefore parser specification, and ensures that,
if successful, it contains binary data valid according to pafter on completion. Thus, such a Write
computation presents the user with a simple parser-indexed parameterized state and error monad,
in such a way that memory safety, binary layout and error propagation details are all hidden away
from the user at once. Returning to the example above, we can define:
(∗ Elementarily write an integer ∗)val write_u32 : U32.t → Write unit parse_empty parse_u32
(∗ A generic higher−order framing operator, to be able to write two pieces of data in a row ∗)val frame (#t: Type) (#pframe #pafter : parser) (f: unit→ Write t parse_empty pafter)
: Write t pframe (parse_pair pframe pafter)
Proc. ACM Program. Lang., Vol. 1, No. CONF, Article 1. Publication date: January 2018.
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
1:20 Aseem Rastogi, Guido Martínez, Aymeric Fromherz, Tahina Ramananandro, and Nikhil Swamy
(∗ A lifting of the serialization lemma, with all details on binary layout hidden. Computationally a no−op. ∗)val write_example_correct : unit→ Write unit (parse_pair parse_u32 parse_u32) parse_example
This last lemma states that, if the output buffer contains valid data for parsing a pair of two integers,
then calling this function will turn that data into valid data for parsing an example struct value. With
those components, the user can now write their formatting code more succinctly, as shown below.
The output buffer, offsets and error propagation are hidden in the effect and so the user no longer
needs to explicitly reason about them. Furthermore, the code becomes much more self-explanatory.
let write_example (x1 x2 : U32.t) : Write unit parse_empty parse_example =
write_u32 x1; frame (_ _ → write_u32 x2); write_example_correct ()
Optimizing Verification. Thanks to the abstraction offered by the Write effect, it is possible to sparethe user from even more mundane details, such as having to explicitly call write_example_correct.
We can embed parser rewriting rules in the subsumption rule for Write. In other words, if repr t
pbefore pafter is the type of the underlying representation of a Write computation, then we can
define a subcomp rule for automatic rewriting of the pafter parser:
val subcomp (t: Type) (p1 p2 p2' : parser) (r: repr t p1 p2) : Pure (repr t p1 p2')
(requires (valid_rewrite p2 p2')) (ensures (_ _ →⊤))
where valid_rewrite p2p2' is a relation on parser specifications stating that any binary data valid
for p2 is also valid for p2'. Since the valid_rewrite goals can automatically be solved via SMT, this
allows us to rewrite our example as simply (write_u32 x1; frame (__ → write_u32 x2)). The only
overhead that remains is framing, which we aim to automate using the techniques of §5.2.
Representing Write: A peek beneath the covers. The representation of Write is interesting, involvinga dependent pair of indexed monads, where p.datatype is the type of the values parsed by p.
type write (t : Type) pbefore pafter = (spec : write_spec t pbefore.datatype pafter.datatype
& write_impl t pbefore pafter spec)
The first field, spec, is a specificational parameterized monad evolving an abstract state from
pbefore.datatype to pafter.datatype. The second field is the Low★implementation, indexed by a
pair of parsers and the spec. As such, write_impl is a parameterized-monad-indexed monad, or a
form of parameterized Dijkstra monad, a novel construction, as far as we are aware.
To compile a Write computation to C, or call it from other Low★code, we simply reify it and
project its Low★implementation:
let reify_spec #t #pbefore #pafter (f: unit→ Write t pbefore pafter)
: write_spec t pbefore.datatype pafter.datatype = (reify (f ())).1
(∗ Extract the Low∗ code of a computation to compile it to C ∗)let reify_impl #t #pbefore #pafter (f: unit→ Write t pbefore pafter)
: write_impl t pbefore pafter (reify_spec f)) = (reify (f ())).2
4.2 Application: TLS 1.3 Handshake ExtensionsWe have applied the Write effect to generate the list of extensions of a TLS 1.3 [Rescorla 2018]
ClientHello handshake message, that a client sends to a server to specify which cipher suites and
other protocol extensions it supports. This is the most complex part of the handshake message
format, involving much more than just pairs: it involves variable-sized data and lists prefixed by
their size in bytes, as well as tagged unions where the parser of the payload depends on the value
of the tag. Our implementation of ClientHello messages written using the Write effect compiles to
Proc. ACM Program. Lang., Vol. 1, No. CONF, Article 1. Publication date: January 2018.
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1:21
C and executes; we are currently integrating it into a low-level rewriting of an implementation of
TLS in F★[Bhargavan et al. 2013].
As an excerpt of this example, we show how to write a variable-sized list of 32-bit integers, the
list being prefixed by its size in bytes. The TLS 1.3 RFC [Rescorla 2018], specifying the data format
description for handshake messages, bounds the size of every such list, excluding the size header,
independently of the size of the actual output buffer. If p is a parser for the elements of the list, then
parse_vllist p min max is a parser that first reads an integer value that will be the storage size of
the list in bytes, checks that it is between min and max, then parses the list of elements using p for
each. Thus, the following code excerpt writes a list of two integers into the output buffer:
let write_two_ints (max_list_size: U32.t)
: Write unit parse_empty (parse_vllist parse_u32 0 max_list_size) =
write_vllist_nil parse_u32 max_list_size;
frame (_ _ → write_u32 18ul);
extend_vllist_snoc ();
frame (_ _ → write_u32 42ul);
extend_vllist_snoc ()
Following the TLS 1.3 RFC, the value of max_list_size constrains the size of the size header, but
thanks to the abstraction provided by Write, the user does not need to know about that actual size.
Indeed, that code relies on two combinators:
val write_vllist_nil (p: parser) (max: U32.t) : Write unit parse_empty (parse_vllist p 0 max)
val extend_vllist_snoc (#p: parser) (#min #max: U32.t) ()
: Write unit (parse_pair (parse_vllist p min max) p) (parse_vllist p min max)
write_vllist_nil actually starts writing an empty list by writing 0 as its size header. Then,
extend_vllist_snoc assumes that the output buffer contains some variable-sized list immediately
followed by an additional element and “appends” the element into the list by just updating the size
header of the list; thus, the new element is not copied into the list, since it is already there at the
right place. extend_vllist_snoc also dynamically checks whether the size of the resulting list is
still within the bounds expected by the parser, and exits with error if not.
A more powerful version of the Write effect with support for Hoare-style pre- and postconditions
to prove functional correctness properties on the actual values written to the output buffer, as well
as error postconditions, in addition to correctness wrt. the data format, is underway. With such
an enhanced version, we leverage pre- and postconditions to avoid dynamic checks on writing
variable-size list items.
5 CASE STUDY: STRUCTURING A DSL FOR SEPARATION LOGICSteel [Swamy et al. 2020] is a shallow embedding of concurrent separation logic (CSL) in F
★,
SteelAtomicA SteelAtomicF
SteelA SteelF
SteelKA SteelKF
with support for atomic computations and dynam-
ically allocated invariants. In this section, we show
how we use layered effects to provide a DSL for
Steel better suited for practical verification. We first
show how to seamlessly interoperate between non-
atomic, atomic, and ghost computations using in-
dexed effects. We then demonstrate how we use
polymonads to automate frame inference in Steel,
and re-implement a library for fork/join concurrency.
Saving the programmer from CPS’ing their code, we
build a final layered effect for continuations on top
Proc. ACM Program. Lang., Vol. 1, No. CONF, Article 1. Publication date: January 2018.
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1:22 Aseem Rastogi, Guido Martínez, Aymeric Fromherz, Tahina Ramananandro, and Nikhil Swamy
of our construction to provide a Unix-like interface
for fork and join, which also benefits from a high-
level of automation for framing through polymonads. We summarize the scaffolding of effects we
use in the figure alongside. Each effect appears in two forms, an A-form for “annotated”, and an
F-form for “framed”—the paired structure is in support of frame inference. Simple lines represent
binds, dashed lines represent subsumption relations, and dotted lines correspond to sub-effects.
5.1 Effects to Structure a Sub-Language of Atomic and Ghost ComputationsSteel enables specification of pre- and postconditions of functions using extensible separation logic
assertions of type slprop. Steel functions have their own effect, a Hoare monad parameterized by a
return type a, a precondition fp, and a postcondition fp' that depends on the return value.5The
program logic is designed for partial correctness, and relies on F★’s existing support for divergence
to model general recursive, potentially deadlocking Steel programs—the divergent annotation on
the effect definition below indicates this.
let steel (a:Type) (fp:slprop) (fp':a → slprop) = (∗ elided, inherited from Swamy et al. [2020] ∗)let bind (f:steel a fp0 fp1) (g:(x:a)→ steel b (fp1 x) fp2) : steel b fp0 fp2 = ...
divergent effect { Steel (a:Type) (fp:slprop) (fp':a → slprop) = steel with bind; ... }
To reason about concurrent programs, the Steel program logic provides a model of total atomic
actions, as well as dynamically allocated invariants that can be freely shared between threads,
similarly to other CSL-based frameworks like Iris [Jung et al. 2018]. Invariants can only be safely
accessed and restored by atomic actions. To assist with verification, Steel also supports a notion of
ghost state, as well as ghost, computationally irrelevant computations. The composition of a ghost
action with an atomic action is also deemed atomic, since ghost code does not actually execute; but
two non-ghost actions cannot be composed atomically.
To distinguish atomic and non-atomic code, we define a new effect, SteelAtomic.
let atomic (a:Type) (u:inames) (g:bool) (fp:slprop) (fp' : a → slprop) = ...
let bind_atomic #a #b #u #fp0 #fp1 #fp2 #g1 (#g2:bool{g1 || g2})
(f:atomic a u g1 fp0 fp1) (g:(x:a) → atomic b u g2 (fp1 x) fp2) : atomic b u (g1 && g2) fp0 fp2 = ...
effect { SteelAtomic (a:Type) (u:inames) (g:bool) (fp:slprop) (fp' : a → slprop) = atomic with ... }
SteelAtomic enhances the Steel effect with two additional indices: a set of currently opened
invariants u, and a boolean g indicating whether the computation is ghost. The type inames is an
abbreviation for a set of invariant names. Note, bind_atomic is partial: the safe composition of ghost
and atomic computations is captured in the refinement on g2: to compose two atomic computations,
one of them has to be ghost.
Using this effect, we can then expose the atomic opening and closing of an invariant as provided
by the program logic. Given an invariant i protecting a separation logic assertion p, an atomic
computation f is allowed to access p as long as p is restored once the computation terminates.
Additionally, i must not be in the set of currently opened invariants u to prevent recursive opening
of invariants. In a sense, with_inv parallels to effect handling in §2.2. Given f which uses invariants
i ⊎ u, with_inv handles i, leaving a computation that only uses invariants u.
val with_inv (i:inv p) (f:unit→ SteelAtomic a (i ⊎ u) g (p ∗ q) (_ x→ p ∗ r x)) : SteelAtomic a u g q r
Since atomic computations are a subset of computations, SteelAtomic is a sub-effect of the
generic Steel effect. Any SteelAtomic computation with no opened invariant can be lifted to a
Steel computation. The use of sub-effecting enables a smooth interoperation between the two
5In practice, the Steel effect contains two additional indices for heap predicates, enabling specifications in the style of
implicit dynamic frames. We omit these indices in this paper: these predicates are always trivial in the examples we present.
Proc. ACM Program. Lang., Vol. 1, No. CONF, Article 1. Publication date: January 2018.
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1:23
effects: a SteelAtomic computation used in a Steel context is lifted automatically, removing this
burden from the developer.
5.2 Controlling Frame Inference with a PolymonadThe bind function presented in section 5.1 requires the postcondition of the first computation to
match exactly with the precondition of the second computation. If the two computations operate on
different resources, it forces a programmer to reason manually about framing each resource. Such
reasoning becomes quickly intractable, as it is required for each composition. Thus, automating
framing is unavoidable in a separation logic-based verification framework.
To this end, one possible approach is to incorporate framing into every bind. Let us consider
two functions f: unit→ Steel unit p (__ → q) and g: unit→ Steel unit r (__ → s). Then, for any
slprops fp and fp', we can apply framing to typecheck f (); g () as Steel unit (p ∗ fp) (__ → s ∗ fp')
as long as the framed postcondition of the first computation, q ∗ fp, is equivalent to the framed
precondition of the second computation, r ∗ fp'. Applying framing at every bind, the number of
frames in the specification will grow linearly with the number of compositions. If the frames
are given, one can partially decide equivalence by associativity-commutativity rewriting in the
commutative monoid (slprop, ∗, emp). But automatically inferring them becomes tricky: when given
an equivalence with more than one frame to infer, several non-equivalent solutions might be
admissible. If for instance p ∗ fp ∗ fp' is equivalent to p ∗ q with fp and fp' to be inferred, then both
fp = q, fp' = emp and fp = emp, fp' = q are valid solutions.
The approach we propose instead is to apply the frame rule as little as possible. If a computation
has already been framed, we do not frame it again when composing it. Then, each pre- and
postcondition contains at most one frame to be inferred. To distinguish framed and not-yet-framed
computations, we present a polymonad based on two effects SteelF, representing computations
types that are always in the form SteelF a (pre ∗ ?f) (_x→ post x ∗ ?f), where ?f : slprop is a frame
metavariable to be solved; and SteelA, a computation type with no metavariables. Although both
effects use the same underlying representation, composing a SteelF computation f with a SteelA
computation g yields a SteelF computation, effectively adding a frame to g. To control the addition
of frame variables around SteelA computations, we use polymonadic binds (§3.2), as shown below.
val bind_steelf_steela (split_post_f:squash (∀ x. post_f x ∼ pre_g x ∗ frame))(f: steel a pre_f post_f) (g: x:a → steel b (pre_g x) post_g) : steel b pre_f (_ y → post_g y ∗ frame)
polymonadic_bind (SteelF, SteelA) ⊲ SteelF = bind_steelf_steela
The main point of interest here is the introduction of the additional variable frame and the
split_post_f hypothesis, which requires proving that the postcondition of f (i.e., post_f x) can
be split into the precondition of g (i.e., pre_g x) and a frame, which is propagated across g and
appears in the postcondition, along with post_g y.6 Compare this to the bind presented in §5.1,
which requires post_f and pre_g to match exactly, without introducing a frame. The polymonadic
bind (SteelA, SteelF) ⊲ SteelF adds a frame around the first computation; (SteelA, SteelA) ⊲ SteelF
frames both; (SteelF, SteelF) ⊲ SteelF frames neither. Using this polymonad, we ensure that each
slprop appearing in a pre- or postcondition has at most one frame to be inferred. Exploiting this
invariant, we design a (partial) decision procedure for frame inference as an F★tactic, to which we
defer the resolution of all frame variables and goals like split_post_f.
From a programmer’s perspective, all Steel functions appear to have a SteelA computation type,
which ensures that they can be called and framed from any context in which the precondition is
satisfied. SteelF is but an internal artifact that provides a clean design to automate frame inference.
6We could use a separating implication -∗ instead of ∼ , though that would allow resources to be dropped silently.
Proc. ACM Program. Lang., Vol. 1, No. CONF, Article 1. Publication date: January 2018.
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1:24 Aseem Rastogi, Guido Martínez, Aymeric Fromherz, Tahina Ramananandro, and Nikhil Swamy
As any composition leads to a SteelF computation, the last missing piece is a way to automatically
retype a SteelF computation as SteelA in order to match a user-provided annotation—this is
achieved below, by lifting and subsuming SteelF to SteelA in a single step. The polymonadic
constructs provide the control needed to orchestrate frame inference.
val subcomp_steelf_steela (_:squash (pre_g ∼ pre_f)) (_:squash (∀ x. post_f x ∼ post_g x))
(f:steel a pre_f post_f) : steel a pre_g post_g
polymonadic_subcomp SteelF <: SteelA = subcomp_steelf_steela
This strategy of adding frames isn’t specific to the Steel effect: we augment the SteelAtomic
effect with an identical construction—we believe it would be also be well-suited for other program
logics that involve framing, such as the formatters from §4. We leave this as future work.
Using this approach, we re-implement several existing Steel libraries. We show below our
implementation of fork. It relies on par, the basic concurrency primitive provided by Steel for
structured parallelism. Using automated framing, the programmer no longer needs to call frame, or
the ghost h_intro_emp_l and h_elim_emp_l, yielding just let t = new_thread q in par (spawn f t) (g t)
and eliminating all proof clutter. The crossed out lines from the original code below are no longer
necessary.
val par (f:unit→ SteelA aL preL postL) (g:unit→ SteelA aR preR postR)
: SteelA (aL & aR) (preL ∗ preR) (_ (xL, xR) → postL xL ∗ postR xR)
let fork (f: (unit → SteelA unit p (_ _ → q))) (g: (thread q → unit→ SteelA unit r (_ _ → s)))
: SteelA unit (p ∗ r) (_ _ → s)
= h_intro_emp_l (p ‘star‘ r);
let t = frame (fun _ -> new_thread q ) (p ‘star‘ r) in
h_elim_emp_l (p ‘star‘ r);
par (spawn f t) (g t);
h_elim_emp_l s
5.3 Layering ContinuationsThe fork function presented above expects both the parent and child threads as arguments and
blocks until they both return. In this section, we show how layering an additional effect, SteelKA,
on top of SteelA allows us to provide a more idiomatic fork/join by capturing the continuation of
the parent thread within an effect. We begin with a steelK representation type of CPS functions
into SteelA, indexed by pre and post conditions. Each steelK function is frameable and parametric
in the final postcondition, but the final answer type is fixed to unit.
let steelK (a:Type) (pre : slprop) (post:a → slprop) =
#frame:slprop → #postf:slprop → f:(x:a → SteelA unit (frame ∗ post x) (_ _→ postf)) →SteelA unit (frame ∗ pre) (_ _ → postf)
effect { SteelKA (a:Type) (pre: slprop) (post: a → slprop) = steelK with ... }
Given SteelKA, we can provide a more direct signature for fork, taking only the function to be
spawned and returning a handle for the child thread. Its implementation basically calls the previous
fork passing the current continuation as the parent thread. One can then use join to wait for the
child thread, and obtain its postcondition q.
val fork (#p #q : slprop) (f : unit→ SteelKA unit p (_ _ → q)) : SteelKA (thread q) p (_ _ → emp)
val join (#q : slprop) (t : thread q) : SteelKA unit emp (_ _ → q)
By replicating our framing methodology for SteelKA (by introducing a sibling effect of framed
computations SteelKF), and lifting SteelA to SteelKA, we now have a C-like fork/join:
Proc. ACM Program. Lang., Vol. 1, No. CONF, Article 1. Publication date: January 2018.
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1:25
let example (r:ref int) : SteelKA (ref int) (pts_to r 0) (_ x → pts_to r 1 ∗ pts_to x 2) =
let p1 = kfork (_ _ → write r 1) in let x = alloc 2 in kjoin p1; x
In this small example, we initially take a reference to an integer as an argument, which we assume
to contain the (unused) value 0. We then fork, performing a write in the child thread while we
allocate a new reference in the parent thread. Both write and alloc are SteelA functions that are
automatically lifted to SteelKA.
6 RELATEDWORK & CONCLUSIONSWe have discussed several strands of related work throughout the paper. We focus here on relating
our work to two main themes not discussed in detail elsewhere.
Unifying frameworks for effectful programming and semantics. Given the variety of monad-
like frameworks for effects, a unifying theory that accounts for all the variants is a subject of
some interest. Tate’s (2013) productors and, equivalently, Hicks et al.’s (2014) polymonads are
attempts at subsuming frameworks, Tate focusing more on the semantics while Hicks et al. consider
programmability, with Bracker and Nilsson [2015] even providing a Haskell plugin for polymonad
programming. However, both frameworks cater to simply typed programs, and as such do not
account for inherently dependently typed constructs such as Hoare and Dijkstra monads. Orchard
et al. [2020] propose to unify graded and parameterized monads by moving to category-indexed
monads, studying them from a semantic perspective only, while also working in a simply typed
setting. Orchard and Petricek [2014] also propose a library to encode effect systems with graded
monads in Haskell. As far as we are aware, we are the first to provide a programming framework
that encompasses monad-like structures with arbitrary indexing in a dependently typed setting,
demonstrating its value for programming and proving.
Programming and proving with algebraic effects. Our library for algebraic effects is perhaps relatedmost closely to Brady’s (2013) Effects DSL in the dependently typed language Idris. The main points
of difference likely stem from what is considered idiomatic in Idris versus F★. For instance, the core
construct in the Idris DSL is a type indexed by a list of effects (similar to our tree a l)—whereas in
Idris the indexing is intrinsic, our trees are indexed extrinsically with a refinement type, enabling
a natural notion of subsumption on indices based on SMT-automated effect inclusion. Idris’ core
effects language is actually a parameterized monad—our supplement and full version of the paper
show a similarly parameterized version of our tree type. More specifically, by packaging our trees
into an effect, we benefit from F★’s monadic elaboration, avoiding the need for monadic syntax,
idiom brackets and the like, with implicit subsumption handled by SMT. Further, unlike Brady, we
provide a way to interpret Read-Write trees into a Dijkstra monad, enabling functional correctness
proofs. While have only taken initial steps in this direction, we appear to be the first to actually
verify stateful programs in this style. Maillard et al. [2019] propose a tentative semantics to interpret
algebraic effect handlers with Dijkstra monads, and use their approach to extrinsically verify the
totality of a Fibonacci program with general recursion. Our work builds on theirs, requires fixing
the interpretation of the operations, but yields a methodology to do a proof of stateful programs.
Besides, with layered indexed effects, we get to choose whether to work with Dijkstra monads
or not—in contrast, Maillard et al.’s framework cannot support the list-of-effects indexed graded
monad. Algebraic effects have also been embedded in Haskell in several styles, notably by Kiselyov
and Ishii’s (2015) “freer” monads, relying on encodings of dependent types in Haskell’s type system
to also index by a list of effect labels, while focusing also on efficient execution, a topic we have
not yet addressed for our Alg effect.
Proc. ACM Program. Lang., Vol. 1, No. CONF, Article 1. Publication date: January 2018.
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1:26 Aseem Rastogi, Guido Martínez, Aymeric Fromherz, Tahina Ramananandro, and Nikhil Swamy
Conclusions. Embracing the diversity of indexed monad-like constructions, and aiming to reap
their benefits when programming with effects in a dependently typed language, we have designed
and implemented layered indexed effects as a feature of F★. The gains are manifold, simplifying
F★’s core logic, enabling new abstractions for programming and proving, and simplifying the
construction of effectful programs. Despite several significant case studies using our new system,
we feel we have just scratched the surface—the possibilities of mixing a variety of new effect
disciplines to better structure programs and their proofs seem endless.
REFERENCESMartín Abadi, Anindya Banerjee, Nevin Heintze, and Jon G. Riecke. 1999. A Core Calculus of Dependency. In Proceedings of
the 26th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (San Antonio, Texas, USA) (POPL’99). Association for Computing Machinery, New York, NY, USA, 147–160. https://doi.org/10.1145/292540.292555
Danel Ahman, Cătălin Hriţcu, Kenji Maillard, Guido Martínez, Gordon Plotkin, Jonathan Protzenko, Aseem Rastogi, and
Nikhil Swamy. 2017. Dijkstra Monads for Free. In 44th ACM SIGPLAN Symposium on Principles of Programming Languages(POPL). ACM, 515–529. https://doi.org/10.1145/3009837.3009878
Robert Atkey. 2009. Parameterised notions of computation. Journal of Functional Programming 19 (2009), 335–376. Issue 3-4.
https://doi.org/10.1017/S095679680900728X
Andrej Bauer and Matija Pretnar. 2015. Programming with algebraic effects and handlers. J. Log. Algebraic Methods Program.84, 1 (2015), 108–123. https://doi.org/10.1016/j.jlamp.2014.02.001
Nick Benton, Andrew Kennedy, Martin Hofmann, and Lennart Beringer. 2006. Reading, Writing and Relations. In Program-ming Languages and Systems, 4th Asian Symposium, APLAS 2006, Sydney, Australia, November 8-10, 2006, Proceedings (Lec-ture Notes in Computer Science, Vol. 4279), Naoki Kobayashi (Ed.). Springer, 114–130. https://doi.org/10.1007/11924661_7
Karthikeyan Bhargavan, Cédric Fournet, Markulf Kohlweiss, Alfredo Pironti, and P Strub. 2013. Implementing TLS with
verified cryptographic security. In IEEE Symposium on Security and Privacy. 445–459.Jan Bracker and Henrik Nilsson. 2015. Polymonad Programming in Haskell. In Proceedings of the 27th Symposium on the
Implementation and Application of Functional Programming Languages (Koblenz, Germany) (IFL ’15). Association for
Computing Machinery, New York, NY, USA, Article 3, 12 pages. https://doi.org/10.1145/2897336.2897340
Edwin C. Brady. 2013. Programming and reasoningwith algebraic effects and dependent types. InACM SIGPLAN InternationalConference on Functional Programming, ICFP’13, Boston, MA, USA - September 25 - 27, 2013, Greg Morrisett and Tarmo
Uustalu (Eds.). ACM, 133–144. https://doi.org/10.1145/2500365.2500581
Arthur Charguéraud. 2011. Characteristic Formulae for the Verification of Imperative Programs. In Proceedings of the 16thACM SIGPLAN International Conference on Functional Programming (Tokyo, Japan) (ICFP ’11). ACM, New York, NY, USA,
418–430. https://doi.org/10.1145/2034773.2034828
Adam Chlipala, Gregory Malecha, Greg Morrisett, Avraham Shinnar, and Ryan Wisnesky. 2009. Effective Interactive Proofs
for Higher-order Imperative Programs. In Proceedings of the 14th ACM SIGPLAN International Conference on FunctionalProgramming (ICFP ’09). 79–90. http://ynot.cs.harvard.edu/papers/icfp09.pdf
Karl Crary, Aleksey Kliger, and Frank Pfenning. 2005. A monadic analysis of information flow security with mutable state.
J. Funct. Program. 15, 2 (2005), 249–291. https://doi.org/10.1017/S0956796804005441
Dorothy E. Denning. 1976. A Lattice Model of Secure Information Flow. Commun. ACM 19, 5 (May 1976), 236–243.
https://doi.org/10.1145/360051.360056
Dominique Devriese and Frank Piessens. 2011. Information Flow Enforcement in Monadic Libraries. In Proceedings of the 7thACM SIGPLAN Workshop on Types in Language Design and Implementation (Austin, Texas, USA) (TLDI ’11). Associationfor Computing Machinery, New York, NY, USA, 59–72. https://doi.org/10.1145/1929553.1929564
Andrzej Filinski. 1999. Representing Layered Monads. In 26th ACM SIGPLAN-SIGACT Symposium on Principles of Program-ming Languages. ACM, 175–188. https://doi.org/10.1145/292540.292557
Michael Hicks, Gavin M. Bierman, Nataliya Guts, Daan Leijen, and Nikhil Swamy. 2014. Polymonadic Programming. In
Proceedings 5th Workshop on Mathematically Structured Functional Programming, MSFP@ETAPS 2014, Grenoble, France, 12April 2014 (EPTCS, Vol. 153), Paul Levy and Neel Krishnaswami (Eds.). 79–99. https://doi.org/10.4204/EPTCS.153.7
Ralf Jung, Robbert Krebbers, Jacques-Henri Jourdan, Ales Bizjak, Lars Birkedal, and Derek Dreyer. 2018. Iris from the
ground up: A modular foundation for higher-order concurrent separation logic. J. Funct. Program. 28 (2018), e20.
https://doi.org/10.1017/S0956796818000151
Shin-ya Katsumata. 2014. Parametric effect monads and semantics of effect systems. In The 41st Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL ’14, San Diego, CA, USA, January 20-21, 2014, SureshJagannathan and Peter Sewell (Eds.). ACM, 633–646. https://doi.org/10.1145/2535838.2535846
Proc. ACM Program. Lang., Vol. 1, No. CONF, Article 1. Publication date: January 2018.
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1:27
Oleg Kiselyov and Hiromi Ishii. 2015. Freer Monads, More Extensible Effects. SIGPLAN Not. 50, 12 (Aug. 2015), 94–105.https://doi.org/10.1145/2887747.2804319
Daan Leijen. 2014. Koka: Programming with Row Polymorphic Effect Types. In Proceedings 5th Workshop on MathematicallyStructured Functional Programming, MSFP 2014, Grenoble, France, 12 April 2014. (EPTCS, Vol. 153), Paul Levy and Neel
Krishnaswami (Eds.). 100–126. https://doi.org/10.4204/EPTCS.153.8
Daan Leijen. 2017. Type directed compilation of row-typed algebraic effects. In Proceedings of the 44th ACM SIGPLANSymposium on Principles of Programming Languages, POPL 2017, Paris, France, January 18-20, 2017, Giuseppe Castagnaand Andrew D. Gordon (Eds.). ACM, 486–499. http://dl.acm.org/citation.cfm?id=3009872
Thomas Letan and Yann Régis-Gianas. 2020. FreeSpec: specifying, verifying, and executing impure computations in Coq. In
Proceedings of the 9th ACM SIGPLAN International Conference on Certified Programs and Proofs, CPP 2020, New Orleans,LA, USA, January 20-21, 2020, Jasmin Blanchette and Catalin Hritcu (Eds.). ACM, 32–46. https://doi.org/10.1145/3372885.
3373812
Paul Blain Levy. 2004. Call-By-Push-Value: A Functional/Imperative Synthesis. Semantics Structures in Computation, Vol. 2.
Springer.
Sam Lindley, Conor McBride, and Craig McLaughlin. 2017. Do Be Do Be Do. In Proceedings of the 44th ACM SIGPLANSymposium on Principles of Programming Languages (Paris, France) (POPL 2017). Association for Computing Machinery,
New York, NY, USA, 500–514. https://doi.org/10.1145/3009837.3009897
Kenji Maillard, Danel Ahman, Robert Atkey, Guido Martínez, Cătălin Hriţcu, Exequiel Rivas, and Éric Tanter. 2019. Dijkstra
Monads for All. Proc. ACM Program. Lang. 3, ICFP, Article 104 (July 2019), 29 pages. https://doi.org/10.1145/3341708
Guido Martínez, Danel Ahman, Victor Dumitrescu, Nick Giannarakis, Chris Hawblitzel, Cătălin Hriţcu, Monal
Narasimhamurthy, Zoe Paraskevopoulou, Clément Pit-Claudel, Jonathan Protzenko, Tahina Ramananandro, Aseem
Rastogi, and Nikhil Swamy. 2019. Meta-F*: Proof Automation with SMT, Tactics, and Metaprograms. In 28th EuropeanSymposium on Programming (ESOP). Springer, 30–59. https://doi.org/10.1007/978-3-030-17184-1_2
Eugenio Moggi. 1989. Computational Lambda-Calculus and Monads. In Proceedings of the Fourth Annual Symposiumon Logic in Computer Science (LICS ’89), Pacific Grove, California, USA, June 5-8, 1989. IEEE Computer Society, 14–23.
https://doi.org/10.1109/LICS.1989.39155
Aleksandar Nanevski, J. Gregory Morrisett, and Lars Birkedal. 2008. Hoare type theory, polymorphism and separation. J.Funct. Program. 18, 5-6 (2008), 865–911. http://ynot.cs.harvard.edu/papers/jfpsep07.pdf
Dominic Orchard, Philip Wadler, and Harley Eades III. 2020. Unifying graded and parameterised monads. In ProceedingsEighth Workshop on Mathematically Structured Functional Programming, MSFP@ETAPS 2020, Dublin, Ireland, 25th April2020 (EPTCS, Vol. 317), Max S. New and Sam Lindley (Eds.). 18–38. https://doi.org/10.4204/EPTCS.317.2
Dominic A. Orchard and Tomas Petricek. 2014. Embedding effect systems in Haskell. In Proceedings of the 2014 ACMSIGPLAN symposium on Haskell, Gothenburg, Sweden, September 4-5, 2014, Wouter Swierstra (Ed.). ACM, 13–24. https:
//doi.org/10.1145/2633357.2633368
Gordon D. Plotkin and John Power. 2003. Algebraic Operations and Generic Effects. Applied Categorical Structures 11, 1(2003), 69–94. https://doi.org/10.1023/A:1023064908962
Gordon D. Plotkin and Matija Pretnar. 2009. Handlers of Algebraic Effects. In Programming Languages and Systems, 18thEuropean Symposium on Programming, ESOP 2009, Held as Part of the Joint European Conferences on Theory and Practiceof Software, ETAPS 2009, York, UK, March 22-29, 2009. Proceedings (Lecture Notes in Computer Science, Vol. 5502), GiuseppeCastagna (Ed.). Springer, 80–94. https://doi.org/10.1007/978-3-642-00590-9_7
Jonathan Protzenko, Jean-Karim Zinzindohoué, Aseem Rastogi, Tahina Ramananandro, Peng Wang, Santiago Zanella-
Béguelin, Antoine Delignat-Lavaud, Cătălin Hriţcu, Karthikeyan Bhargavan, Cédric Fournet, and Nikhil Swamy. 2017.
Verified Low-Level Programming Embedded in F*. PACMPL 1, ICFP (Sept. 2017), 17:1–17:29. https://doi.org/10.1145/
3110261
Tahina Ramananandro, Antoine Delignat-Lavaud, Cédric Fournet, Nikhil Swamy, Tej Chajed, Nadim Kobeissi, and Jonathan
Protzenko. 2019. EverParse: Verified Secure Zero-Copy Parsers for Authenticated Message Formats. In Proceedings of the28th USENIX Conference on Security Symposium (Santa Clara, CA, USA) (USENIX Security 2019). USENIX Association,
USA, 1465–1482.
E. Rescorla. 2018. The Transport Layer Security (TLS) Protocol Version 1.3. IETF RFC 8446. https://tools.ietf.org/html/rfc8446
Alejandro Russo, Koen Claessen, and John Hughes. 2008. A Library for Light-Weight Information-Flow Security in Haskell.
In Proceedings of the First ACM SIGPLAN Symposium on Haskell (Victoria, BC, Canada) (Haskell ’08). Association for
Computing Machinery, New York, NY, USA, 13–24. https://doi.org/10.1145/1411286.1411289
Nikhil Swamy, Nataliya Guts, Daan Leijen, and Michael Hicks. 2011. Lightweight monadic programming in ML. In Proceedingof the 16th ACM SIGPLAN international conference on Functional Programming, ICFP 2011, Tokyo, Japan, September 19-21,2011 (ICFP ’11). 15–27. https://www.cs.umd.edu/~mwh/papers/swamy11monad.html
Nikhil Swamy, Cătălin Hriţcu, Chantal Keller, Aseem Rastogi, Antoine Delignat-Lavaud, Simon Forest, Karthikeyan Bharga-
van, Cédric Fournet, Pierre-Yves Strub, Markulf Kohlweiss, Jean-Karim Zinzindohoué, and Santiago Zanella-Béguelin.
Proc. ACM Program. Lang., Vol. 1, No. CONF, Article 1. Publication date: January 2018.
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1:28 Aseem Rastogi, Guido Martínez, Aymeric Fromherz, Tahina Ramananandro, and Nikhil Swamy
2016. Dependent Types and Multi-Monadic Effects in F*. In 43rd ACM SIGPLAN-SIGACT Symposium on Principles ofProgramming Languages (POPL). ACM, 256–270. https://www.fstar-lang.org/papers/mumon/
Nikhil Swamy, Aseem Rastogi, Aymeric Fromherz, Denis Merigoux, Danel Ahman, and Guido Martínez. 2020. SteelCore:
An Extensible Concurrent Separation Logic for Effectful Dependently Typed Programs. https://www.fstar-lang.org/
papers/steelcore/steelcore.pdf
Nikhil Swamy, Joel Weinberger, Cole Schlesinger, Juan Chen, and Benjamin Livshits. 2013. Verifying Higher-order Programs
with the Dijkstra Monad. In Proceedings of the 34th annual ACM SIGPLAN conference on Programming Language Designand Implementation (PLDI ’13). 387–398. https://www.microsoft.com/en-us/research/publication/verifying-higher-order-
programs-with-the-dijkstra-monad/
Ross Tate. 2013. The Sequential Semantics of Producer Effect Systems. In Proceedings of the 40th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (Rome, Italy) (POPL ’13). Association for Computing
Machinery, New York, NY, USA, 15–26. https://doi.org/10.1145/2429069.2429074
Philip Wadler. 1992. The Essence of Functional Programming. In Conference Record of the Nineteenth Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, Albuquerque, New Mexico, USA, January 19-22, 1992, RaviSethi (Ed.). ACM Press, 1–14. https://doi.org/10.1145/143165.143169
Lantian Zheng and Andrew C. Myers. 2007. Dynamic Security Labels and Static Information Flow Control. Int. J. Inf. Secur.6, 2-3 (March 2007), 67–84.
Proc. ACM Program. Lang., Vol. 1, No. CONF, Article 1. Publication date: January 2018.