1

Comparing Semantic and Syntactic Methods in Mechanized Proof Frameworks

C.J. Bell, Robert Dockins, Aquinas Hobor, Andrew W. Appel, David Walker

2

• In the last decade, dozens of researchers have been investigating proof-carrying code (PCC)

• These researchers have split into two camps:– those using syntactic proof methods– those using semantic proof methods

3

• We want to be able to investigate different proof methodologies, such as syntactic and semantic type systems

• The list-machine benchmark is– assembly language– operational semantics– type system specification– two implementaions of a type system

• This benchmark is– simple, so that it is easy to understand– modular, so that it is flexible– publically available at

• http://www.cs.princeton.edu/~appel/listmachine/2.0

List-Machine Benchmark

4

Changes to the List-Machine Benchmark for 2.0

• Implemented only in Coq

• Added a semantic type system

• Reorganized the framework

5

Outline

Introduction

• Organization of the List-Machine framework

• Extend the List Machine with fault tolerance

• Semantic and syntactic methods in large systems

6

Machine Specification

7

Modules

8

Modules

Typechecking Algorithm

Typechecker Soundness Proof

Type System

Type System Specification

Typechecking Algorithmcheck(Π,Ψ) = true

Typechecker Soundness Proofcheck(Π,Ψ) = true → Π⊢blocksΨ

Type SystemProves: Π⊢blocks Ψ → safe Ψ

Type System Specification•type operators•definitions of typing rules•statement of safety• Π⊢blocks Ψ → safe Ψ

9

Typechecking Algorithmcheck(Π,Ψ) = true

Typechecker Soundness Proofcheck(Π,Ψ) = true → Π⊢blocksΨ

Type SystemProves: Π⊢blocks Ψ → safe Ψ

Type System Specification•type operators•definitions of typing rules•statement of safety• Π⊢blocks Ψ → safe Ψ

10

Syntactic Type System

• Type operators defined inductively

• Typing rules defined inductively

• The type system is proven sound using metatheorems (progress & preservation) using induction over definitions.

Type System Specification

Syntactic Soundness ProofΠ⊢blocks Ψ → safe Ψ

11

Semantic Type System

reusable

Type System Specification

Semantic Soundness ProofΠ⊢blocks Ψ → safe ΨList Machine Hoare LogicΠ⊢blocks Ψ Π;Ψ⊢block ι:P Π;Ψ⊢instr P{ι}QModal Specification Logic

Modal Model Library

12

Outline

Introduction

Organization of the List-Machine framework

• Extend the List Machine with fault tolerance

• Semantic and syntactic methods in large systems

13

Fault Tolerance

• Extend the List-Machine framework to provide fault tolerance

– Requires non-trivial modifications to the framework

– Demonstrates the flexibility of the framework

14

Simple List-Machine Example(without faults)

Fault Model

• Single Event Upset– assume a fault will occur at most once

• A fault may change just one register’s value to any other value.

16

Simple List-Machine Example(with faults)

17

Fault-TolerantModified Machine Specification

18

Fault-Tolerant Example

19

Incorrect Fault-Tolerant Example

20

Is the modified code fault-tolerant?

• Fault tolerance becomes part of the safety property

• Type system ensures proper use of colors

• Model possible occurrences of faults

21

Modify the Operational Semantics

22

Modify the Operational Semantics

Branch instructions require green and blue computations to agree

23

FT SummarySemantic

Syntactic

Machine syntax

Operational semantics

Typechecker

Type systems

Definition of “safe” to include fault states

• Safety (colors, no faults)

Model faults

Safety in the presence of faults

24

Outline

Introduction

Organization of the List-Machine framework

Extend the List Machine with fault tolerance

• Semantic and syntactic methods in large systems

25

How Semantic and Syntactic Methods Scale

Princeton Foundational Proof-Carrying Code (FPCC)Vs.

Carnegie Mellon ConCert project

FPCC :: Semantic ConCert :: Syntactic

26

Common Traits

• Include a TAL for ML compiled to machine code• Goal: guarantee a memory property for

untrusted code• Written in Twelf• Industrial-strength TALs• Large systems

27

Composition

Trusted Computing Base

T + L + M << P

Machine – SPARC or x86 definitions

Logic – example: definition of modular arithmatic

Theorems – statement of the safety property

Proof

Checker – theorem checker for FPCC and a metatheorem checker for ConCert

28

Token count of TCB components

FPCC ConCert0

50000

100000

150000

200000

250000

300000

350000

400000

CheckerRuntimePolicyMachine DefinitionAxioms

29

Token count of TCB components

The TCBs are equivalent in size except for the Checker

FPCC ConCert0

5000

10000

15000

20000

25000

30000

CheckerRuntimePolicyMachine DefinitionAxioms

30

Interface Safety

Requires• updating the policy• moving the type system from Proof to Theorem

– now part of the TCB

Should the type system be semantic or syntactic?

31

Scaling Law

Semantic: new definition per type constructor

Syntactic: new definition per expression constructor

Toy systems have few expression constructors…

32

Real systems have more expression constructors than type constructors.

semantic methods require fewer definitions

Is the average type definition larger than theaverage typing rule?

33

In toy systems, typing rules are simple...

|- stmt_prim_lbladd_ADD_imm: judge_stmt (e_prim A (p_lbladd V1 (val_diff L0 Lab I2))) Prog L CCEnv AENV KL Ps Phi L' CCEnv AENV KL Ps' Phi' <- regbind A At Prog <- targetreg At Ar <- regbind_val Prog V1 Vt <- realreg Vt Vr <- diff_value Prog (val_diff L0 Lab I2) Vc <- imm13 Vc (c Vimm13) <- valueTy Prog KL Phi V1 (offset I1 (int pi= (addr Lab))) <- valueTy Prog KL Phi (val_diff L0 Lab I2) (offset I2 (diff L0 Lab)) <- check_lbladd_offset I1 I2 <- num_add I1 I2 I1+I2 <- venv_add\ Prog A (offset I1+I2 (int pi= (addr L0))) Phi Phi' <-decode_list L L' Ps Ps' (instr_ADD Vr (inject_imode Vimm13) Ar) = ...

34

How does this balance in FPCC & ConCert?

Semantic FPCC

Syntactic FPCC

ConCert (XTALT)

ConCert (TALT)

05000

100001500020000250003000035000

Size of Type System Specification

• FPCC’s semantic definitions are half the size of syntactic definitions for FPCC

• This will become even more pronounced according to the scaling law if the compiler wishes to generate more instructions.

35

Conclusion

Introduction

Organization of the List-Machine framework

Extend the List Machine with fault tolerance

Semantic and syntactic methods in large systems

36

Appendix

37

Modified Typing Rules

38

Modified Operational Semantics

w = (n,ρ,a) w = (n,ρ,a,ρ’,κ)• ρ’ – FT register store• κ – color store

(and equivalent for the syntactic system)

39

Modified Semantic Type System

39

40

List-Machine Benchmark 2.0

• Easily extended

• Facilitates small scale comparisons between many proof methods (semantic and syntactic).

41

• Compare how type systems scale between semantic and syntactic proof methods

Princeton’s Foundational Proof Carrying Code (FPCC)vs

Carnegie Mellon’s ConCert

42

Modules

43

Type System Specification

Typechecking Algorithm

Typechecker Soundness Proof

Type System

Typechecking Algorithmcheck(Π,Ψ) = true

Typechecker Soundness Proofcheck(Π,Ψ) = true → Π⊢blocksΨ

Type SystemΠ⊢blocks Ψ → safe Ψ

Type System Specification

44

Typechecking Algorithmcheck(Π,Ψ) = true

Typechecker Soundness Proofcheck(Π,Ψ) = true → Π⊢blocksΨ

Type SystemProves: Π⊢blocks Ψ → safe Ψ

Type System Specification•type operators•definitions of typing rules•statement of safety• Π⊢blocks Ψ → safe Ψ

45

Modules