Computer Science School of Computing Clemson University Specification and Reasoning in SE Projects Using a Web IDE Charles T. Cook (Clemson) Svetlana V

Computer Science School of Computing Clemson University

Specification and Reasoning in SE Projects Using a Web IDE

Charles T. Cook (Clemson)Svetlana V. Drachova-Strang (Limestone College)

Yu-Shan Sun (Clemson)Murali Sitaraman (Clemson)

Jeffrey C. Carver (Alabama)Joseph E. Hollingsworth (IU Southeast)

This research is funded in part by NSF grants CCF-0811748, CCF-1161916, DUE-1022191, and DUE-1022941.


Part I: Overview

School of Computing Clemson University

About Clemson

School of Computing has about 600 undergrads and 200 grads

Clemson University has 17,000 students

Located in a town (also Clemson) with an official population of about 13,000 in South Carolina

Has a football stadium with capacity 85,000

3


This talk

This talk is about undergrad courses, though the ideas have been used in both undergrad and grad courses for many years

4


Goals of the SE Projects

Role of formal specifications as contracts in team software development and integration

Role of formal specifications in reasoning about software correctness

Other uses Specification-based test case design

5


Intro. and Advanced Projects

Introductory projects involve only use of and reasoning with existing library components 2 weeks of lectures

Advanced projects involve development and reasoning about new components 3-5 weeks of lectures

Other variations

6


Sample Courses and Institutions

Sample introductory projects Alabama (Software Engineering) Cleveland State (Software Engineering)

Sample advanced projects Clemson (Software Engineering) Denison (Independent study projects)

Other variations Depauw (Theory), NC State (Data

structures), Southern Wesleyan (Data Structures), Ramapo College (Programming Languages),…

7


Clemson University Details

Two-course sequence Sophomore-Level, CP SC 215: Software

Development Foundations Junior/Senior-Level: CP SC 372: Software

Engineering Experimentation and assessment

Piloting (2007/2008) Institutionalization (2009 to present)

8


Soft. Dev. Foundations Course

Intro to Java, object-based computing, software engineering, design patterns, etc. 10 weeks (includes usual materials and

projects) Intro to formal specifications and

reasoning (interspersed with above topics) 4 weeks

Uses RESOLVE-style specifications in a Java context 9


Software Engineering Course

Software life cycle, process models, requirements analysis and design 8 weeks (includes usual materials and

projects) Transition from informal to formal

1 week Specification-based component

development and quality assurance 5 weeks

Uses RESOLVE10


Clemson University Key Points

Specification/reasoning included in syllabi for two required courses for majors (2009)

Graduating student learning outcome modified to include “development of bug-free software according to specifications” (2012)

11


Part II: Example Projects


RESOLVE

An integrated specification and programming language for verified, component-based software development

A Verifying compiler A github project A freely-available web interface to

use the verifying compiler: www.cs.clemson.edu/group/resolve

13


Sample Intro Assignments

Generate VCs and prove the given Queue Remove_Last operation.

Make each of the following changes and explain what is unprovable. Comment out the first Dequeue

operation. Change the maintaining clause (loop

invariant) to #Q = <E> o Q. Change the decreasing clause

(termination progress metric) to |T|.14


Sample Advanced Assignments

Implement and verify: Extension operations on Queues to

Insert_After and Remove_After Sequence_Template using

Queue_Template with extensions Multiple implementations of

Queue_Template satisfying given internal contracts

Assignments to put it all together Involve a dozen components and teams of 3

students15


Sample Components

Stack_Template Queue_Template Preemptable_Queue_Template Sequence_Template List_Template Search_Store_Template Map_Template Prioritizer_Template … 16


Part III: A Web IDE Demo


Getting Started

www.cs.clemson.edu/group/resolve Tab: Web IDE

Google: RESOLVE web IDE RESOLVE verifier Clemson RESOLVE

18


Sample Intro Assignments

Generate VCs and prove the given Queue Remove_Last operation.

Make each of the following changes and explain what is unprovable. Comment out the first Dequeue

operation. Change the maintaining clause (loop

invariant) to #Q = <E> o Q. Change the decreasing clause

(termination progress metric) to |T|.19


Elements of the Assignment

Queue_Template concept specification

A specification of Remove_Last Operation

An annotated implementation of Remove_Last operation

20


Mathematical Modeling

Concepts provide mathematical models for programming objects

To write formal specifications, we need to model the state mathematically

Some objects we use in programming, such as Integers and Reals, have implicit models

For others, such as stacks, queues, lists, etc., we need to conceive explicit mathematical models


Mathematical Modeling of Queues

Concept Queue_Template(type Entry; Max_Length: Integer);

uses String_Theory;

Type Family Queue is modeled by …

Operation Enqueue…Operation Dequeue……

end Queue_Template;


Concept Queue_Template(type Entry; Max_Length: Integer);

uses String_Theory;

Type Family Queue is modeled byStr(Entry);

exemplar Q;constraints |Q| <= Max_Length;initialization ensures Q = ;

…

end Queue_Template;

Mathematical Modeling of Queues


Operation Remove_Last (updates Q: Queue; replaces E:

Entry); requires |Q| /= 0; ensures #Q = Q o <E>;

A Specification of Remove_Last


Procedure Remove_Last (updates Q: Queue; replaces E:

Entry);Var T: Queue;Dequeue (E, Q);While (Length(Q) /= 0)do

Enqueue(E,T);Dequeue(E,Q);

end;Q :=: T;

end Remove_Last;

An Implementation of Remove_Last


Assume #Q = <10, 20, 30, 40>Dequeue (E, Q);While (Length(Q) /= 0)

T E QIter. 1 10 <20, 30,

40> Iter. 2 <10> 20 <30, 40>Iter. 3 <10, 20> 30 <40>Iter. 4 <10, 20, 30> 40 do

Enqueue(E,T);Dequeue(E,Q);

end;

Understanding Loop Invariant


Assume #Q = <10, 20, 30, 40>Dequeue (E, Q);While (Length(Q) /= 0)

T E Q

Iter. 1 10 <20, 30, 40> Iter. 2 <10> 20 <30, 40>

Iter. 3 <10, 20> 30 <40>Iter. 4 <10, 20, 30> 40

Loop maintains this invariant: #Q = T o <E> o Q

Understanding Loop Invariant


Procedure Remove_Last (updates Q: Queue; replaces E: Entry);

Var T: Queue;Dequeue (E, Q);While (Length(Q) /= 0)

maintaining #Q = T o <E> o Q;decreasing |Q|;

doEnqueue(E,T);Dequeue(E,Q);

end; …

An Implementation of Remove_Last


Verification

Press Verify Button Generate and prove automatically a

series of verification conditions (VCs) Students understand why the VCs

arise Understand connections between

contracts, code, and proofs Can prove VCs

29


Impact of Incorrect Code

The first statement, call to Dequeue, is removed.

Leads to one unprovable VC (Verification Condition)

VC 0_1: Base Case of the Invariant of While

Statement in Procedure Remove_Last: Remove_Last_Realiz.rb(8)

Goal: Q = ((empty_string o <E>) o Q) Givens: …

30


Part IV: Assessment


Assessment Summary

For details See the paper See Drachova Ph. D. dissertation (2013,

Clemson), available at website A quick summary

Likert items to assess perceived benefits of web IDE (avg. 4.0/5.0)

Project grades (avg. from 80%); about the same as non-formal parts

RCI-item based analysis32


What reasoning skills are necessary?Reasoning Concept Inventory

Boolean Logic Standard Logic Symbols, Standard Proof Techniques

Discrete Math Structures Sets, Strings, Numbers, Relations, and other mathematical theories as needed

Precise Specifications Mathematical Descriptions of Software interfaces for clients and implementers. Math models for structures. Pre and Post conditions for operations.

Modular Reasoning Each Module needs to be proven correct only once.

Correctness Proofs Mathematical Assertions equivalent to the correctness of the program.Application of Proof Techniques to the program

http://www.cs.clemson.edu/resolve/teaching/inventory.html


Clemson RCI-Based Evaluation

34

Sem # Avg. % scoring 70% or higher

RCI 3.4.3 3 82 87% 85%RCI 4.2 2 56 76% 59%

RCI 4.3.1 1 24 77% 67%

3.4.3: Precise specifications 4.2: Design by contract 4.3.1: Internal contracts


SE Project Benefits on RCI-Based Learning Outcomes

35

Avg. % scoring 70% or higher

Before project (mid-term)

59% 25%

After project (final)

84% 75%

RCI Topics: 3.4.2, 3.4.3: Precise specifications

Sample size: 24 students, Spring ‘12


Summary

Students can practice formal specification and reasoning principles effectively using the RESOLVE web IDE and its verifier There are some benefits for even a

minimal intro (3 lectures plus project) Significant benefits with longer-term

exposure IDE and materials online

www.cs.clemson.edu/group/resolve

Documents

Computer Science School of Computing Clemson University Specification and Reasoning in SE Projects Using a Web IDE Charles T. Cook (Clemson) Svetlana V