Why Art is Needed for Software Development

Le développement de logiciel, est-ce un art ?

Gregor v. Bochmann

School of Information Technology and Engineering (SITE)

University of Ottawa

September 26, 2002

My personal background• MSc in physics and mathematics; PhD in theoretical physics;

experience in computer programming; changed to computer science in 1971 (after PhD), professor at University of Montreal for 25 years

• People important for me– My mother was high-school teacher for music and visual arts (my

father, biologist, died in 1942 during the war)– The grandfathers of my father were both well-known artists– My high-school physics and mathematics teachers– My piano teacher

• Hobbies– Travelling, hiking, observing nature– Playing music: chamber music, orchestra– Setting up an inventory of the paintings of my great-grandfather

(see http://beethoven.site.uottawa.ca/MalerBochmann/ )

About this talk• Theme: Art Software Development

artisan, artist software engineer

Historical development – artisanal economy– first industrial revolution: energy and automation– second industrial revolution: information processing

Artisans, artists, software engineers– similarities and differences: the importance of inventions

The activities of a software engineerAutomation of software engineering

– research objectives– intrinsic limitations (the importance of inventions)

Some comments on my research contributions Conclusions

Art Software Development

artisan, artist software engineer

What do these people do ?– build something useful

– Artisan (shoe maker makes shoes)– Engineer (mechanical engineer designs a bridge)

– learn the rules of the profession– Shoe maker: how to make shoes– Engineer: physical laws concerning statics and dynamic mechanics– Artist (composer): rules of harmony

– produce something beautiful– elegant shoes – nice looking bridge– beautiful music

– Artists: convey a sentimental or social message– Mozart’s piano concert or Zola’s book

Economy based on artisans

Example of a shoe maker– builds something useful

– notes the client’s requirements: size, style of shoes, desired color, etc.

– follows the rules of the profession– selects the material to be used for different parts of the shoe– knows how to fabricate these parts– knows the best order in which to assemble them (this may be

adapted to the particular circumstances – inventions)

– produces something beautiful– adds on some ornamentation – inventions (as requested by

the client and/or according to his own invention and temperament)

Typically, each object is individually designed and fabricated

First industrial revolutionCharacteristics

• Energy replaces man or horse power• Chemical engineering: better materials• Automated manufacturing (e.g. assembly lines for cars,

automatic weaving, etc.)

People’s activities:1. Design product to be manufactured (inventions)

2. Design and build machines for manufacturing (inventions)

3. Assist the machines on the assembly lineNote: This is boring ! -- Why ?

Typically, many identical objects are manufactured for each design

Relative production cost decreases

first unit additional unit– House– Plane– Vinyl music album– MP3 music album– Income tax program– New e-commerce appl.

Software and digital media: no copying costs

Second industrial revolution (IT)

Consequences and open questions• Only activity (1) remains (to design the product -- need

for inventions)

• Free copying (e.g. Napster) vs. Copyright– or ”Open Source Software” ??

– Providing useful software components without remuneration

• Globalization and uniformization … – or sub-cultures ??

• Do we need more software engineers ?– No: software for all useful applications will soon be available

– Yes: the power of computer and communication hardware increases exponentially over time (Morse’s law); therefore many new applications will be possible in the future and they must be developed

Back to: art – science – engineering useful – rules – beautiful – message

• Simplicity – Beauty (or Triviality ?)– Nature– Mathematics– Paintings: too simple becomes trivial– Science: to find simple explanations for nature’s behavior; this is

never trivial– Engineering: simple, elegant designs are easier to verify, build

• Rules vs. Freedom– The free space allows the expression of the artist’s message– This is the space for inventions

» Bach’s “Inventions“» Communication through human or computer language» Engineering: design choices

– Filling the free space with random choices (e.g. automatic art)

Shoe Maker vs. Softw. Engineer– builds something useful

• notes the client’s requirements: size, style of shoes, desired color, etc.• notes the requirements of the system to be built

– follows the rules of the profession• selects the material to be used for different parts of the shoe• selects software components to be re-used• knows how to fabricate these parts• knows techniques for software design, implemention and testing • knows the best order in which to assemble them • follows a suitable SE process

– produces something beautiful• adds on some ornamentation (as requested by the client and/or according to

his own invention and temperament)• tries to built the system in a simple manner (as far as possible)• a program can be elegant, beautiful ! ?• where requirements leave freedom, designs a nice user interface and user

options

SE : an artisanal profession ?. . . as indicated by above comparison

• Most software has too many bugs !!

• Much software is created– without a well-defined development process– without the use of automated tools

• Sometimes, older disciplines look down to SE and computer science, which is a relatively young discipline– Each car manufacturer has a well-defined and optimized manufacturing process

– There is automation not only in the manufacturing process, but also during the design (e.g. simulation tools, static analysis packages, etc.)

• Much of my research: towards automation of SE

• Can the SE process be completely automated ?

An activity (in general)

IT activitiesAlgorithm: “process or rules for calculation” (Concise Oxford Dict.)

Program: algorithm coded in a language executable by a computer, or automatically translatable by a compiler into executable code

Example: The activity of sorting

Requirements (properties of output, depending on input):(1) output list is sorted by names (increasing order)(2) the set of output lines should be equal to the set of input lines

Activities

input outputprocessing

Fred 12 000Tom 7 000Alice 10 000Suzanne 5 000

sortingalgorithminput output

Alice 10 000 Fred 12 000Suzanne 5 000Tom 7 000

Example of a sorting algorithm

create empty

output list

extract an element from the input list;

call it E

insert E into the output list

at the appropriate place

is input list empty ?

yes

no

end

start

Activities of a programmer-analyst

Requirements AlgorithmFind (invent)

appropriate algorithmand describe itinput output

RequirementsFind appropriate

decomposition, reusing if possible existing modulesinput output

Algorithm(in executable

specification language)

executable code

Translationfrom high-level language

to executable codeinput output

Requirements“correct” or

“in error”Verification

(does the algor. or compositionsatisfy requirements ?)

inputoutput

A

C

B

Algorithm or composition

Some commentsCan these activities be

automated?

Try to establish reusable components

distributed applications, communication protocols

Can one find algorithms for automating the verification process ? (see next slides)

Completely automated: Compilers exist for programming languages and certain executable specification languages

Requirements AlgorithmFind (invent)

appropriate algorithmand describe it

input output

RequirementsFind appropriate

decomposition, reusing if possible existing modules

input output

Algorithm(in executable

specification language)

executable code

Automatic translationfrom high-level language

to executable codeinput output

Requirements

“correct” or “in error”

Verification(does the algor. or composition satisfy the required properties ?)

input

output

A

C

B

Algorithm or composition

Compiler

The “verification” activity

Requirements“correct” or

“in error”Verification

(does the algorithms satisfy the required properties ?)

inputAlgorithm or

composition includingalgorithms

output

Can the solution of this problem be automated ?

Can one find an algorithm that solves this problem ?

Two approaches: proving – testing - try to make a proof of the correctness

- execute the given algorithms with various inputs (test inputs) and check whether the obtained outputs satisfy the requirements

Verification: … towards a proof

• Different levels of formalization– How are the requirements and the algorithm defined ?

• Intuition, informal description / documentation, formal specification

• Undecidable problems– Important result of computer science: Certain problems

are undecidable (it is known that no algorithm exists that would solve them !!)

• The verification problem is undecidable

• Simplified cases of the verification problem are decidable– e.g. a proof (prepared by human) of the correctness of some algorithm may

be verified by a proof checking algorithm

Verification: … through testing

Approach: To execute the program (or to simulate the execution of an algorithm) with particular inputs and to verify that the obtained outputs satisfy the requirements

Difficulty: If one does not test for all possible input sequences (exhaustive testing) there is no assurance that all errors are found. Exhaustive testing is normally not possible.

Example: Test of a telephone switch supporting N lines (the case of M simultaneous connect requests to various destinations): I = (N)M (number of cases)

M=N = 4: I = 44 = 256 M=N=10: I = 1010 M=N=20: I = 2020 = 1026

Note: Many practical problems have an exponential complexity, that is, the execution time depends exponentially on the size of the input data (here the size of the switch)

Formal methods and related toolsSpecification languagesbased on predicate calculus: e.g. Z, for assertions

about inputs and outputs, invariant assertions (for requirements)

based on state machines (e.g. SDL) or process algebras (e.g. LOTOS); executable specification languages (for algorithms)

Algorithms for the derivation of protocol specifications

Equation solving (given A and C, find B)

Tools for verificationSemi-automatic theorem proversAutomated tools for exhaustive exploration of

reduced finite state models (model checking)Simulated execution environments for composed

systemsTools for test development with guaranteed

coverage criteria (non-exhaustive)

Requirements

“correct” or “in error”

Verification(does the algorithm satisfy the required properties ?)input

outputAlgorithm or executable program

RequirementsFind appropriate

decomposition, reusing if possible existing modules

input output

A

C

B

Requirements AlgorithmFind (invent)

appropriate algorithmand describe it

input output

human creativity

My own contributions (in red)

Development of state machine languages, including standardization of the Estelle language (similar to SDL and UML state charts): definition, execution environments, testing, application to communication protocol standards

Seminal work on protocol derivation and equation solving

Methods and tools for test development (in the context of an industrial research chair at the University of Montreal with Hewlett-Packard, 1989-97)

Now: Also applications in the area of distributed multimedia systems (teleconferencing, IP telephony) and photonic networks

Specification languagesbased on predicate calculus: e.g. Z, for assertions

about inputs and outputs, invariant assertions (for requirements)

based on state machines (e.g. SDL and Estelle) or process algebras (e.g. LOTOS); executable specification languages (for algorithms)

Algorithms for the derivation of protocol specifications

Equation solving (given A and C, find B)

Tools for verificationSemi-automatic theorem proversAutomated tools for exhaustive exploration of

reduced finite state modelsSimulated execution environments for composed

systemsTools for test development with guaranteed

coverage criteria (non-exhaustive)

Conclusions• Nature of software development

– To build according to requirements (somethings useful)– To follow rules of best practices; note that most activities cannot be

completely automated since the problems are either undecidable or of exponential complexity

– To invent new solutions – freedom of choice – creativity– de la programmation considérée comme un des beaux-arts (Pierre Lévy, 1992)

• Need for software engineers with– good communication skills to understand and to explain the problem and its

solution

– creativity to find appropriate system architectures, components and algorithms

( t( this is an Art )t )

– sense of rigor to verify that the constructed software satisfies the requirements and does not contain errors ( an

• Ongoing research tries to further automate the SE process, but it is not clear how far this process may go, given the inherent limitations