FromSoftware Architecture toSoftware Gardening: New ... · The Big Picture 3 Software Architecture...

From Software Architectureto Software Gardening:New Paradigms for the Developmentof Complex Adaptive Systems

Thomas Gabor, LMU MunichRoSI Lecture 2019-12-16

The Big Picture 2

Complex Adaptive Systems

Complexity

Dynamicity

Trust

The Big Picture 3

Software Architecture Complex Adaptive Systems

Complexity

Dynamicity

Trust

The Big Picture 4

Software Architecture Complex Adaptive Systems

Complexity

Dynamicity

Trust

preserve initial consistencythroughout development

The Big Picture 5

Software Architecture Complex Adaptive Systems

Complexity

Dynamicity

Trust

preserve initial consistencythroughout development

continously (re-)establish(partial) consistency frominitial inconsistency

The Big Picture 6

Software Architecture

Software Gardening?

continously (re-)establish(partial) consistency frominitial inconsistency

preserve initial consistencythroughout development

The Small Picture 7

The Small Picture 8

▹ robot starts top left

▹ must visit workstationsof specific colors

in specific order

▹ must not collidewith obstacles

▹ should choosethe shortest path

The Small Picture 9system test

System and Test 10

system test

Systems 11

situations

beha

vior

classical system

Adaptive Systems 12

classical system

adaptive system

situations

beha

vior

Adaptive Systems 13

classical system

adaptive system

situations

beha

vior

Adaptation could be realized by…OptimizationAlgorithms

Reinforcement Learning many more

Optimizing Adaptive Systems 14

classical system

adaptive system

situations

beha

vior

optimal behaviorwith respect togoal function

optimal behaviormay changedynamically

More Adaptive Systems? 15

situations

beha

vior

Orange is at least as adaptive as Red<=> Orange can solve at least the situations Red can solveAND Orange performs at least as close to the optimum as Red

System and Test 16

system test

adaptation

System and Test 17

System test

adaptation

bettersystem

System and Test 18

System test

adaptation

BesseresSystemeven

bettersystem

System and Test 19

System test

adaptation

BesseresSystemeven

bettersystem

Dieselgate problemIf an adaptive system is much more powerful than its test,then it may prefer tricking the test to solving its actual objective.

situations

beha

vior

Tests 20

test case

situations

beha

vior

Tests of Adaptive Systems 21

test case

scenarios

situations

beha

vior

Tests of Adaptive Systems 22

test casetest generator

situations

beha

vior

Tests of Adaptive Systems 23

test casetest generator

scenarios

logical constraints

requirements

test cases data points

use cases realized baseline

components

situations

beha

vior

Harder Tests? 24Green is at least as hard as Purple<=> For every scenario generated by Purple, Green generates a scenario that is at least as close to optimal behavior.

System and Test 25

System test

adaptation

BesseresSystemeven

bettersystem

adaptation

System and Test 26

System Test

adaptation

BesseresSystemeven

bettersystem

adaptation

bettertest

System and Test 27

System Test

adaptation

BesseresSystemeven

bettersystem

adaptation

bessererTesteven

bettertest

System and Test 28

System Test

adaptation

BesseresSystemeven

bettersystem

adaptation

bessererTesteven

bettertest

is testedby

System and Test 29

System Test

adaptation

BesseresSystemeven

bettersystem

adaptation

bessererTesteven

bettertest

is testedby

is used fortraining

System and Test 30

System Test

adaptation

BesseresSystemeven

bettersystem

adaptation

bessererTesteven

bettertest

is testedby

is used fortraining

arms race between system and testScenario Co-Evolution

Coevolution 31

situations

beha

vior

Coevolution 32

situations

beha

vior

Coevolution 33

situations

beha

vior

situations

beha

vior

Coevolution 34

situations

beha

vior

situations

beha

vior

situations

beha

vior

Coevolution 35

situations

beha

vior

situations

beha

vior

situations

beha

vior

When you have too many mice,keep a cat.When you need your lawn mown,keep sheep.

Scenario Co-Evolutionwith Reinforcement Learningand Evolutionary Algorithms

The Small Picture 37system test

The Small Picture 38system test

ReinforcementLearning

Random Generator

Reinforcement Learning 39

Training

Decision

Execution

Reward

RandomInitialization

Result

Training against Random Test 40System Test

ReinforcementLearning

Random Generator

Training against Random Test 41Test

System

Training against Random Test 42Test

System

Random Generator

Training against Random Test 43Test

Scenarios

System

Random Generator

Training against Random Test 44Test

Scenarios

System

Random Generator

reinforce

Training against Random Test 45Test

Scenarios

System

Random Generator

System

reinforce

update

Training against Random Test 46Test

Scenarios

System

Random Generator

System

reinforce

update

reinforce

Training against Random Test 47Test

Scenarios

System

Random Generator

System

reinforce

update

reinforce

updateSystem

Training against Random Test 48Test

Scenarios

System

Random Generator

System

reinforce

update

reinforce

updateSystem

reinforce

The Small Picture 49system test

ReinforcementLearning

EvolutionaryAlgorithm

Evolutionary Algorithms 50

RandomInitialization Selection

Mutation Recombination

Result

Training against SCoE Test 51System Test

ReinforcementLearning

EvolutionaryAlgorithm

Training against SCoE Test 52

Test

System

Random Generator

Training against SCoE Test 53

Test

System

Random Generator

System

reinforce

update

reinforce

Training against SCoE Test 54

Test

System

Random Generator

System

reinforce

update

reinforce

evolve

Test

Training against SCoE Test 55

Test

System

Random Generator

System

reinforce

update

reinforce

updateSystem

evolve

Test

Systemupdate

reinforcereinforce

Experimental Setup 56

System

ReinforcementLearning

SCoETest

EvolutionaryAlgorithm

RandomTest

Random Generator

System

ReinforcementLearning

Learning Rates 57

Experimental Setup 58

System

ReinforcementLearning

SCoETest

EvolutionaryAlgorithm

Evaluation

Random Generator

RandomTest

Random Generator

System

ReinforcementLearning

Evaluation

Random Generator

Effective Performance 59

Performance over Time 60

Successful Runs 61

Generated Test Cases 62

Scenario Co-Evolutionas a Tool for Software Engineering

Criticality Focus 64

scenario pool

Criticality Focus 65

scenario pool

all means of generatingnew scenarios arecontinuously beingexplored in order toachieve sufficient coverageof the scenario space

Criticality Focus 66

scenario pool

acquisition

requirements

all means of generatingnew scenarios arecontinuously beingexplored in order toachieve sufficient coverageof the scenario space

Criticality Focus 67

augmentation

scenario pool

acquisition

system performance: ??

requirements

all means of generatingnew scenarios arecontinuously beingexplored in order toachieve sufficient coverageof the scenario space

Criticality Focus 68

augmentation

scenario pool

break down

acquisition

system performance: ??

requirements

all means of generatingnew scenarios arecontinuously beingexplored in order toachieve sufficient coverageof the scenario space

Criticality Focus 69

augmentation

scenario pool

break down

evaluationacquisition

system performance: 42

system performance: ??

requirements

all means of generatingnew scenarios arecontinuously beingexplored in order toachieve sufficient coverageof the scenario space

Criticality Focus 70

augmentation

scenario pool

break down

evaluationacquisition

system performance: 42

system performance: ??

requirements

all means of generatingnew scenarios arecontinuously beingexplored in order toachieve sufficient coverageof the scenario space

Criticality Focus

The importance ofscenarios is weightedaccording to inverse

performance of system-under-test.

Adaptation Cooldown 71start of development

Adaptation Cooldown 72start of development

off-site adaptation borders

defining the desired limits of the adaptationprocess for all eventual customer scenarios

Adaptation Cooldown 73start of development

scenario evolution finds faulty behavior earlyand brings the adaptive system to behavewithin the adaptation borders

off-site adaptation borders

defining the desired limits of the adaptationprocess for all eventual customer scenarios

Adaptation Cooldown 74start of development

scenario evolution finds faulty behavior earlyand brings the adaptive system to behavewithin the adaptation borders

off-site adaptation borders

defining the desired limits of the adaptationprocess for all eventual customer scenarios

Adaptation Cooldown 75start of development

scenario evolution finds faulty behavior earlyand brings the adaptive system to behavewithin the adaptation borders

adapting to a single customer‘s specificenvironment changes the optimal behaviorslightly

off-site adaptation borders

defining the desired limits of the adaptationprocess for all eventual customer scenarios

Adaptation Cooldown 76start of development

running product

scenario evolution finds faulty behavior earlyand brings the adaptive system to behavewithin the adaptation borders

adapting to a single customer‘s specificenvironment changes the optimal behaviorslightly

off-site adaptation borders

defining the desired limits of the adaptationprocess for all eventual customer scenarios

on-site adaptation borders

during deployment narrower boundaries foradaptation with respect to one customer‘sspecific environment may be necessary

Adaptation Cooldown 77start of development

running product

Adaptation Cooldown

As systemdevelopmentprogresses,the space of

possible behavioravailable to the

adaptationmechanismdecreases.

scenario evolution finds faulty behavior earlyand brings the adaptive system to behavewithin the adaptation borders

adapting to a single customer‘s specificenvironment changes the optimal behaviorslightly

off-site adaptation borders

defining the desired limits of the adaptationprocess for all eventual customer scenarios

on-site adaptation borders

during deployment narrower boundaries foradaptation with respect to one customer‘sspecific environment may be necessary

Eternal Deployment 78

Eternal Deployment 79

Don’t deploy!

Eternal Deployment 80

Don’t deploy!

Deploy?

Eternal Deployment 81

Deploy??Don’t deploy!

Deploy?

Eternal Deployment 82

Deploy??Don’t deploy!

Deploy?

Eternal Deployment 83

Deploy??Don’t deploy!

Deploy?

system-under-tests immediatelyadapts to scenario evolution

Eternal Deployment 84

Deploy??Don’t deploy!

Deploy?

system-under-tests immediatelyadapts to scenario evolution

Eternal Deployment 85

Deploy??Don’t deploy!

Deploy?

(more) independent line of scenarioswith no immediate feedback for thesystem-under-test

system-under-tests immediatelyadapts to scenario evolution

Eternal Deployment 86

Deploy??

Deploy All

Trustworthiness is aidedby deploying all tests so they can be repeated at

the customer‘s site.

Don’t deploy!

Deploy?

(more) independent line of scenarioswith no immediate feedback for thesystem-under-test

system-under-tests immediatelyadapts to scenario evolution

Tools for a New Framework 87

Adaptation Cooldown

As systemdevelopmentprogresses,the space of

possible behavioravailable to the

adaptationmechanismdecreases.

Criticality Focus

The importance ofscenarios is weightedaccording to inverse

performance of system-under-test.

Deploy All

Trustworthiness is aidedby deploying all tests so they can be repeated at

the customer‘s site.

The Big Picture 88

Software Architecture

Software Gardening?

The Big Picture 89

Software Architecture

Software Gardening?

We need AI to adequately control AI.

Where to go from here? 90

Where to go from here? 91

System Test

adaptation

BesseresSystemeven

bettersystem

adaptation

bessererTesteven

bettertest

is testedby

is used fortraining

Where to go from here? 92

System Test

adaptation

BesseresSystemeven

bettersystem

adaptation

bessererTesteven

bettertest

is testedby

is used fortraining

“Penguin recombination” specifically designed for tests with sequential execution steps.

Explicit curriculum learning for “gentle introduction” of requirements according to specification into reward.

Generalization of this 1v1 setting to a multi-agent system with various (degrees of) adversaries.

Construction of whole test suites based on diversity and/or incremental complexity.

Qua

ntify

ing

and

dete

ctin

g un

certa

inty

in n

ew s

ituat

ions

AI and the Compute Method 93

1) “AI researchers have often tried to build knowledge into their agents,

2) this always helps in the short term, and is personally satisfying to the researcher, but

3) in the long run it plateaus and even inhibits further progress, and

4) breakthrough progress eventually arrives by an opposing approach based on scaling computation by search and learning.”

AI and the Compute Method 94

1) “AI researchers have often tried to build knowledge into their agents,

2) this always helps in the short term, and is personally satisfying to the researcher, but

3) in the long run it plateaus and even inhibits further progress, and

4) breakthrough progress eventually arrives by an opposing approach based on scaling computation by search and learning.”

“The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin.”

Computation Power used in AI 95

Computation Power used in AI 96

“Since 2012, the amount of compute used in the largest AI training runs has been increasing exponentially with a 3.5 month doubling time (by comparison, Moore’s Law had an 18 month doubling period).”

Options for the Future 97

AI experimentsbecome more

expensive

Progress in AI research slows

down

We find a way toincrease availablecomputing power

Options for the Future 98

AI experimentsbecome more

expensive

Progress in AI research slows

down

We find a way toincrease availablecomputing power

Options for the Future 99

AI experimentsbecome more

expensive

Progress in AI research slows

down

We find a way toincrease availablecomputing power

Quantum

Computing?

An Awful Lot of Expertise 100

QuantumPlatform

DomainAnalysis

AIAlgorithms

The PlanQK Platform 101

QAI conceptsspecialistscommunity

AnalysisStandardization

The PlanQK Platform 102

QAI concepts

QAI algorithms

specialistscommunity

AnalysisStandardization

Implementation developers

The PlanQK Platform 103

QAI concepts

QAI algorithms

QAI applications

specialistscommunity

AnalysisStandardization

Implementation developers

Search & Order users

The PlanQK Platform 104

QAI concepts

QAI algorithms

QAI applications

specialistscommunity

AnalysisStandardization

Implementation developers

Search & Order users

packagedsolution

So what else is new? 105

Quantum Computing

superposition entanglement

Superposition 106

particle

classical physics

Superposition 107

particle

spin

classical physics

Superposition 108

particle

spin

measure spin

classical physics

Superposition 109

particle

spin

measure spin

measured spin

classical physics

Superposition 110

particle

spin

measure spin

measured spin

classical physics quantum physics

Superposition 111

particle

spin

measure spin

measured spin

classical physics quantum physics

Superposition 112

particle

spin

measure spin

measured spin

classical physics quantum physics

Superposition 113

particle

spin

measure spin

measured spin

classical physics quantum physics

Superposition 114

particle

spin

measure spin

measured spin

classical physics quantum physics

changed particle

0 0 0 0 0 0 0 0

1 1 1 1 1 1 1 1

Registers 115

Registers 116

0 0 0 0 0 0 0 0

1 1 1 1 1 1 1 1

ASCII letter A

Superposition on Registers 117

050%

050%

050%

050%

050%

050%

050%

050%

50%1

50%1

50%1

50%1

50%1

50%1

50%1

50%1

all ASCII lettersat the same time

Superposition on Registers 118

050%

050%

050%

050%

050%

050%

050%

050%

50%1

50%1

50%1

50%1

50%1

50%1

50%1

50%1

measuring

all ASCII lettersat the same time

Superposition on Registers 119

050%

050%

050%

050%

050%

050%

050%

050%

50%1

50%1

50%1

50%1

50%1

50%1

50%1

50%1

measuring

0 0 0 0 0 0 0 0

1 1 1 1 1 1 1 1

ASCII letter E withprobability 1/256

all ASCII lettersat the same time

Superposition on Registers 120

measuring

0 0 0 0 0 0 0 0

1 1 1 1 1 1 1 1

ASCII letter & withprobability 1/256

050%

050%

050%

050%

050%

050%

050%

050%

50%1

50%1

50%1

50%1

50%1

50%1

50%1

50%1

all ASCII lettersat the same time

Superposition on Registers 121

090%

010%

090%

090%

090%

010%

090%

010%

10%1

90%1

10%1

10%1

10%1

90%1

10%1

90%1

measuring

0 0 0 0 0 0 0 0

1 1 1 1 1 1 1 1

ASCII letter E withprobability 0.43

all ASCII lettersat the same time (but probably E)

Entanglement 122

multiple quantum particles

measure single spin

measured spin

changed particles

Entanglement 123

multiple quantum particles

measure single spin

measured spin

changed particles

entangled

Entanglement 124

multiple quantum particles

measure single spin

measured spin

changed particles

entangled

Entanglement 125

multiple quantum particles

measure single spin

measured spin

changed particles

entangled

Entanglement 126

multiple quantum particles

measure single spin

measured spin

changed particles

entangled

Entanglement 127

multiple quantum particles

measure single spin

measured spin

changed particles

entangled

Entanglement on Registers 128

ASCII letters @ A B C at the same time

measuring

ASCII letter A withprobability 1/4

0 0 0 0 0 0 050%

050%

1 1 1 1 1 150%

150%

1

0 0 0 0 0 0 0 0

1 1 1 1 1 1 1 1

Entanglement on Registers 129

measuring

ASCII letter A withprobability 1/2

0 0 0 0 0 0 050%

050%

1 1 1 1 1 150%

150%

1

0 0 0 0 0 0 0 0

1 1 1 1 1 1 1 1

entangled

ASCII letters A B at the same time

Gate Model 130

050%

050%

050%

050%

050%

050%

050%

050%

50%1

50%1

50%1

50%1

50%1

50%1

50%1

50%1

CNOT

050%

050%

050%

050%

050%

050%

050%

050%

50%1

50%1

50%1

50%1

50%1

50%1

50%1

50%1

Gate Model 131

090%

010%

090%

090%

090%

010%

090%

010%

10%1

90%1

10%1

10%1

10%1

90%1

10%1

90%1

Gate Model 132

090%

010%

090%

090%

090%

010%

090%

010%

10%1

90%1

10%1

10%1

10%1

90%1

10%1

90%1

Gate Model 133

090%

010%

090%

090%

090%

010%

090%

010%

10%1

90%1

10%1

10%1

10%1

90%1

10%1

90%1

0 0 0 0 0 0 0 0

1 1 1 1 1 1 1 1

at the end measurequbits

Gate Model

▹direct pathway to universal quantum computer

▹similar architecture to classical computers

▹only protoypes in laboratories

▹currently < 100 qubits

134

Annealing 135

050%50%

1

050%50%

1

050%50%

1

050%50%

1

050%50%

1

050%50%

1

Annealing 136

050%50%

1

050%50%

1

050%50%

1

050%50%

1

050%50%

1

050%50%

1

cons

traint

Annealing 137

050%50%

1

050%50%

1

050%50%

1

050%50%

1

050%50%

1

050%50%

1

cons

traint

Annealing 138

050%50%

1

050%50%

1

050%50%

1

050%50%

1

050%50%

1

050%50%

1

cons

traint

Annealing 139

050%50%

1

050%50%

1

050%50%

1

050%50%

1

050%50%

1

050%50%

1

cons

traint

add constraintsinfinitely slowly

while addingno energy

adiabatic quantum computing

Annealing 140

050%50%

1

050%50%

1

050%50%

1

050%50%

1

050%50%

1

050%50%

1

cons

traint

add constraintsquite slowlywhile addingalmost no

energy

practical quantum annealing

Annealing 141

0

1

0

1

0

1

0

1

0

1

0

1

cons

traint

add constraintsquite slowlywhile addingalmost no

energy

at the end measurequbits solution to

constraints

Annealing

▹potentially equally powerful

▹architecture built for optimization

▹available commercially

▹currently > 2000 qubits

142

Bringing it all together…

Quantum Neural Blockchain AI 144

all these currently relevant buzzwords are…▹derived from irreversible computation▹based on probabilistic processes▹to some extent compatible…?

Main References

▹ Thomas Gabor, Andreas Sedlmeier, Marie Kiermeier, Thomy Phan, Marcel Henrich, Monika Picklmair, Berhard Kempter, Cornel Klein, Horst Sauer, Reiner Schmid, and J. Wieghardt. Scenario Co-Evolution for Reinforcement Learning on a Grid-World Smart Factory Domain. In The Genetic and Evolutionary Computation Conference (GECCO), 2019.

▹ Thomas Gabor, Marie Kiermeier, Andreas Sedlmeier, Bernhard Kempter, Cornel Klein, Horst Sauer, Reiner Schmid, and Jan Wieghardt. Adapting Quality Assurance to Adaptive Systems: The Scenario Coevolution Paradigm. In International Symposium on Leveraging Applications of Formal Methods (ISoLA), 2018.

▹ Matthias Homeister. Quantum Computing verstehen. Friedr. Vieweg & Sohn Verlag, 2008.

145

Image Sources

▹ https://www.bostonmagazine.com/news/2015/07/30/boston-2024-winners-losers

▹ https://en.wikipedia.org/wiki/The_Thinker#/media/File:Le_Penseur_in_the_Jardin_du_Musée_Rodin,_Paris_14_June_2015.jpg

▹ https://www.boredpanda.com/jumping-cats/

▹ https://kinder.wdr.de/tv/wissen-macht-ah/bibliothek/kuriosah/bibliothek-daumen-hoch-100.html

▹ https://www.medicalnewstoday.com/articles/320289.php#carry_message

▹ https://en.wikipedia.org/wiki/Mutation#/media/File:Darwin_Hybrid_Tulip_Mutation_2014-05-01.jpg

▹ https://www.heise.de/ct/artikel/Die-Woche-Microsoft-und-Linux-1283059.html

▹ https://phys.org/news/2019-10-guillotine-cruel-poisoning.html

▹ http://blog.stephenwolfram.com/2018/04/buzzword-convergence-making-sense-of-quantum-neural-blockchain-ai/

146