Model-Based Development and Validation of Multirobot

Doctoral course ’Advanced topics in Embedded Systems’. Lyngby' 10

Cooperative System

Jüri Vain

Dept. of Computer Science

Tallinn University of Technology

Syllabus� Monday morning: (9:00 – 13.30)

� 9:00 – 10:30 Intro: Model-Based Development and Validation of Multirobot Cooperative System (MCS)

� 10:30 – 12:00 MCS model construction and learning� 12:00 – 13:30 Model-based testing with reactive planning testers

� Tuesday morning: (9:00 – 12.30)� 9:00 – 10:30 Model checking Multirobot Cooperative Systems� 10:30 – 12:00 Hands-on: Distributed intruder capture protocol

Lecture #L2 : Model construction & learningMotivation

� Model construction is one of the bottlenecks in MBD� is time consuming� needs understanding of the system modelled� needs understanding why it is modelled� needs understanding how has to be modelled

� Choose the right modelling formalism that:� is intuitive� has right modelling power� has efficient decision procedures� has good tool support

� UPTA

Model construction techniques

� Model extraction from program code

� Text analysis (used in some UML tools)

� Pattern-based model construction

� Model learning

Terminology of machine learning

� Passive vs active

� Supervised vs unsupervised

� Reinforcement learning (reward guided)

� Computational structures used:

� FSA

� Hidden Markov Model

� Kohonen Map

� NN

� Timed Automata

� etc

Learning XTA (lecture plan)

� Problem context

� Assumptions:

� I/O observability

� Fully observable (output determinism)

� Generally non-deterministic models

� The learning algorithm

� Evaluating the quality of learning

Camera2

Marker

Camera1

Marker

Patient

Monitor for endoscope

Operatingsurgeon

Assistant

Endoscope assistant

Scrub nurse

Surgical instruments

Camera2

Camera1

Anesthetist

Problem context: SNR scene and motion analysis

insert

extract

return

prepare_istr

pick_instr

hold_wait

withrow

stretch

wait-return

receive

Scrub Nurse:

Surgeon:

Anesthetic machine & monitors for vital signs

Scrub Nurse Robot (SNR): Motion analysis

Camera2

Marker

Photos from CEO on HAM, Tokyo Denki University

3rd generation Scrub Nurse

Robot “Michael”

SNR Control Architecture

Reactive action planningReactive action planning

Targeting and motion control

Motion recognition

3D position tracking3D position trackingDirect manipulator

controlDirect manipulator

control

Predicted further motions

Data samplingsof hand position

Force & position feedback

Control parameters

Model(s)of Surgeon’s

motions

Reactive ctrl. layer

Deliberative ctrl. layer

Symbolic control loop

SNR’s “world” model

Surgeon’s behaviour model

Nurse’s behaviour model

Reference scenario

Instrumentationlayer

Recognizedmotion

SNR action to be taken

Collision avoidanceCollision avoidance

Current coordinates

Preprocessing and filtering

NN-based detection

Statistics based detection Decision making

( )( ) ( )

dxdydzepXTX

−−Σ−−

Σ= ∫∫∫1

( )( )( )( )( )

( )( )( )( )( )( )( )( )( )

−+−+−+−+−+−+−+−+−+

Motion recognition

Kohonen Map-based detection

Motion recognition using statistical

models

- working->extracting, - extracting->passing, - passing->waiting, -waiting->receiving, - receiving->inserting, - inserting->working

SNR Control Architecture

Reactive action planningReactive action planning

Targeting and motion controlTargeting and motion control

Motionrecognition

3D position tracking3D position trackingDirect manipulator

controlDirect manipulator

control

Predicted further motions

Data samplingsof hand position

Force & position feedback

Control parameters

Model(s)of Surgeon’s

motions

Reactive ctrl. layer

Deliberative ctrl. layer

Symbol level control loop

SNR’s “world” model

Surgeon’s behaviour model

Nurse’s behaviour model

Reference scenario

Instrumentationlayer

Recognizedmotion

SNR action to be taken

Collision avoidanceCollision avoidance

High-level behavior learning of SNR

||(Initial)

interaction

learning

Role models of

interacting actors

RTIOCO

monitoring

(with Uppaal

Model-based

behavior

planning of SNR

Learning

counter

examplesRTIOCO violation:

counter example

traces

Correctness

checking of

extension

Uppaal)

Extended

role model

Switching SNR

planning to

extended role

modelRevision of actor

ontology/correctness

criteria

OnlineOffline

positive

negative Ontology/ correctness

criteria revision request

Recovery request

Update request

(Timed) automata learning algorithm

� Input:� Definition of actors’ Observable inputs/outputs X

� Rescaling operator R: XObs

→ XXTA

� where XXTA

is a model state space

� Equivalence relation “~” defining the quotient state space X /~

� Time-stamped sequence of observed i/o events (timed trace TTr(Obs))

� Output:� Uppaal timed automaton A s.t.

TTr(A) ⊇RTIOCO R(TTr(Obs))/~

� Initialization

L ← {l0} % L – set of locations, l0 – (auxiliary) initial locationT ← ∅ % T – set of transitionsk,k’ ← 0,0 % k,k’–indexes distinguishing transitions between same

location pairsh ← l0 % h – history variable storing the id of the motion

currently processed in the TTr FIFO Eh’ ← l0 % h’ – variable storing the id of the motion before previoushcl ← 0 % hcl – clock reset historyl ← l0 % l – destination location of the current switching eventcl ← 0 % cl – local clock variable of the automaton being learnedg_cl ← ∅ % g_cl - 3D matrix of clock reset intervalsg_x ← ∅ % g_x - 4D matrix of state intervals that define state

switching conditions% E TTr FIFO, consisting of switching triples:

[target_action_ID, switching time, switching state]

Algorithm 1: model compilation (one learning session)

Match with existing eq.class

Encode new motion

Encode motion previously observed

Extend existing eq.class

Create a new eq.class

1: while E ≠ ∅ do2: e ← get(E ) % get the motion switching event record from buffer E3: h’← h, h ← l4: l ← e[1], cl ← (e[2] - hcl), X ← e[3]5: if l ∉ L then % if the motion has never occurred before 6: L ← L ∪ {l},7: T ← T ∪ {t(h,l,1)} % add transition to that motion8: g_cl(h,l,1) ← [cl, cl] % add clock reset point in time9: for all xi ∈ X do10: g_x(h,l,1,xi) ← [xi, xi] % add state switching point 11: end for12: else % if switching e in existing equivalence class13: if ∃k∈ [1,|t(h,l,.)|], ∀xi∈ X,: xi ∈ g_x(h,l,k,xi) ∧ cl ∈ g_cl(h,l,k) then 14: goto 3415: else % if switching e extends existing equival. class16: if ∃k∈ [1,|t(h,l,.)|], ∀xi∈ X: xi ∈ g_x(h,l,k,xi)↨Ri ∧ cl ∈ g_cl(h,l,k)↨Rcl

17: then 18: if cl < g_cl(h,l,k)- then g_cl(h,l,k) ← [cl, g_cl(h,l,k)+] end if19: if cl > g_cl(h,l,k)+ then g_cl(h,l,k) ← [g_cl(h,l,k)-, cl] end if20: for all xi ∈ X do21: if xi < g_x(h,l,k,xi)- then g_x(h,l,k,xi) ← [xi, g_x(h,l,k,xi)+] end if22: if xi > g_x(h,l,k,xi)+ then g_x(h,l,k,xi) ← [g_x(h,l,k,xi)-, xi] end if23: end for24: else % if switching e exceeds allowed limits of existing eqv. class25: k ←|t(h,l,.)| +126: T ← T ∪ {t(h,l,k)} % add new transition27: g_cl(h,l,k) ← [cl, cl] % add clock reset point in time28: for all xi ∈ X do29: g_x(h,l,k,xi) ← [xi, xi] % add state switching point 30: end for31: end if32: end if33: a(h’,h,k’) ← a(h’,h,k’) ∪ Xc % add assignment to previous transition34: end while

Create a new eq.class

HRI learning algorithm (3)

35: for all t(li,lj,k) ∈ T do % compile transition guards and updates

36: g(li,lj,k) ← ‘cl ∈ g_cl(li,lj,k) ∧ /\s ∈[1,|X|] xi ∈ g_x(li, lj, k, xs)’

37: a(li,lj,k) ← ‘Xc ← random(a(li,lj,k)), cl ← 0’ % assign random value in a

38: end for

39: for all li ∈ L do

40: inv(li) ← ‘/\k g(tki) /\ ¬ \/j g(tij)’ % compile location invariants

41: end for

Interval extension operator:[.,.]↨R : [x-, x+]↨R = [x- - δ, x+ + δ], where δ = R - (x+- x-)

Finalize TA syntax formatting

� Given

� Observation sequence E =

� System configuration

� Rescaling operator with region [0,30] for all xi

� Granularity of the quotient space X /~: 2

Learning example (1)

NurseSurgeon

Learning example (2)

Does XTA exhibit the same behavior as the

traces observed?

� Question 1: How to choose the equivalence relation “~” to define a feasible quotient space?

� Question 2: How to choose the equivalence relation to compare traces TTr(XTA) and TTr(Obs)?

TTr(XTA) = R (TTr(Obs)) /~ ?↓↓↓↓

TTr(XTA) ⊇RTIOCO R(TTr(Obs)) /~21

How to choose the equivalence relation

“~” do define a feasible quotient space?

� State space granularity parameter γi defines the maximum length of an interval [x-i,x

+i), of

equivalent (regarding ~) values of xi where x-i,x

∈ Xi for all Xi (i = [1,n]).

� Partitioning of dom Xi (i = [1,n]) to intervals (see line 16 of the algorithm) is implemented using interval expansion operator [.,.]↨R :

Interval extension operator:[.,.]↨R: [x-, x+]↨R = [x--δ, x++δ], where δ=R-(x+-x-)

On Question 2:

� Possible candidates:

� Equivalence of languages?

� Simulation relation?

� Conformace relation?

� ...?

Conclusions

� Proposed UPTA learning method makes the on-line synthesis of model-based planningcontrollers for HA robots feasible.

� Other practical aspects:

o learning must be incremental, i.e., knowledge about the previous observations can be re-used;

o UPTA models can be verified - functional correctness and performance can be verified on the model before used e.g., for planner synthesis;

o adjustable level of abstraction of the generated model to keep the analysis tractable.

Model-Based Development and Validation of Multirobot

Documents

qsar model validation

Evolutionary-Based Control Approaches for Multirobot Systemscubesat.arizona.edu/thanga08_bkc.pdf · Evolutionary-Based Control Approaches for Multirobot Systems 37 humidity, chemical,

day11 model selection validation - GitHub Pages · Model validation and selection Model Validation What approaches can we use to evaluate the performance of a model? What metrics

Evolutionary Online Learning in Multirobot Systems

Model-Based Development and Validation of Multirobot Cooperative System

Introduction Model selection Model validationjitkomut.eng.chula.ac.th/ee531/modelsel.pdf · Model Selection and Model Validation • Introduction • Model selection • Model validation

Partially Distributed Multirobot Control with Multiple Cameraswebdiis.unizar.es/~glopez/papers/ArandaACC2013.pdfPartially Distributed Multirobot Control with Multiple Cameras M. Aranda,

RR194 - Human factors assessment model validation · PDF fileHuman factors assessment model validation study ... Human factors assessment model validation study ... (F 1337 - 91 Re-approved

Validation of Simulation Model

Iron: Model Validation

Multirobot autonomous landmine detection using distributed

Model Validation and Risk Modeling - ACAMSfiles.acams.org/pdfs/2015/Model-Validation-and-Risk-Modeling-2015... · Model Validation and . Risk Modeling. ... oTo utilize in identifying

Evolutionary-Based Control Approaches for Multirobot Systemsasrl.utias.utoronto.ca/~tdb/bib/thanga_fer08.pdf · Evolutionary-Based Control Approaches for Multirobot Systems Jekanthan

Model Based Validation

Model Validation - Enterprise Architect · Model Validation - Model Validation 7 August, 2019 ·xx is a hexadecimal number corresponding to the position of the validation rule in

Model Validation for Community Bankers Validation for Community Bankers ... Review and Data Validation. The first part of a Model Validation is a Model ... Overview your core banking

PHEV Hymotion Prius Model Validation and Control Improvements - Papers/Validation/phev_hymotion_prius.pdf · PHEV Hymotion Prius Model Validation and Control Improvements Qiandong

Model calibration and validation

Model Selection and Validation

Model validation: flux sites