Mission Planning Framework for Manned-Unmanned Teams · Mission Planning Framework for Manned-Unmanned Teams Safe and Secure Systems and Software Symposium (S5) July 12-14, 2016

Imagination at work

Mission Planning Framework for Manned-Unmanned Teams Safe and Secure Systems and Software Symposium (S5) July 12-14, 2016

Dayton, Ohio

Overview

Motivation

Technical challenges

Past attempts

Proposed approach

Conclusions

S5 — Dayton, OH | July 12-14 2016 2

--

GE’s Military Customers Asking for Mission Systems with Support for MUM-T

Vision: Develop and deliver integrated capabilities that enable the war fighter to safely and effectively conduct missions in increasingly complex environments

•  Manned Unmanned Teaming and Mission Autonomy—enablers for:

–  Better workload management to maximize human performance

–  Increased survivability

–  Increased lethality

•  High Assurance

3 S5 — Dayton, OH | July 12-14 2016

--

GE’s Internal and External Industrial Customers

Increased used of autonomy and fleets of robots to optimize asset health management

•  Manned Unmanned Teaming for Inspection, Maintenance & Repair:

–  Better workload management to maximize human performance

–  Reduced downtime

–  Efficient inspection and better data quality

•  High Assurance

4 S5 — Dayton, OH | July 12-14 2016

--

5

White logo. Do not move. Send logo to front. Send photo to back.

S5 — Dayton, OH | July 12-14 2016

Canonical Scenario

Technical Challenges

Multiple agents: communication, coordination, cooperation •  Cooperation usually viewed as an emergent property

Mission planning: combination of task planning and motion planning •  Merging of task and motion planning in a multi-agent scenario

Interaction between manned and unmanned •  Situational awareness

•  Common representations

Assurance (safety, security, certification) •  Safety, security guarantees, certification

S5 — Dayton, OH | July 12-14 2016 6

Sample of Previous Attempts

Early attempts •  D. C. Mackenzie, R. C. Arkin, and J. M. Cameron. Multi-agent Mission

Specification and Execution. In Robot Colonies, pages 29–52. Springer US, Boston, MA, 1997.

•  C. Miller, M. Pelican, and R. Goldman. ‘tasking’interfaces for flexible interaction with automation: Keeping the operator in control. In Proceedings of the Conference on Human Interaction with Complex Systems, pages 123–128, 2000.

•  R. P. Goldman. MACBeth: A Multi-Agent Constraint-Based Planner Robert P. Goldman, Karen Zita Haigh, David J. Musliner, Michael Pelican. Working Notes of the AAAI Workshop on Constraints and AI Planning, 2000.

•  B. Browning, J. Bruce, M. Bowling, and M. Veloso. STP: Skills, tactics, and plays for multi-robot control in adversarial environments. Proceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Engineering, 219(1):33–52, 2005.

S5 — Dayton, OH | July 12-14 2016 7

--

Small Sample of Previous Attempts Using Formal Methods Techniques

More recent work •  C. Finucane, G. Jing, and H. Kress-Gazit. LTLMoP: Experimenting with

language, Temporal Logic and robot control. In Intelligent Robots and Systems (IROS), 2010 IEEE/RSJ International Conference on, pages 1988–1993, Oct 2010.

•  Neil T. Dantam, Zachary K. Kingston, Swarat Chaudhuri, and Lydia E. Kavraki. Incremental Task and Motion Planning: A Constraint-Based Approach. Proceedings of the Robotics Science and Systems XII Conference, 2016

•  L. Humphrey. Model checking for verification in UAV cooperative control applications. In F. Fahroo, L. Y. Wang, and G. Yin, editors, Recent Advances in Research on Unmanned Aerial Vehicles, volume 444 of Lecture Notes in Control and Information Sciences, pages 69–117. Springer Berlin Heidelberg, 2013.

S5 — Dayton, OH | July 12-14 2016 8

--

Our approach

Hallmarks

Framework •  Collection of concepts and tools to solve for mission planning problems in the

context of manned-unmanned teams

•  Two variants of the problem: –  Offline: before mission execution

–  Online: during mission execution

Compositional approach Use of formal verification technology Use of synthesis technology Generation of specifications using GE’s proprietary formalization technology S5 — Dayton, OH | July 12-14 2016 10

--

Overview

11 S5 — Dayton, OH | July 12-14 2016

--

Mission CommanderUser Interface

Front-end

RequirementsFormalization

• Mission Requirements

• Key Performance Metrics and Objectives

Naturalistic Interaction

• Role Allocation• Task Decomposition• Contract Generation

Supervisory Controller Synthesis

F



Specs / Contracts Asset i

User Interface Back-end


Specs / Contracts Sub-team j

Specs / Contracts Asset j1

Specs / Contracts Asset n



Supervisory Controller i

Supervisory Controller j1


Supervisory Controller n

Requirements Translation

Asset i

Low Level Control i

+

-

Low Level Control j1

+

-


+

-

Low Level Control n

+

-

Deliberative Layer

Reactive Layer

Aut

onom

y / A

utom

atio

n

World Model

Available Assets Capabilities

Playbook

Model Library

SpecsContracts

Mission Planning

Role Allocation • Task Decomposition • Contract Generation

Asset j1 Asset j2 Asset n

Com

posi

tiona

l Ver

ifica

tion

Synt

hesi

s / R

efine

men

t

World Model

Playbook

Model Library


Com

posi

tiona

l Ver

ifica

tion

Synt

hesi

s / R

efine

men

t

Supervisory Controller

Requirements Formalization

S5 — Dayton, OH | July 12-14 2016 12

--


Front-end





F



Role Allocation – Task Decomposition – Contract Generation

S5 — Dayton, OH | July 12-14 2016 13

--


Front-end







F












World Model

Available Assets Capabilities

Playbook

Model Library

SpecsContracts

Mission Planning

Role Allocation • Task Decomposition • Contract Generation

Com

posi

tiona

l Ver

ifica

tion


S5 — Dayton, OH | July 12-14 2016 14

-- • Role Allocation• Task Decomposition• Contract Generation











Supervisory Controller i



Supervisory Controller n

Asset i

Low Level Control i

+

-


+

-


+

-

Low Level Control n

+

-

Deliberative Layer

Reactive Layer

Aut

onom

y / A

utom

atio

nMission Planning

Asset j1 Asset j2 Asset n

World Model

Playbook

Model Library


Com

posi

tiona

l Ver

ifica

tion

Synt

hesi

s / R

efine

men

t

Supervisory Controller

Flight Plan Generation

S5 — Dayton, OH | July 12-14 2016 15

--

Performance Models

UAV 1

UAV 2

UAV 3

Temporal Spatial Contracts Contingencies

Specifications

Synthesis

Adversary's Behavior Models

Mission Plan Templates

("Playbook")

Plan UAV1 Plan

UAV2

Plan UAV3

UAV 1 UAV 2

UAV 3

Run-time Mission Assurance for Collaborative Manned/Unmanned Teams

Key elements: •  Run time verification at individual asset level

–  Asynchronous, based on event detection, or periodic

•  Hierarchical compositional verification

–  Composition of verification results from team members at lower levels of the hierarchy

•  Communications protocol

–  Contracts flow down to same or lower levels of hierarchy

–  Verification results from lower to higher levels of hierarchy

Ideal for introduction of third-party components

S5 — Dayton, OH | July 12-14 2016 16

--

The Path Towards High Assurance Autonomous Systems

S5 — Dayton, OH | July 12-14 2016 17

--

Formal synthesis

Verification

Test and Evaluation

Run-time Assurance

QUESTIONS ?

Conclusions

Key takeaways •  We presented a framework with a decomposition into sub-problems and technical

challenges

•  High assurance as a key theme in deployment of robots / autonomous systems

•  GE working on V&V of autonomous systems

Numerous open challenges •  Unexpected / un-modeled aspects

•  Models of human behavior

•  More reliance on “black box” technology to enable autonomy (perception), e.g.: machine learning

•  Intractable/undecidable problems

S5 — Dayton, OH | July 12-14 2016 18

--

Documents

Mission Planning Framework for Manned-Unmanned Teams · Mission Planning Framework for Manned-Unmanned Teams Safe and Secure Systems and Software Symposium (S5) July 12-14, 2016