F.L. Lewis, Assoc. Director for Research Moncrief-O ... · Toyota Kanban Assembly Plant Job 1 Job 2 Job 3 Part Flow Worker 1 Worker 2 Worker 3 Two Simple Rules: 1. Do jobs in sequence

Automation & Robotics Research Institute (ARRI)The University of Texas at Arlington

F.L. Lewis, Assoc. Director for ResearchMoncrief-O’Donnell Endowed Chair

Head, Controls, Sensors, MEMS Group

http://ARRI.uta.edu/acs

Autonomous Systems

What is Autonomy?

Fancy Architectures?

ORGANIZATION LEVEL

COORDINATION LEVEL

EXECUTION LEVEL

ABSTRACTION

PRECISION

THREE-LAYER PLANNING AND CONTROL ARCHITECTURE

Saridis

NASREM- Jim Albus

Sense Model Execute

Hum

an In

terf

ace

Kno

wle

dge

Bas

e

Sense Model Execute

Sense Model Execute

Sensors and Actuators

SensorFusion

Model Update (Learning)

TaskDecomp.

Swarm Behavior – Cooperative?

Three agents to lift 100 lb heavy object.Each agent can lift only 40 lbs. ?

Solution- Two Simple Rules:1. Attract towards object2. Repel from each other

RESULT- even spacing around the object

Decentralized Emergent Intelligence

A little example -

Can easily handle adding or taking away agents

Potential Fields

Initial

Final

Combining Behaviors Through Potential Fields

S. S. Ge and Y. J. Cui, ``New Potential Functions for Mobile Robot Path Planning'', IEEE Transactions on Robotics Automation, Vol. 16, No. 5, pp.615 -620, October 2000.

Ron Arkin

Sam Ge

Toyota Kanban Assembly Plant

Job 1 Job 2 Job 3

Part Flow

Worker 1 Worker 2 Worker 3

Two Simple Rules:1. Do jobs in sequence on any part you have until it is taken from you2. When you are idle, take the part from the worker upstream

Kanban Pull System

Result- self-balancing flow line

Note- part flows forward but information flows backwards!

Traffic Platooning

Leaderd43

d42

d41

3 step lookahead

Lookahead gives smoother control

Solution- Two Simple Rules1. Attract towards your station2. Repel from each other

Leader

Define- Station – Position WRT Leader

Better Solution:

Before - rules had equal weight for goal.Now – rule 2 is weaker – only for safety.

Leader’s Rule1. Attract towards your target

Formation Flight

Leader

Station Keeping

Simple rules for each agent Overall “Intelligent” Behaviorof squadron

Where is the Intelligence?

Autonomous?

Decentralized Intelligence and ControlEMERGENT BEHAVIOR

Depends on Leader !!

Internal Organization Autonomy (proprio-) VS. External Autonomy (extero-)

Behavior-Based Control

The Example of the Mobile Robot

follow wall

go through door

seekgoal

avoid obstacle

Behaviors-

st.1

st.2

st.3

st.4go throughdoor

follow wallseekgoal

avoid obstacle

vehicleCtrl. 1

output 1input 1ref. traj. 1

vehicleCtrl. 2


vehicleCtrl. 3


vehicleCtrl. 4


door found obst. detected

obst. avoided

gone through door

Behavior-Based Control

use ctrl.1

Fini

te S

tate

Mac

hine

FB c

ontro

ller

HYBRID CONTROL

Rules

TableXY Stage mounted on

rotational stageResolution: 3 µm, 0.01 deg

MP-285XYZ degree of freedom

Motorized probeResolution: 40 nm

VerticalFocusing

Stage

Rotational stageResolution: 0.01 deg

CameraResolution: NTSC

Microscope RS-232

MOTIONCONTROLLERS

IMAGE AQUISITIONCARD

Analog / Digital Input- output CARD

LabView&

Windows CVI

MEMS Microassembly StationSemiautonomous Teleoperation

Vision Feedback

Map FromMEMS-Pro3D info

Visioninfo

LocalizationWith 3D info

Stored map

Feature-based

Based or pre-programmed behaviours

OpenGL LabWindows /CVI

RS-232

MOTIONCONTROLLERS

IMAGE AQUISITIONCARD

Analog / Digital Input- output CARD

LabView&

Windows CVI

VR control

Behavior-Based Hybrid Control needs good sensors, good controls

ACTUATORSENSORS/

SIGNAL PROC.

PLANT

performanceoutput

CONTROLLAW

RobustAdaptive

Fuzzy / NeuralPID

trackingerror

Tracking Loop

Inner Feedback Loops

select fromseveral outputs

select fromseveral Control Algorithms

select fromseveral inputs

select fromseveral referenceinputs

ReferenceInput

Needs pre-designed control algorithms forPrescribed Behaviors

Example:Wall following behaviourGoal Seeking behaviour

Inputsteeringsteering and speed

Measured outputdistance to walldistance to goal

Mission ControlMission includes segments:• Autonomous Takeoff• Waypoint Steering• Weapon Delivery• Weapon Evasion• Failure Accommodation• Upset Recovery• Autonomous Land

MODIFY

Aircraft ControlJim Buffington- Lockheed Martin, Fort Worth TX

Aircraft Controlincludes autopilots:• Takeoff Angle Hold• Course Hold• Altitude Hold• Wing Leveler• Turn Coordinator• Glide Slope Coupler

GuidanceSystem

Outer LoopControlSystem

Inner LoopControlSystem

ActuatorCommands

Flight PathFeedback

RateFeedback

AttitudeFeedback

PositionFeedback

Command Supervisory System

On-boardCommands

Off-boardCommands

Intelligent Guidance and Control System

On-Board Model

Health Management

Data

SensorData

Jim Buffington- Lockheed Martin, Fort Worth TX

SUPERVISION/ASSESSMENT

IDENTIFICATION

EVALUATION

ACTUATOR SENSORS/SIGNAL PROC.

PLANT

Trend Analysis

prognosis

performanceoutputCONTROL

LAW

RobustAdaptive

Fuzzy / NeuralPID

BLEND

probing

diagnostic output

Model Matching

diagnosis

Model-based Reasoning

Operator Interface

Task Decomposition

Traj./Path Planning

Exception Handling:

Sensor/Act. select

Control Alg. select/tune

On-Line Control Design

Knowledge Base

KB SYSTEM:

RLS, ELS, etc.

reference trajectory

HUMANOPERATOR

operator control input

exception signalcontrol reinforcement signal

Operator Interface

SharedControl

Rule-BasedFuzzyNeural Net

Neural or Fuzzy Classifier

Neural Net

select/tune

Agent-BasedModel-Based

Hierarchical ControlDecentralized ControlGame TheoryMaster/SlaveTask DecompositionPetri Net / DEDS SequencingAI Techniques

Multiple Agent Coordinator

Encapsulated Agent

Knowledge Base

trackingerror

EXECUTIVE

Tracking Loop

Inner Feedback Loops

A Control Engineer's View of Intelligent Control

Note:

All the architectures first shown capture these ideas extremely well!

Albus’ NASREMSaridis Three-Level

AUTONOMOUS SYSTEM

RECONFIGURATION

OPERATORDEPENDENT

DISCRETE EVENT, RULE-BASED

ADAPTIVE,LEARNED

OPTIMALITY

OPERATORDEPENDENT

PRE-PROGRAMMED

ON-LINE REVISION

INTELLIGENCE

AUTOMATION

DEPENDENT

INVOLUNTARY

INDEPENDENT

MANUAL

AUTOMATED

AUTONOMOUS

OPE

RA

TIO

NS

SYSTEM C

ON

TRO

LAutonomy is Enabled by Intelligence and Automation

Jim Buffington- Lockheed Martin, Fort Worth TX

WORKLOAD

CONTINUOUSINPUT

RELAXEDOVERSIGHT

ONLY HIGHESTLEVEL OBJECTIVES

Distributed decision-making based on often simple rules

Decentralized Intelligence leading to Emergent Behaviors

Autonomy is based on

Multiple sensors

Multiple good control algorithms

Event states related to prescribed pre-programmed behaviors

Wireless Distributed Sensor Networks

Management Center(Database large storage, analysis)

PDA

BSC(Base Station

Controller, Preprocessing)BTS

Wireless Sensor

Machine Monitoring

Medical Monitoring

Wireless SensorWirelessData Collection Networks

Wireless(Wi-Fi 802.11 2.4GHz

BlueToothCellular Network, -CDMA, GSM)

Data Acquisition Network

Data Distribution Network

Printer

Wireland

(Ethernet WLAN, Optical)

Animal Monitoring

Vehicle Monitoring

Onlinemonitoring

Wireless Sensor Networks

Servertransmitter

Any where to monitor and any where to access

Notebook Cellular Phone PC

Submarine Monitoring

The Problem of Complexity

Communication Protocols in a network must be restricted and organizedto avoid Complexity problems

e.g. in ManufacturingThe general job shop allows part flows between all machinesThe Flow Line allows part flows only along specific Paths

We have shown that the job shop is NP-complete

But the reentrant flow line is of polynomial complexity

Think of the military chain of command

Personnel Secure Area Monitoring

• Smart sensors and wireless MEMS sensor networks• Video surveillance and tracking• Biometrics and sensor fusion for personnel

monitoring• Databases and Data Mining• Pervasive and mobile computing (PICO)• Agent-oriented software engineering• Psychology and Human Performance Prediction • Fail-safe construction and smart materials

Contact Behrooz [email protected]

Contact Behrooz [email protected]

LOWSECURITY

MEDIUMSECURITY

HIGHSECURITY

VENTILATIONGas Sensors

. .Human Performance

Video Tracking/SurveillanceImage Processing

Data FusionData Mining

WALLS

Blast Layer

SensorLayers

....

Biometrics

Smart MaterialsSmart SensorsSmart Structures

Wireless NetworksPICO

Screening

Multiple-Clearance-Level Secure Access Areas

PHM Architecture and Enabling Technologies Air Vehicle On-BoardHealth Assessment

Health Management,Reporting & Recording

Autonomic Logistics& Off-Board PHM

PVI

MAINTAINERVEHICLE INTERFACE

Mission Critical

PHMData

Displays & ControlsCrashRecorder

MaintenanceInterface Functions

IETMsConsumables

On-Board Diagnostics

PMD

.

PMA

In-Flight &Maintenance Data Link

Flight Critical

PHM / Service Info

Database

AMD/PMD

PHM Area Managers

MS Subsystems

• Sensor Fusion• Model-Based Reasoning• Tailored Algorithms• Systems Specific

Logic / Rules• Feature Extraction

Provides:

• AV-Level Info Management• Intelligent FI• Prognostics/Trends• Auto. LogisticsEnabling/Interface

Methods Used:

FCS/UtilitySubsystems

NVMICAWSManager

AVPHMHosted in ICP

Structures

MissionSystems

• Decision Support• Troubleshooting and Repair• Condition-Based Maintenance• Efficient Logistics

Vehicle Systems

Propulsion

Results In:

ALIS• Automated Pilot / Maint. Debrief

• Off-Board Prognostics

• Intelligent Help Environment

• Store / Distribute PHM Information

Hosted in ICP

Mike Gandy, Lockheed

Autonomous Machinery Monitoring

IEEE Singapore SMC, R&A, and Control Chapters Short Course

Wireless Sensor Networks for Monitoring Machinery,Human Biofunctions, and Biochemical Agents

Date : Thursday, 13 November 2003Time : 9:00 am. - 5:00 pmVenue: National University of Singapore

Wireless Networks DSP and User Interface

WirelessData Collection Networks

Wireless Sensor

Machine Monitoring

Security Personnel and Vehicle Monitoring

C

O O

HH2O

h+

h+

h+

H2OC

O O

H

C

O O

H

C

O O

HC

O O

H h+

C

O O

H

C

O O

H

C

O O

H

C

O O

H

C

O O

H

C

O O

H

e-

e-

e-

e-

TiO2TiO2

Ni

C

O O

H

C

O O

HH2O

h+

h+h+

h+

H2OC

O O

H

C

O O

H

C

O O

H

C

O O

H

C

O O

H

C

O O

H

C

O O

HC

O O

H

CO O

H h+

C

O O

H

C

O O

H

C

O O

H

C

O O

H

C

O O

H

C

O O

H

C

O O

H

C

O O

H

C

O O

H

C

O O

H

C

O O

H

C

O O

H

e-

e-

e-

e-

TiO2TiO2

Ni

Biochemical Monitoring

EnvironmentalMonitoring

WirelessData Collection Networks

Wireless Sensor

Machine Monitoring

Security Personnel and Vehicle Monitoring

C

O O

HH2O

h+

h+

h+

H2OC

O O

H

C

O O

H

C

O O

HC

O O

H h+

C

O O

H

C

O O

H

C

O O

H

C

O O

H

C

O O

H

C

O O

H

e-

e-

e-

e-

TiO2TiO2

Ni

C

O O

H

C

O O

HH2O

h+

h+h+

h+

H2OC

O O

H

C

O O

H

C

O O

H

C

O O

H

C

O O

H

C

O O

H

C

O O

HC

O O

H

CO O

H h+

C

O O

H

C

O O

H

C

O O

H

C

O O

H

C

O O

H

C

O O

H

C

O O

H

C

O O

H

C

O O

H

C

O O

H

C

O O

H

C

O O

H

e-

e-

e-

e-

TiO2TiO2

Ni

Biochemical Monitoring

EnvironmentalMonitoring

N

S

N

S

Ferromagneticmaterial

Non- -ferromagnetic

material

Permanentmagnet

MEMSChip

Vibrating bodywith the coil

MEMS PowerGeneration

Documents

F.L. Lewis, Assoc. Director for Research Moncrief-O ... · Toyota Kanban Assembly Plant Job 1 Job 2 Job 3 Part Flow Worker 1 Worker 2 Worker 3 Two Simple Rules: 1. Do jobs in sequence