The Robots are Coming - Engineers Australia...IEAus September 2009 Slide 1 The Robots are Coming ! Hugh Durrant-Whyte ARC Federation Fellow, Research Director ARC Centre of Excellence

Slide 1IEAus September 2009

The Robots are Coming !

Hugh Durrant-WhyteARC Federation Fellow, Research Director

ARC Centre of Excellence for Autonomous Systems (CAS)The University of Sydney, Australia

[email protected]


Robots in Australia


CAS/ACFR

• Research capabilities• Over 200 researchers, $15m/year of support• World-leading research in sensing, data fusion,

navigation, machine learning, cooperative control and human-robot interaction

• World’s best equipped laboratories and field test facilities for robotics research

• Fundamental and applied research, through to commercial outcomes

• Strong Government and Industry Support• ARC Centre of Excellence• Rio Tinto Centre for Mine Automation• BAE Systems Centre for Intelligent Mobile Systems• DSTO Centre for Autonomous Systems• US Army, Navy and Air Force projects

• Strong international engagement


Perception:SensorsRepresentationsData Fusion

Systems:ModellingDesignHRI

Control:ModellingPlanningCooperation

Learning:RecognitionDecision MakingMulti-Agent

ACFR SupportedUAV, UGV, AUV Demonstrators

Fire Fighting

Construction Defence

Mining

Cargo Handling

Health Care

Intelligent Buildings

Ecology Management

Entertainment

Agriculture

Search and Rescue Education

CAS Research and Development R

esea

rch

Dev

elop

men

tIn

dust

ry


Research: Perception

• Focus on unstructured outdoor environments

• Research in:• Mixed sensors, mm-radar• Rich Representations• Probabilistic Models• SLAM, DDF


Research: Control

• Focus on understanding platforms and cooperation

• Research in:• Platform models• Uncertainty management• Planning methods• Cooperative Control


Research: Learning

• Focus on perception and decision making

• Research in:• Scene understanding• Multi-agent systems• Human-robot interaction


Research: Systems

• Focus on modelling and design of robust systems

• Research in:• Formal methods• Middleware (ORCA)• Large-scale systems


Working with Industry

• Three Case studies• Cargo Handling: First success in field robotics• Defence: Pushing the boundaries• Mining: The current grand challenge

• Key Lessons• From research to industry/exploitation• How to achieve national benefits• Driving future research

• A future robotics industry in Australia


• The simplest possible field robotics problem:• A structured environment• Well defined task• Well defined pay-off

• Some key research challenges:• Control a large, fast platform• Guarantee performance• Ensure safety

Automated Container Handling


The FRAIT Project

• My first attempt at field robotics, 1990-1993.

• Technically advanced vehicle system:• Radar navigation• High-speed control• Safety systems

• Weak exploitation plan:• Small start-up company• Technically focussed• Shoe-string budget


Not thinking things through…


The Australian Waterfront

• Lang Corp. 1995 purchase of Australian National Terminals

• University involved in quay-crane and operation R&D

• Investment in new straddle operations in 1997

• A decision to look at the automation option

End-user understands drive to automate


Enhanced Straddle Carrier (ESC)

• Technical Development:• Lessons and new research

integrated into design• Work on navigation and

systems engineering• Development of team at

Sydney Uni. and at Patrick • Commercial Development:

• Total support from Directors• Long-term and consistent

investment by end-user• Flexible plan for development

and exploitation


Operational System

• In operation since 2004• Four automated berths

including 30 AutoStrad systems

• Operation located in Brisbane but run from Sydney

• A new start-up company producing and supporting automation systems

• Significant value for end-user: • 50% personnel reduction• 40% fuel reductions• 70% maintenance reduction• Improved planning and

availability

IEEE International Robotics Award 2007


How to Exploit ?

• Sell automation: $100k per AutoStrad• Sell AutoStrads: $1.2m each• Sell Automated terminals: $300m each• Operate terminals per box move:

$2bn/year

• Vertical integration in a large end-user can make good sense


Lessons Learnt (Part I)

Field robotics requires close cooperationbetween users, suppliers and researchers

Having a small part of something big is better than having a large part of nothing


Autonomous Systems in Defence:Pushing the Boundaries


Defence Robotics in Australia

• Australia has a small, professional defence force

• Minimal indigenous support for autonomy R&D

• A focus on algorithms• Network data fusion• GPS-denied navigation• Autonomy

• Primary research support• BAE Systems (UK, US)• AFOSR, ONR, ARL (US)• DSTO, ADF (Australia)

A vast country with few defence resources: Autonomy is key


BAE Systems Partnership

• A 20-year partnership with BAE• Decentralised Data Fusion (DDF)• Early work on SLAM• Example: OxNav-1991-94


BAE Systems: ANSER Program

• Rapid increase in interest in network-centric systems for land/air/sea

• Multi-million dollar R&D programme fully funded by BAE Systems from 1998-2002, 2003-2006, 2007-

• To develop and deploy a fully decentralised data fusion system on a network of UAVs

• To demonstrate, key network-centric algorithms: Modularity, Scalability, Flexibility and Survivability


ANSER Trials Campaign


Every UAV Project has One…


ANSER Outcomes

• Outputs:• World-first cooperative UAV demonstrations• Demonstration of fully autonomous network-

centric operations• BAE Systems Chairman Gold Award

• Outcomes:• BAE program wins in UK, US and Australia

(Type-45, JSF, MAST, etc)• Establishment of BAE Systems Autonomy

Research Centre at CAS and Global Capability at BAE Systems Australia

• Engagement in international programs: [UK]MOD, [US] ARL, AFRL, ONR

• Future autonomy for Australian civilian and defence applications


Outcomes in Australian Defence I

• Armoured Autonomous SAS Target-Training System

• Multi-platform autonomous localisation, mapping, control

• Indoor and outdoor long-term robust operation

• Group reactive responses• Script-like scenario generation


Sniper Training System


Outcomes in Agriculture and Marine


The Defence/Autonomy Dilemma

• True autonomy is a long way off (for ADF)• Limited to “platforms” and “remote control”• Not yet in current needs and doctrine• R-NIH: Buy existing US or UK equipment

• Defence autonomy does not push the boundaries as far as civilian applications• Not easy to field true autonomy outcomes• Not easy to compute NPV and make case• Information rather than platforms are the need

• The future of autonomy is in civilian applications


Lessons Learnt (Part II)

Industry will support long-range strategic research at world centres of excellence

In robotics, research needs time tosucceed for both technical and cultural reasons


Mining Automation:The Grand Challenge


Mining Automation

• Total mine automation:• A grand challenge for Field Robotics• Brings together all elements of systems,

integrity, perception and more

• Driven by safety, predictability, precision and efficiency

• Dirty, Dull and Dangerous…


The Initial Strategy (1995-00)

• Early work focused on equipment automation

• Benefits to the bottom line were hard to identify

• Lack of automation culture and how to exploit benefits of automation


The Revised Strategy

• Problems:• Mining companies make money,

not technology• Many large players create a

credibility gap• Steps toward automation:

• Develop increments toward the automated mine

• A portfolio of sensing and automation R&D projects

• Build confidence and understanding• Focus: Safety and predictability


Safety is a priority in mining !


Localisation and Mapping


Proximity Tracking

• Network-centric tracking, and collision data fusion:• Mining equipment• Light vehicles• People

• Integrated safety system• Beyond line-of-sight proximity

warning• Local operations control

• Operation in• Bracalba, Grasberg, Brockman


Mining Automation

• Total mine automation:• A grand challenge for Field Robotics• Integrates all elements of systems, integrity,

perception and more

• Key projects• Mine Automation System (MAS)-Architecture• Mine Picture Compilation (MPC)-Data Fusion• Equipment Automation–Trucks, drills, excavators, etc


Mine Automation System (MAS)

• MAS Provides• Standards for Information• Standards for Equipment• Machine-human integrity

monitoring• Client-server structure

• Mine-wide data fusion (MPC)

• Mine equipment coordination and control

• Integrates to Remote Operations Centre (ROC)


Mine-wide Data Fusion

• Build and maintain an integrated mine view• Geometric data • Geophysical data• Equipment disposition

• Real time fusion• All data types• All time scales• All spatial scales

• Both manned and unmanned Systems

• Remote Operations Centre (ROC)


Equipment Automation

• First automated pit• Trucks• Drills• Tracking systems

• In operation 7/2010


Rock Modelling and Recognition

• Characterise geometry/geology of active mining area• Drilling sensors

• Pull down/rotational pressures• Natural gamma, LIBS

• Face sensors• Laser/Radar and Camera• Hyperspectral


MPC-Block Model Gaussian Process


Learning Geology and Hardness

Non-stationary, Multi-task GPs

Drill data

Fusion into MPC/Block Model


Associating Geometry and Geology


Drill Rig in Operation


Lessons Learnt (Part III)

Researchers need to build deep and long-termcommitments to industry and industry outcomes

Industry will support large-scale fundamentalresearch if they can see the business bottom-line


The Future of Field Robotics in Australia

• What have we done in 10 years?• Over $40m in industry research funding• Estimated > $1bn in industry outcomes • including four new companies• Key research underpinning Field Robotics• A great deal of excitement and fun !

• Where are we going in the next 10 years?• 30 current and new industry projects in 2009 • Mining will remain the main challenge• Developing new applications: Precision agriculture,

marine resources, bush fire fighting, …, in Australia• Building an exciting science for field robotics • Continue going to far away exotic places…and

automating them ☺


The Next Ten Years


Documents

The Robots are Coming - Engineers Australia...IEAus September 2009 Slide 1 The Robots are Coming ! Hugh Durrant-Whyte ARC Federation Fellow, Research Director ARC Centre of Excellence