Rescue Robot Project

2

Project overview

Modular components to be re-used in future years

Tasks and Team1. Robot graphical user interface (Zwivhuya Tshitovha)2. Robot control interface (Jaco Colyn)3. Robot visual system (Jonathan Dowling)

8 Masters students working in the Robotics and Agents lab

3

Importance of Research

Reduce risk for rescue workers Map generation Rescue tasks Detection and localisation of victims Detection of hazardous materials

Disasters that have used robots: Japan nuclear disaster Three Mile Island nuclear disaster World Trade Centre terrorist attack

4

Rescue robots in disasters

Usage in Japan nuclear disaster

And in the World Trade Centre

5

Rescue robots in disasters

World trade centre (continued)

6

The UCT Rescue Robot

7

Research Questions

How can an effective system be built to automatically detect simulated disaster zone objects? human victims hazmat signs rolling E’s

Using the normal IR cameras available on the robot and

Applying existing computer vision techniques to extract the features

8

Research Questions

How can a usable and efficient GUI be built by hiding information and preventing sensory overload?

9

Research Questions

How can an intuitive robot control system be created by mapping the robots functions onto a human interface device?

10

RoboCup Rescue Robot overview

RoboCup – an international organisation based in Sweden

Urban search and rescue tests

11

Tests and Competitions

Simulate human subjects and disaster environments

Robot vision tests Rolling E’s Hazmat signs Victim detection

Remote operation of robot through competition arena to carry out tasks such as retrieving an object

12

Rolling E’s

13

Hazardous Materials (Hazmat)

14

Victim detection

15

Deliverables

Client for software engineering project: Stephen Marais from Robotics and Agents Lab

The deliverables required for this project are: A visual system for the robot to be able to pass the

“rolling E’s” test Hazardous material signs detection Human victim detection Easy to use interface for controlling the robot via a

human interface device Intelligent graphical user interface that highlights

only important information to the operator

Robot Control InterfaceGraphical User Interface

Goals and challenges:

17

Fundamental Robot Control Goals

Driving the robot

Controlling the robotic arm

Adjusting the robots tracks

18

Prevent Operator Mistakes No direct line of

sight to the robot

Robot is “on its own” Has to return by itself

Prevent Falling over Driving out of range

19

Robot Control Challenges Efficient control

Single Teleoperator Reduce fatigue Interpret and perform commands in real time

Reduce input delay “lag” Efficient mapping of commands to the control

interface

Micire, M., Mccann, E., Desai, M., Tsui, K.M., Norton, A., and Yanco, H.A. Hand and Finger Registration for Multi-Touch Joysticks on Software-Based Operator Control Units. (2011), 88-93.

20

Graphical User Interface challenges

Designing a “clean” Graphical User Interface (GUI)

Reducing sensory overload – which leads to fatigue

Relay locations of important objects that cannot be directly seen Only seen through the control interface

and GUI Interface must provide situational

awareness

21

Robot Control Interface RC & GUI

Iterative and Incremental software engineering.

Human Computer Interaction (HCI) Human Robot Interaction (HRI)

Use case scenarios taken from Urban Rescue Scenarios

Robot Vision(RV)

Goals and challenges:

23

Fundamental RV goals

Detect Human body parts (primarily limbs)

Detect and identify Hazmat signs

Detect the “rolling-E”

24

Robot vision system

Input stream of video The general procedure

Transform the image’s data into a simpler representation

Apply a machine learning algorithm onto the transformed data to detect features.

Testing

26

Testing

Robot control interface and GUI User centred design approach. Qualitative and Quantitative results Feedback can be generated from ▪ Test subjects ▪ Asked to complete specific tasks using the interface.

Performance measurements and quantitative results from simulation testing. Hazmat signs Human body parts Rolling “E”

27

Testing

Requirements RoboCup Rescue Competition (RRC) Urban Search and Rescue

Initial testing on each sub-system

Final testing on the system as a whole Based on expected challenges in the

RRC

28

Ethical, Professional and Legal Issues

Demonstrate its capabilities on dummies

No external users because robot quite expensive

Ethical clearance

29

Related Work

Robot vision Mikolajczyk et al [1]

Robot control and Interface Richer et al [2] Adams [3]

30

Related Work cont...

Team Description Papers 2003 Team RoBrno [4]

31

Resources

Hardware 19 inch wide screen Robot Human interface device

Software OpenCV Bosch SDK ROS

32

Risks

Accessing the robot

Robot hardware damage

Missing project milestones

33

Anticipated Outcomes

System used for a rescue robot that is developed by the Robotics and Agents Research Laboratory.

Key success factors Robot vision -- Accurate detection in smallest

time Robot interface and control -- Usable and

efficient given by measure of time and effort

Timeline

35

Timeline

36

Timeline

37

Timeline

38

References

[1] Mikolajczyk, K,Schmid, C & Zisserman ,A 2004, „Human detection based on a probabilistic assembly of robust part detectors‟ . European Conference on Computer Vision, Oxford, United Kingdom.

[2] Adams, JA 2002,‟Critical Considerations for Human-Robot Interface Developmen‟, Proceedings of 2002 AAAI Fall Symposium, 2002 - aaai.org, Rochester Institute of Technology Rochester, New York.

[3] Richer, J & Drury, JL 2006 „A Video Game-Based Framework for Analyzing Human-Robot Interaction ‟, Characterizing Interface Design in Real-Time Interactive Multimedia Applications. MITRE Corporation, United states of America

[4] Uschin, K, Inteam, S, Sae-eaw,N, Trakultongchai, A, Klairith, P, Jitviriya, W & Jenkittiyon , P,‟ RoboCup Rescue 2009 - Robot League Team iRAP_Pro (Thailand)‟,King Mongkut‟s University of Technology North Bangkok (KMUTNB) , Bangkok Thailand.

39

Questions

?