B31XP Robotics Project 1

Autonomous Quadcopter Navigation

Group: Waka Waka

H. Kidane, I. Sadek, M. Elawady

Supervisor: Prof. Y. Petillot

B31XP Robotics Project 2

Agenda

• Overview• Related Work• Control Processing• Vision Processing• Future Work

B31XP Robotics Project 3

Agenda

• Overview• Related Work• Control Processing• Vision Processing• Future Work

B31XP Robotics Project 4

OverviewProblem Definition

Detecting and avoiding frontal obstacles using quad-rotor Ar.drone 2

B31XP Robotics Project 5

Overview

Indoor 420g

HD Camera 720P 30FPS

Outdoor 320g

Ultrasound Sensor

Very Light and High Resistance Plastic

Specific Propeller

What is Ar.Drone?

B31XP Robotics Project 6

Agenda

• Overview• Related Work• Control Processing• Vision Processing• Future Work

B31XP Robotics Project 7

Related Work I

Blob Based Obstacle Avoidance

Algorithm• Assuming the floor as only one pattern or one color.• The blob repressing the floor is the largest blob.• Each image pixel is classified as an obstacle or the ground

based on its color appearance.

LimitationFor AUV the floor is neither the largest blub nor mono-color .

B31XP Robotics Project 8

Related Work I

B31XP Robotics Project 9

Related Work II

Single Image Perspective Cue

Algorithm• Use the Canny edge detector and apply a probabilistic Hough

Transform to find long lines.• In a corridor environment, the long lines converge towards a

vanishing point at the end of the corridor.

LimitationDistinct and confined environments should be provided.

B31XP Robotics Project 10

Related Work II

B31XP Robotics Project 11

Related Work III

Stereo VisionLimitation

Require at least two cameras, and the robot has only one eye!

Monocular Stereo Vision

LimitationIn our task, robot is moving forward!

Algorithm

Assuming Static environment (two frames considered as two different cameras).

B31XP Robotics Project 12

Related Work IV

Optical FlowAlgorithm

• Optical flow is an algorithm to tell the direction and magnitude change of the intensity from successive images.• Points closer to the camera move more quickly across the image plane.

B31XP Robotics Project 13

Agenda

• Overview• Related Work• Control Processing• Vision Processing• Future Work

B31XP Robotics Project 14

Control ProcessingFlow Diagram

B31XP Robotics Project 15

Control Processing6 DOF

B31XP Robotics Project 16

Control ProcessingRos Driver

• "ardrone_autonomy" :ROS driver for Parrot AR-Drone quadrocopter

• AR-Drone update frequency at 200Hz

• Driver operates in two modes:• real-time : publish information instantly• fixed rate: publish data at a fixed rate

B31XP Robotics Project 17

Control ProcessingSending Control Command

• Publishing an Empty ROS messages to the ff topics to takeoff/land and rest– ardrone/takeoff– ardrone/land– ardrone/reset

B31XP Robotics Project 18

Control ProcessingSending Control Command

• Publish a message of type geometry_msgs::Twist to velocity command topic

B31XP Robotics Project 19

Control ProcessingObstacle Avoidance Control

Algorithm

1. Fly in the x-direction straight.

2. Fly sideways when the obstacle is found in the field of view.

B31XP Robotics Project 20

Agenda

• Overview• Related Work• Control Processing• Vision Processing• Future Work

B31XP Robotics Project 21

Vision ProcessingFlowchart

Extract Features

CurrentImage

FilterFeatures

Find Obstacles

Select NearestObstacle

Estimate Position & Orientation of

Selected Obstacle

Publish Flight

Command

PreviousImage

B31XP Robotics Project 22

Vision ProcessingResults - Input

After K frames

Frame X

Frame X+K

B31XP Robotics Project 23

Vision ProcessingResults – SURF I

After K frames

Frame X

Frame X+K

Tomoyuki Mori and Sebastian Scherer, "First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles," International Conference on Robotics and Automation, May, 2013

B31XP Robotics Project 24

Vision ProcessingResults – SURF II

Matching Result

Filtered Matching Result

Tomoyuki Mori and Sebastian Scherer, "First Results in Detecting and Avoiding Frontal Obstacles from a Monocular Camera for Micro Unmanned Aerial Vehicles," International Conference on Robotics and Automation, May, 2013

B31XP Robotics Project 25

Vision ProcessingResults – Optical Flow (Sparse)

Features of 1st Frame

Matching Result

B31XP Robotics Project 26

Vision ProcessingResults – Optical Flow (Dense)

Features of 1st Frame

Matching Result

Qiyuan Qiu, "Reconnaissance Drone," ECE and ME Spring Senior Design Show and ECE Senior Honors Show, 2013

B31XP Robotics Project 27

Vision ProcessingProblems

• Computation time of feature matching -> GPU processing (CUDA) in ROS!!

• Lack of important features -> Color-based segmentation for tree trunks

• Background Noise -> Outdoor pre-processing step (denoising, contrast correction)

• No advantage of using color images -> Selection of suitable color space

B31XP Robotics Project 28

Agenda

• Overview• Related Work• Control Processing• Vision Processing• Future Work

B31XP Robotics Project 29

Future Work

B31XP Robotics Project 30