Students Aaron Roney, Albert Soto, Brian Kuehner, David Taylor, Mark Hibbeler,

Nicholas Logan, Stephanie Herd

Tele-Operation and Vision System for Moon Exploration

NASA JSC Mentors Dr. Bob Savely

Mike Goza

Project Mentor Dr. Giovanni Giardini

Project AdvisorProf. Tamás Kalmár-Nagy

Project Members

Nuclear Engineering

Mechanical Engineering

Aerospace Engineering

Aerospace Engineering

Aerospace Engineering

Mechanical Engineering

Computer Engineering

Aaron Roney

Albert Soto

Brian Kuehner

David Taylor

Mark Hibbeler

Nicholas Logan

Stephanie Herd

Sophomore

Sophomore

Senior

Senior

Sophomore

Freshman

Freshman

Motivations

Lunar surface exploration

Human perspectiveIn safetyWith low risk

3D environment reconstruction

Self location with artificial vision system

Objectives

Vision SystemEgo-Motion estimationEnvironment reconstruction

Tele-Operation System with Visual Feedback

Tele-Operation SystemRemote Vehicle Control

Hardware and Mechanical Implementation

Visual System (onboard the Vehicle)

Ground Station

Vehicle Hardware

WIRELESS

NETWORK

WIRELESS

NETWORK

Visual System (onboard the Vehicle)

Ground Station

Vehicle Hardware

WIRELESS

NETWORK

WIRELESS

NETWORK

Theory

Left image Right image

uleftp uright

p

vleftp vright

p

uleftp

It is impossible to compute the 3D coordinates of an object with a single image

Solution: Stereo Cameras

Disparity computation

3D reconstruction

Image

Main Goal: digital environment 3D reconstruction Object detection (i.e. obstacles) High level planning Self localization

Use Stereo Cameras to generate

3D environment

Environment Reconstruction

Disparity map computation: Given 2 images, it is a collection of pixel disparities Point distances can be calculated from disparities

Environment can be reconstructed from disparity map

Left Image Right Image Disparity Map

Environment Reconstruction

Perspective Projection Equation

Main goal: evaluate the motion (translation and rotation) of the vehicle from sequences of images

Ego-Motion Estimation

Solving will give velocities of the vehicle

Optical Flow Example Optical Flow is related to vehicle movement through the

Least Square solution

Reference Motion [mm] Detected Motion [mm]

Tx0 4.5

Ty0 -0.9

Tz50 45.3

Ωx0 -0.1

Ωy0 -0.2

Ωz0 0

Ego-Motion: ExampleOptical Flow Left Image Optical Flow Right Image

Visual System (onboard the Vehicle)

Ground Station

Vehicle Hardware

WIRELESS

NETWORK

WIRELESS

NETWORK

Calibration and Filtering

Calibration: removes image distortion

Filtering Process:Improves image qualityIncreases the robustness

of the vision system

Visual System (onboard the Vehicle)

Ground Station

Vehicle Hardware

WIRELESS

NETWORK

WIRELESS

NETWORK

Tele-OperationsLaptop on TAMUBOT

TAMUBOT Control System Wireless Router Control PC

Tropos Router Picture PC

VehicleVehicle Courtesy of Prof. Dezhen Song

Baseline

D

L

FOV1 FOV2

α

Horizontal View

Camera support system3-DOF mechanical neck:

Panoramic rotationTilt rotationTelescopic capability

Controlled height and baseline length

Conclusions andFuture Work

Demonstrated:Ego-motion estimationEnvironment ReconstructionVehicle control and movement

Future Developments:System integrationFiltering and improving results

Thanks to:– Prof. Tamás Kalmár-Nagy– Dr. Giovanni Giardini– Prof. Dezhen Song– Change Young Kim– Magda Lagoudas– Tarek Elgohary– Pedro Davalos

Acknowledgements