Robotic Hand-Eye Systems

CMPUT 610

Martin Jagersand

What is involved in making real vision-guided motion?

Hand-EyeSystem

System requirements

Solve many very different motion tasks– Flexible, teachable/re-programmable

Real time– On special embedded computers or general

workstations Different special HW Multiprocessors

Toolbox

System design

Interpreted “scripting” language gives flexibility Compiled language needed for speed and HW

interface.Examples Matlab

Greencard

Haskell

C, C++, fortran

PVM

C, C++

Dyn linking (mex)

PVMParallel Virtual Machine

Message passing based distributed systems Implemented for many machine architectures Runs separate (heavyweight) unix processes. Advantage: when can’t link together binaries:

– Linux libc5, libc6; IRIX lib32, libn32, lib64 High level process monitoring

Library structure

/usr/erskine2/prof/jag/matlabdirs

Ideas:

Data flow architecture natural. Higher order functions to go from general to

specific motion primitives. Toolbox of primitive motion behaviors.

Usage example:

Specialize robot– projandwait(zero3,’robotmovehill’,A3D,’WaitForHill’);

Initialize goals and trackers– [TrackCmd3D,N] = InitTrackers([1 1],[0,1]);– PU = GetGoals([1 1],[0,1]);

Servo control– J3s = LineMove(‘projandwait’,TrackCmd3D,J3i,PU,Ndi,err)

Fran/Frob motivation:Imperative Software Limits

Integration leads to recurring implementation chores– Writing loops to step forward discretely in time– Time slicing time-varying components that operate in parallel

Code reuse– Two pieces of code need to do almost the same thing, but

not quite What’s correct?

– The design doesn’t look at all like the code– Hard to tell if its a bug in the code, or a bug in the design

Programs should describe what to do not how to do it

New XVision Programming Model

StreamPartitioning

ImageFiltering

FeatureTracking

ImageFiltering

FeatureTracking

ImageFiltering

FeatureTracking Combination

Video In

Data Out

Programming Dynamical Systems

trackMouth v = bestSSD mouthIms (newsrcI v (sizeof mouthIms))trackLEye v = bestSSD leyeIms (newsrcI v (sizeof leyeIms))trackREye v = bestSSD reyeIms (newsrcI v (sizeof reyeIms))trackEyes v = composite2 (split, join) (trackLEye v) (trackREye v) where split = segToOrientedPts --- some geometry join = orientedPtsToSeg --- some more geometrytrackClown v = composite2 concat2 (trackEyes v) (trackMouth v)

Summary

Most current demos solve one specific movement

For solving many everyday tasks we need flexibility and reprogrammability– Interpreted scripting language– Higher order functions

What is SwEng anyway?

Bill