« Structure from motion » algorithms for the CYCAB robot Guillaume CERNIER ENSIMAG - 2nd year internship 19th June - 15th September 2006

« Structure from motion »« Structure from motion » algorithms for the algorithms for the

CYCAB robotCYCAB robot

Guillaume CERNIER

ENSIMAG - 2nd year internship

19th June - 15th September 2006

Tuesday, 16th January 07 Guillaume CERNIER « Structure from motion » algorithms for the CYCAB robot

INRIA presentationINRIA presentation Well-known french research centre in :

– Computer science

– Control theory

– Applied mathematics 6 research centres (Lorraine, Rennes, Rhône-Alpes, Rocquencourt,

Sophia-Antipolis, Futurs) Partner of the new research centre of Microsoft at Orsay INRIA Rhône-Alpes unit :

– 26 research teams among 8 associated with foreign laboratories

– Near 500 people (engineers, researches, PhD students, interns, …)

– Local partners : IMAG, UJF, INPG, CNRS, …

– Member of 10 european research networks

– 271 contracts among 53 with industry and 26 european

– 14 start-ups created since 1998 (more than 560 jobs created)


e-Motion team presentatione-Motion team presentation

“GEometry and Probability for MOtion and AcTION” Belongs to the GRAVIR lab

(INRIA, CNRS, UJF, INPG) Goals :

– Developing algorithmic models and methods allowing to build artificial systems

– With capacities of perception, decision, and action

– Operating in open and dynamic environments Start-ups : Itmi (1982), Getris Images (1985), Aleph Technologies

(1989), Aleph Med (1992), Probayes (2003). Industrial collaborations : Robotsoft, Renault, PSA, AW Europe

(linked to Toyota), XL-Studio, Aesculap, Teamlog, Kelkoo.


CYCAB presentationCYCAB presentation Large R&D program between

INRIA & INRETS Autonomous vehicle intended

for urban areas Automatic taxi

computer-guided Can be adapted for blind

and/or handicapped people Autonomous + urban areas => avoiding obstacles Before my internship : avoidance realized by a laser telemeter Internship goal : use a video camera to estimate the structure of the

whole scene in front of the vehicle Called “structure from motion”

QuickTime™ et undécompresseur codec YUV420

sont requis pour visionner cette image.


Internship goalsInternship goalsVideos, sequence of pictures 3D estimation of the structure

Constraints :

Software development : C/C++

Real-time processing

Integration to the whole platform embedded in the Cycab

(plug-in format)

Helps :

Use of an open-source 3D vision library : “OpenCV”


““Structure from motion” (1/3)Structure from motion” (1/3)

The problem : only one camera => unknown motion between 2 snapshots

So, structure from motion is a 2-step process :– Optical flow (vector field) computation between 2 pictures

– Thanks to this optical flow, computing :• First, the motion of the camera between the 2 pictures

• Finally, the 3D structure of the scene


Optical flow computation (2/3)Optical flow computation (2/3) SPARSE Vector field

– To lighten computation

– Vectors tracking real points

in the scene How choose the points to track ?

– GoodFeaturesToTrack algorithm

– Exists in OpenCV library Computation of the optical flow :

– Pyramidal algorithms

– Establishing correspondence

between points of the 2 pictures

– Eliminate “outliers”

– Let to know the “fundamental

matrix”


3D structure computation (3/3)3D structure computation (3/3) 2 steps :

– Find the motion between the 2 pictures :• Thanks to fundamental & intrinsic matrixes• Find the rotation and translation resulting

– Here, we know the position of the cameraat the 2 snapshots

– Computes the 3D point for each point pair

(q1,q2) in the 2 pictures :• Triangulation methods• Least-square methods (retained)• Depends of the projection matrixes P1 & P2

• Pi = KRi( I3x3 | -ti ) (4x3 matrixes)


Other developmentOther development

Camera Calibration GUI


Difficulties and successesDifficulties and successes 3D vision library OpenCV contains some bugs … Tutors absence …

Very interesting research field ! Reach the internship goals 2 weeks before the planned date Research article writing for a robotics conf (ICRA 07) Learning a GUI API (Gtk+)


Team lifeTeam life Different culture people (Italian, Brazilian, Asiatic, …) Summer at INRIA is …

Documents

« Structure from motion » algorithms for the CYCAB robot Guillaume CERNIER ENSIMAG - 2nd year internship 19th June - 15th September 2006