Challenges in Perception for Learning, Cognition and Control

Approaches

DariusBurschkaMachineVisionPerceptionGroup(MVP)TechnischeUniversitätMünchen,Germany

DariusBurschka–MachineVisionandPerceptionGroup(TUM)

Structure of a CNN for Robotics Applications

Rawsensorinformation(depthimagefromanRGB-Dsensor)istakenasinputtoconnectwiththerobotcontrolcommanddirectly

Idealcase:learnthecouplingbetweenperceptionandcontrolcommandsfromdemonstrations

DariusBurschka–MachineVisionandPerceptionGroup(TUM)

Curse of Complexity in Direct ProcessingIntheory,onecouldusealltheextractedfeatureswithaclassifiersuchasasoftmaxclassifier,butthiscanbecomputationallychallenging.Forinstanceimagesofsize96x96pixelswith400learnedfeaturesover8x8inputs.Eachconvolutionresultsinanoutputofsize(96−8+1)∗(96−8+1)=7921,andsincewehave400features,thisresultsinavectorof892∗400=3,168,400featuresperexample.Learningaclassifierwithinputshaving3+millionfeaturescanbeunwieldy,andcanalsobepronetoover-fitting.

DariusBurschka–MachineVisionandPerceptionGroup(TUM)

Deep-Learning for Perception in Robotics

Whatdoallthesetaskshaveincommon?

Source:L.Tai,“Deep-LearninginMobileRobotics…”

DariusBurschka–MachineVisionandPerceptionGroup(TUM)

Learning Approaches vs. Conventional Tools

Imagesource:Berkley

Grouping/Segmentation,Labeling,Identification

becausedifferencesinthepropertiesofpixelstoentireimageareconsideredduetothestructureofCNNs

Problem:notclear,whattheprocessingisbasedon!

?

Hand-designedfeatureshelptoestimatemetricdistancesbetweenthemusingcalibrationparameters

DariusBurschka–MachineVisionandPerceptionGroup(TUM)

Deep-Learning for Control in Robotics

Source:L.Tai,“Deep-LearninginMobileRobotics…”

DariusBurschka–MachineVisionandPerceptionGroup(TUM)

Can we avoid Metric Information in Representation of Environments?

DariusBurschka–MachineVisionandPerceptionGroup(TUM)

Fast Uncalibrated Monocular Estimation of Independent Motion Components

Representation of the Environment in Collision Space

DariusBurschka–MachineVisionandPerceptionGroup(TUM)

Examples of non-metric Control (RoboMobil)

DariusBurschka–MachineVisionandPerceptionGroup(TUM)

Examples of non-metric Control (RoboMobil)

DariusBurschka–MachineVisionandPerceptionGroup(TUM)

Conclusions

• Imagelabelingandimageretrieval(indexing)representalargeapplicationfieldforperceptionandimageprocessing(Google,Apple,etc.),butroboticrequiresoftenametricmappingfromthesensorontocontrolvalues• Workonalternativesforcouplingbetweenthesensorandtheactuatorshouldbeoneoftheperceptiongoals• Currentlyuseddatarepresentationsneedtobere-definedforlearningapplications.