Smart Cars that See (and Act) Dariu M. Gavrila Environment Perception Research and Development
Behavior Design Meetup, 24-04-2014, Amsterdam
D. M. Gavrila 2
We originally thought Machine Intelligence would look like
1956 "Forbidden Planet" Robby the Robot (Flickr)
D. M. Gavrila 3
Then more recently, some suggested it would be more like
2004 iRobot 1983-2009 Terminator
D. M. Gavrila
when in fact, Machine Intelligence is already with us, and has a familiar embodiement …
The new Mercedes-Benz S Class (2013) 4
D. M. Gavrila
Sensors and Coverage (Mercedes-Benz E- and S-Class)
5
360°CAMERA 4 m range
D. M. Gavrila 6
Driver Assistance at Mercedes Benz (E- and S-Class, 2013)
Adaptive High Beam
Pre-Crash Braking (longitudinal & lateral traffic)
(Active) Lane Keeping
Nightview Attention Traffic Signs
Parking Adaptive Cruise Control with Steering Assist
(Active) Body Control S-Class only
D. M. Gavrila 7
Challenge: Active Pedestrian Safety
D. M. Gavrila 8
Worldwide Traffic Fatalities 2010 – Share of Vulnerable Road Users
Illustration: Bosch
D. M. Gavrila
Typical Scenario
9
D. M. Gavrila 10
Why is it difficult?
Large variation in pedestrian appearance (viewpoint, pose, clothes).
Dynamic and cluttered backgrounds.
Pedestrians can exhibit highly irregular motion.
Real-time processing required.
Stringent performance requirements (especially for emergency maneuvres).
D. M. Gavrila 11
Better Algorithms: System Architecture
Pedestrian Classification
Tracking / Fusion
Driver Warning / Vehicle Control
Path Prediction & Risk Assessment
Regions of Interest (Stereo, Motion, Geometry)
Gavrila and Munder, „Multi-Cue Pedestrian Detection and Tracking from a Moving Vehicle”. Int. J. Computer Vision, 2007 S. Munder, C. Schnörr and D.M. Gavrila. “Pedestrian Detection and Tracking Using a Mixture of View-Based Shape-Texture Models.” IEEE Transactions on Intelligent Transportation Systems, 2008
D. M. Gavrila 12
Dense Stereo: Better ROIs, Classification and Localization
C. Keller, M. Enzweiler, M. Rohrbach, D.-F. Llorca, C. Schnörr, and D.M. Gavrila. „The Benefits of Dense Stereo for Pedestrian Detection.“ IEEE Trans. on Intelligent Transportation Systems, 2011.
D. M. Gavrila 13
Lots of Real Data …
Ulm Stuttgart München Aachen Amsterdam Parma Brussels Tokyo Paris Barcelona San Francisco … O(107) images O(106) labels
D. M. Gavrila 14
EU WATCH-OVER (2008) 85%
50 km/h
10
Pedestrian Recognition Performance Last Decade (Downtown)
Correctly recognized pedestrians
Number of falsely recognized pedestrian trajectories per hour
100%
50%
10 1000 0 100 100
EU SAVE-U (2005) 65%
40 km/h
EU PROTECTOR (2003) 40%
600
30 km/h
N.B. # False alarms per hour << # Falsely recognized trajectories per hour
DE AKTIV-SFR (2010) 90%
D. M. Gavrila
Driver Assistance at Mercedes Benz (E- and S-Class, 2013)
Adaptive High Beam
Pre-Crash Braking (longitudinal & lateral traffic)
Nightview Attention Traffic Signs
Parking
with Pedestrian Recognition (Active) Body Control S-Class only
15
(Active) Lane Keeping
Adaptive Cruise Control with Steering Assist
D. M. Gavrila 16
D. M. Gavrila
Will the Pedestrian Cross? (Through the Eyes of the Sytem)
17
D. M. Gavrila
Will the Pedestrian Cross? (Through the Eyes of the Driver)
18
D. M. Gavrila
(Future) Context-based Pedestrian Path Prediction
19
Preliminary experiments suggest that emergency braking could be applied 0.5 s earlier, without increasing false alarms, reducing chances for hospital stay by 30% (TTC = 0.7s, vehicle speed = 50 km/h)
19
D. M. Gavrila 21
(Future) Automatic Evasion
C. Keller, T. Dang, A. Joos, C. Rabe, H. Fritz, and D.M. Gavrila. Active Pedestrian Safety by Automatic Braking and Evasive Steering, IEEE Trans. on Intelligent Transportation Systems, 2011
300 ms from first sight of pedestrian to initiation of vehicle maneuver (braking or evasion)
D. M. Gavrila
(Future) Autonomous Driving
22
Mercedes-Benz S500 Intelligent Drive Research Vehicle (Full videoclip available on YouTube)
D. M. Gavrila
Final Remarks
Dramatic progress on vision-based pedestrian sensing, mirroring the success of computer vision technology in driver assistance over the past decade.
Perseverance pays off.
Challenges still remain for pedestrian sensing. Additional focus: data fusion, situation understanding, new actuation concepts. Accident analysis at Mercedes-Benz indicates that the currently deployed pedestrian recognition technology could avoid 6 percent of pedestrian accidents and reduce the severity of a further 41 percent.
23
1999 2013
D. M. Gavrila 24
Better Algorithms + Market-Ready Hardware + More Data + Extensive Testing =
More Lives Saved.
D. M. Gavrila 25
Thank you
Credits
Markus Enzweiler, Christoph Keller, Stefan Munder, Jan Giebel and others …
Recommended