Smart Cars that See (and Act) Dariu M. Gavrila Environment Perception Research and Development

Behavior Design Meetup, 24-04-2014, Amsterdam

D. M. Gavrila 2

We originally thought Machine Intelligence would look like

1956 "Forbidden Planet" Robby the Robot (Flickr)

D. M. Gavrila 3

Then more recently, some suggested it would be more like

2004 iRobot 1983-2009 Terminator

D. M. Gavrila

when in fact, Machine Intelligence is already with us, and has a familiar embodiement …

The new Mercedes-Benz S Class (2013) 4

D. M. Gavrila

Sensors and Coverage (Mercedes-Benz E- and S-Class)

5

360°CAMERA 4 m range

D. M. Gavrila 6

Driver Assistance at Mercedes Benz (E- and S-Class, 2013)

Adaptive High Beam

Pre-Crash Braking (longitudinal & lateral traffic)

(Active) Lane Keeping

Nightview Attention Traffic Signs

Parking Adaptive Cruise Control with Steering Assist

(Active) Body Control S-Class only

D. M. Gavrila 7

Challenge: Active Pedestrian Safety

D. M. Gavrila 8

Worldwide Traffic Fatalities 2010 – Share of Vulnerable Road Users

Illustration: Bosch

D. M. Gavrila

Typical Scenario

9

D. M. Gavrila 10

Why is it difficult?

Large variation in pedestrian appearance (viewpoint, pose, clothes).

Dynamic and cluttered backgrounds.

Pedestrians can exhibit highly irregular motion.

Real-time processing required.

Stringent performance requirements (especially for emergency maneuvres).

D. M. Gavrila 11

Better Algorithms: System Architecture

Pedestrian Classification

Tracking / Fusion

Driver Warning / Vehicle Control

Path Prediction & Risk Assessment

Regions of Interest (Stereo, Motion, Geometry)

Gavrila and Munder, „Multi-Cue Pedestrian Detection and Tracking from a Moving Vehicle”. Int. J. Computer Vision, 2007 S. Munder, C. Schnörr and D.M. Gavrila. “Pedestrian Detection and Tracking Using a Mixture of View-Based Shape-Texture Models.” IEEE Transactions on Intelligent Transportation Systems, 2008

D. M. Gavrila 12

Dense Stereo: Better ROIs, Classification and Localization

C. Keller, M. Enzweiler, M. Rohrbach, D.-F. Llorca, C. Schnörr, and D.M. Gavrila. „The Benefits of Dense Stereo for Pedestrian Detection.“ IEEE Trans. on Intelligent Transportation Systems, 2011.

D. M. Gavrila 13

Lots of Real Data …

Ulm Stuttgart München Aachen Amsterdam Parma Brussels Tokyo Paris Barcelona San Francisco … O(107) images O(106) labels

D. M. Gavrila 14

EU WATCH-OVER (2008) 85%

50 km/h

10

Pedestrian Recognition Performance Last Decade (Downtown)

Correctly recognized pedestrians

Number of falsely recognized pedestrian trajectories per hour

100%

50%

10 1000 0 100 100

EU SAVE-U (2005) 65%

40 km/h

EU PROTECTOR (2003) 40%

600

30 km/h

N.B. # False alarms per hour << # Falsely recognized trajectories per hour

DE AKTIV-SFR (2010) 90%

D. M. Gavrila

Driver Assistance at Mercedes Benz (E- and S-Class, 2013)

Adaptive High Beam

Pre-Crash Braking (longitudinal & lateral traffic)

Nightview Attention Traffic Signs

Parking

with Pedestrian Recognition (Active) Body Control S-Class only

15

(Active) Lane Keeping

Adaptive Cruise Control with Steering Assist

D. M. Gavrila 16

D. M. Gavrila

Will the Pedestrian Cross? (Through the Eyes of the Sytem)

17

D. M. Gavrila

Will the Pedestrian Cross? (Through the Eyes of the Driver)

18

D. M. Gavrila

(Future) Context-based Pedestrian Path Prediction

19

Preliminary experiments suggest that emergency braking could be applied 0.5 s earlier, without increasing false alarms, reducing chances for hospital stay by 30% (TTC = 0.7s, vehicle speed = 50 km/h)

19

D. M. Gavrila 21

(Future) Automatic Evasion

C. Keller, T. Dang, A. Joos, C. Rabe, H. Fritz, and D.M. Gavrila. Active Pedestrian Safety by Automatic Braking and Evasive Steering, IEEE Trans. on Intelligent Transportation Systems, 2011

300 ms from first sight of pedestrian to initiation of vehicle maneuver (braking or evasion)

D. M. Gavrila

(Future) Autonomous Driving

22

Mercedes-Benz S500 Intelligent Drive Research Vehicle (Full videoclip available on YouTube)

D. M. Gavrila

Final Remarks

Dramatic progress on vision-based pedestrian sensing, mirroring the success of computer vision technology in driver assistance over the past decade.

Perseverance pays off.

Challenges still remain for pedestrian sensing. Additional focus: data fusion, situation understanding, new actuation concepts. Accident analysis at Mercedes-Benz indicates that the currently deployed pedestrian recognition technology could avoid 6 percent of pedestrian accidents and reduce the severity of a further 41 percent.

23

1999 2013

D. M. Gavrila 24

Better Algorithms + Market-Ready Hardware + More Data + Extensive Testing =

More Lives Saved.

D. M. Gavrila 25

Thank you

Credits

Markus Enzweiler, Christoph Keller, Stefan Munder, Jan Giebel and others …