1 Ph. Bonnifait

Localization and perception for autonomous navigation using automotive sensors

Philippe Bonnifait

Professor at the Université de Technologie de Compiègne, France

Heudiasyc UMR 7253 CNRS

Autonomous Driving Technology Radisson Schwarzer Bock Wiesbaden, Germany

21st - 22nd May 2014

2 Ph. Bonnifait

Outline

1. Renault valet parking robot

2. Autonomous electrical vehicles

3. Navigation, perception and localization systems

4. Integrity issues

5. Conclusion

3 Ph. Bonnifait

Purpose An integrated valet service for the use

of EVs within the Technocentre Renault with autonomous driving technologies.

Objectives To provide a System Solution

which includes an ecosystem for the use of computer controlled EVs evolving in constrained spaces

To develop safe and reliable systems for autonomous vehicles, using automotive type components

To build technological modules that constitute the basis for intelligent mobility

Maintenance

Battery Charging

Automated

Valet Parking

Manual Autonomous

(Standby)

Power-off

Operational Mode

Automatic Switching

Manual Switching

The Renault valet parking project

4 Ph. Bonnifait

Vehicle parking area

Pick-up points

An integrated Autonomous Driving Service

5 Ph. Bonnifait

The valet parking robot

6 Ph. Bonnifait

Scientific and technological goals of the project

- Study and experiment a valet functionality for an electrical automotive vehicle using automotive type components

- Develop prototypes based on standard electrical vehicles

- Identify good information processing methods in terms of localization, perception, navigation, control, decision and integrity monitoring

- Gain experience on the expected performance given a sensor suite configuration

7 Ph. Bonnifait

Autonomous vehicles

8 Ph. Bonnifait

Autonomous system overview

Communications Module

Integrity Supervisor

Map

10 Ph. Bonnifait

Navigation

- Generates trajectories for Reaching the mission goal Avoiding obstacles and stay on the drivable space

11 Ph. Bonnifait

Architecture

Global Planning System

Local Planning System

Navigation System

Supervisor control

Goal

Perception system

Localisation System

Global Trajectory

Local Trajectory

Control system

Map Vehicle Model

12 Ph. Bonnifait

Perception sensors field of view

13 Ph. Bonnifait

Perception System Output

Body frame

Coherent detections are fused together

14 Ph. Bonnifait

Localization System

-Components CAN bus proprioceptive data GPS receiver Mobileye camera Map of the drivable area

-Output (10Hz) Position and heading Speed Confidence indicators

15 Ph. Bonnifait

Localization system « key ideas »

- No IMU Very different from existing systems The key idea is to rely on proprioceptive sensors already installed on board the vehicle

- It is automotive-integrated (no Velodyne used and no additional encoder installed on a wheel)

- Try to be as much as possible GPS standalone no additional GNSS service (e.g. DGPS, omnsitar)

16 Ph. Bonnifait

Localization system « key ideas »

- It exploits sensors used for other functionalities e.g. - The Mobileye camera initially installed for obstacle detection/recognition, - The ESP yaw rate sensor.

- Enhanced map with geo-referenced lane markings

-Thanks to the use of CAN-bus dead-reckoning, the system is able to handle short GPS outages and short sections without lane marking for instance at cross-roads. The availability for the navigation system is high.

17 Ph. Bonnifait

Frames

18 Ph. Bonnifait

GPS measurements

X

λ

Y

Z

φ

z

x

y

O

19 Ph. Bonnifait

Localization solver

20 Ph. Bonnifait

GPS measurements

21 Ph. Bonnifait

GPS errors

GPS positioning errors are not white and time-correlated • Pseudo-range measurements are affected by atmosphere biases, • Satellites positions used in real-time ephemeris are inaccurate, • Position fixes are computed by a Kalman filter that has its own dynamic.

22 Ph. Bonnifait

GPS errors modeling

- first order auto-regressive process

- random constant

- Shaping filter

23 Ph. Bonnifait

Evolution model

24 Ph. Bonnifait

Complete GPS observation model

25 Ph. Bonnifait

Camera measures

Parameter name Definition

LaneQuality Quality

C0 Position Parameter

C1

Heading Angle

26 Ph. Bonnifait

Observation model

27 Ph. Bonnifait

Map-Matching camera measurements

- Select the correct segment [AB] from the map

- Two main stages Lane marking type selection Metric and side selection

28 Ph. Bonnifait

Map-matching properties

29 Ph. Bonnifait

30 Ph. Bonnifait

ULS implementation - Sensor interfaces

NMEA GPS and Time Pulse (for synchronisation)

Vehicle can bus

- wheel speeds, yaw rate, steering wheel angle

- Mobileye lane markings

-Map OpenStreetMap and Spatialite format

- Algorithms GPS and CAN bus: C++

Map Management : C++

Map Matching: C++

Kalman filter real-time implementation: C++

31 Ph. Bonnifait

Experimental results

32 Ph. Bonnifait

Integrity issues

33 Ph. Bonnifait

Integrity monitoring and fail-safe operation

- When navigating autonomously, the valet robot has nobody aboard

- A supervision system is needed to deal with hazardous situations

- fail safe state = vehicle stopped

- Integrity monitoring • Infrastructure supervisor • On-board the vehicle

34 Ph. Bonnifait

Supervisor Box

Renault Vehicle Interface

Command and Mission Messages

HeartbeatMessages

V2V Comm Link

Operator Interface

Infrastructure supervisor

To ensure the external supervision of the autonomous vehicles and their interaction within the application environment

35 Ph. Bonnifait

Autonomous integrity monitoring

- Perception system Object/obstacle missed-detections are crucial Method: - Use different sensors technologies and redundant fields of view - Check the state of the tracking perception systems - Every sensor report is kept (even if it is a false alarm)

- Localization system Reliable position confidence domains are used to check that the estimated accuracy is compatible with the required accuracy for the current navigation task Method: - Exploit data redundancy - Apply internal fault detection tests (GPS multi-path) - Robustify the map-matching algorithm

36 Ph. Bonnifait

Conclusion

- Use of automotive type sensors for autonomous navigation - Robotic approach: the vehicle is controlled trough a localization feedback given a mission goal and can evolve in dynamic environments - An enhanced map and accurate sensors modeling is crucial - Many tests have been conducted by Renault on dedicated test tracks and different conditions - The performance of the perception/localization system is very encouraging by considering the automotive sensors that have been used More tests and validation are needed (accuracy and integrity)

- A “system engineering approach” is crucial for the design of such kind of system

37 Ph. Bonnifait

Associated publications

-Tao, Z. and Bonnifait, Ph. and Fremont, V. and Ibanez-Guzman, J. « Mapping and localization using GPS, lane markings and proprioceptive sensors » IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Tokyo, Japon, pp. 406-412, Nov., 2013

-Tao, Z. and Bonnifait, Ph. and Fremont, V. and Ibanez-Guzman, J. « Lane marking aided vehicle localization » 16th International IEEE Annual Conference on Intelligent Transportation Systems (ITSC 2013), La Hague, Pays-Bas, pp. 1509-1515, Oct., 2013

-Fouque, C. and Bonnifait, Ph. « Matching Raw GPS Measurements on a Navigable Map Without Computing a Global Position » IEEE Transactions on Intelligent Transportation Systems, vol. 13, num. 2, pp. 887-898, June, 2012