Driver-Vehicle-Environmentmonitoring modules

Katia Pagle, ICCS

www.aide-eu.org

AIDE Driver – Vehicle – Environment module: the problem that addresses

The objective is to develop techniques for real-time monitoring of the driver-vehicle-environment state, in order to enable real-time adaptation of the driver – vehicle interface.

www.aide-eu.org

DVE detected conditions

A stored user profile combined with a user’s driving record in terms of user’s preferences and driving behaviour.

Driver profile

The total level of risk concerning the environmental and the traffic conditions (environment type, traffic density, environmental conditions etc.).

Traffic and environmental risk

The driver’s intention, e.g. for a lane change.Driver intent

The physical ability of the driver to drive (fatigue, sleepiness, etc.).

Driver impairment

Cognitive load or shift of visual attention away from the road ahead, induced by an external event or a secondary task.

Driver Distraction

Driver’s “level of availability” to receive and process information, according to the requirements of the current driving task (not available high driving demand).

Driving demand

Explanation DVE parameter

www.aide-eu.org

DVE component

PM

Profile data

Sensor array

DVEMDAE moduleDriver’s availability assessment with respect to the

primary driving taskCAA module

Driver’s Cockpit activity assessment

+detection of driver

intention

DSD moduleDriver’s

physiological and behavioural diagnostic

TERA moduletraffic and

environment risk assessment functions

+detection of driver

intention

DC moduleDriver’s

characteristics and preferences parameters

Functional architecture of DVE modules

www.aide-eu.org

Sensor definition and adaptation

Sensor array

Radar Laser Scannerobject listBUILT-IN

InertialVehicle stateBUILT-INCAN bus

Frontal CameraLaneBUILT-IN

GPS, gyro and digital mapsSynergies with MAPS&ADAS PReVENT

Camera based Eye tracking

AIDE Maps interface

Eye gaze and head positionstereo vision

www.aide-eu.org

Eyelid sensor

The Eyelid sensor is a vision based system which processes the video flow of the driver's face to provide the eye opening and blink behaviour signals.Physically the sensor is composed of a camera including a set of pulsed near infrared (NIR) LEDs placed in the instrument panel linked to a laptop which processes the image flow provided by the camera.

www.aide-eu.org

Map data extraction

MAPS Attributes

Segment Crossing Traffic Sign Speed limits Curvature

Data base

ADASRP S/W

AIDE interface

DVE Module

DVE Module

…

www.aide-eu.org

Traffic Environment Risk Assessment (TERA) module

TERA role:To calculate in real time a total level of risk related to traffic and environmental parameters.To calculate environmental and traffic parameters according to the requirements of the other DVE modules.To estimate the drivers’ intention (e.g. for maneuver of a possible lane change).

Environmentlightweather

Infrastructuretypedistance

For example, existing sensors used by collision-warning systems and Global Positioning System (GPS) measurements of localization combined with a table of corresponding roadway characteristics could be used to help understand the environment outside the vehicle and adapt the HMI accordingly.

www.aide-eu.org

Frontal Radar

Inertial vehicle sensors

Frontal Camera

Map and positioning

data

interfaces

Scenariorepresentation

(data collection)

RiskAssessment(Decision)

CAA driver intention

TERA

TERA algorithms

TERA outputto

ICA and applications

TERA outputto

other DVEmodules

Risk functions

Driver intention (Internal

parameters)

www.aide-eu.org

Driver State Degradation (DSD) module

The DSD is based on the fusion of information provided by physiological and mechanical sensors.

Several diagnostics are calculated: physiological diagnostic, and behavioural diagnostics; then fused with additional in-vehicle information.

Goal: Assess the evolution of the driver state degradation(physical ability).

Behavioral Sensors Lateral position

Learning physiologicaldiagnostic

Physiological sensorsEyelid movement

Behavioraldiagnostic

Diagnostic fusion

Driver state degradation level

Filtering and Featureextraction

In -Vehicle information

Filtering detection and classification

Driver vigilance status level

Abnormal Driving index

BehavioraldiagnosticPhysiological

diagnostic

www.aide-eu.org

Driver Availability Estimation (DAE) module

Goal: Assess the availability of the driver to receive and process an information at a given time

Three levels of Availability / Unavailability:Available (Green): “I can receive and process information”Unavailable (Red): “I drive my car! Please don’t disturb me”Intermediary level (Amber): “You can send information to me, if needed. ButI’m not sure that I will be able to detect or to process it correctly at this time”

www.aide-eu.org

Goal: monitor the activities of the driver and assess visual distraction, cognitive distraction, and signs of lateral manoeuvring intent.

Cockpit Activity Assessment (CAA) module

Lateral manoeuvringintent

Head/eye sensor

Lane position sensor

Standard vehicle sensors

Gaze-world mapping

Vehicle data post

processing

Head/eye data post

processing

Cognitive distraction indicator calculation

Lateral manoeuvring intent detection

Eyes-off-road detection

Visual time sharing detection

Cognitive distraction detection

Sensor data post processing Feature extractionInput Activity assessment Output

Eyes-off-road

Visual time sharing

Cognitive distraction

Lateral manoeuvringintent

Head/eye sensor

Lane position sensor

Standard vehicle sensors

Gaze-world mapping

Vehicle data post

processing

Head/eye data post

processing

Cognitive distraction indicator calculation

Lateral manoeuvring intent detection

Eyes-off-road detection

Visual time sharing detection

Cognitive distraction detection

Sensor data post processing Feature extractionInput Activity assessment Output

Eyes-off-road

Visual time sharing

Cognitive distraction

www.aide-eu.org

Driver Characteristics (DC) module

User profile configuration

agentPreferred interface elements

Disability

Driving experience

Language

Age

STATIC

PA

RA

METER

S

Self-input of profile by the user

INTERFACE

Preferred interface elements

Disability

Driving experience

Language

Age

STATIC

PA

RA

METER

S

Self-input of profile by the user

INTERFACE

Customisationagent

Reaction Time

TLC

TTCDY

NA

MIC

PA

RA

METER

S Reaction Time

TLC

TTCDY

NA

MIC

PA

RA

METER

S

Interface agent

INPUT DC MODULE OUTPUT

• Time of warning (early, normal, late; continuous within ranges)

• Preferred or not-to-be used modality of warning/info

• Intensity (soft, normal, hard; discrete values)

• Adaptation of specific ADAS (applications)

www.aide-eu.org

Thank you for the attention!

Katia Pagle

email: katia@iccs.grphone: 0030 210 772 3865

fax: 0030 210 772 3557

AIDE Integrated Project web site: www.aide-eu.org