Looking-in and Looking-out Vision for Urban Intelligent Assistance:Estimation of Driver Attentive State and Dynamic Surround

for Safe Merging and Braking

Ashish Tawari1, Member, Sayanan Sivaraman2, Member, Mohan Manubhai Trivedi1, Fellow, IEEE,Trevor Shannon2 and Mario Tippelhofer2

Abstract— This paper details the research, development, anddemonstrations of real-world systems intended to assist thedriver in urban environments, as part of the Urban IntelligentAssist (UIA) research initiative. A 3-year collaboration betweenAudi AG, Volkswagen Group of America Electronics ResearchLaboratory, and UC San Diego, the driver assistance portionof the UIA project focuses on two main use cases of vitalimportance in urban driving. The first, Driver Attention Guard,applies novel computer vision and machine learning researchfor accurately tracking the driver’s head position and rotationusing an array of cameras. The system then infers the driver’sfocus of attention, alerting the driver and engaging safetysystems in case of extended driver inattention. The secondapplication, Merge and Lane Change Assist, applies a novelprobabilistic compact representation of the on-road environ-ment, fusing data from a variety of sensor modalities. Thesystem then computes safe and low-cost merge and lane-changemaneuver recommendations. It communicates desired speedsto the driver via Head-up Display, when the driver touchesthe blinker, indicating his desired lane. The fully-implementedsystems, complete with HMI, were demonstrated to the publicand press in San Francisco in January of 2014.

I. INTRODUCTION

In 2012, there were 5.6 million police-reported motor ve-hicle crashes in the United States with over 33, 000 fatalities,a 3.3-percent increase from the previous year [1]. Fatalities inurban crashes alone increased by 4.9-percentage. The urbandriving environment presents an array of challenges to thedriver. Navigation and path planning are made more difficultdue not only to the “urban canyon,” but also local trafficstates, construction, other factors - like events as well as otherroad users such as pedestrians, cyclists, and other vehicles.

The Urban Intelligent Assist project was formulated toaddress these challenges, to help drivers during in an in-creasingly urbanized world. A 3-year research collaborationbetween Audi AG, VW Electronics Research Laboratory, and3 leading California universities, the goals of the projectinclude enhancement of the drivers convenience, comfort,and safety in the urban driving environment. Various univer-sities have worked on applications including routing, parkingprediction, and driver assistance tasks. In this work, we detail

1Ashish Tawari Mohan M. Trivedi with the Laboratory of Intelligent andSafe Automobiles (http://cvrr.ucsd.edu/lisa/), University of California, SanDiego, La Jolla, CA, USA {atawari, mtrivedi}@ucsd.edu

2 Sayanan Sivaraman, Trevor Shannon and Mario Tippelhofer withVolkswagen Group of America Electronics Research Laboratory in Bel-mont, CA, USA ssivaram@ucsd.edu {trevor.shannon,Mario.Tippelhofer}@vw.com

two driver assistance applications, which observe the driver,and the on-road environment.

Driving tasks involve decision making in three differ-ent time-scales: strategic, tactical and/or critical. Thesetimescales have been proposed by prior researchers [2], [3],[4] in driver modeling. The strategically-planned maneuversare associated with long-term time scale, minutes or hoursof prior planning, and are often motivated by destinationgoal and sometimes comfort as well such as route planning,parking choice etc.

Tactical, or short-term, timescales are on the order ofseconds, and encompass many successive critical operations.Tactically-planned maneuvers are usually motivated by anuncomfortable situation, or occasionally by a recently modi-fied destination goal of the driver, for example, lane changes,turns, stops, upcoming exit etc. Finally, the critical or oper-ational timescale, on the order of hundreds of milliseconds,is the shortest possible timescale for human interaction andare a generally a result of a driver’s desire to remain safein following the rules of the road (posted speed limit, roadcurves, etc.), while carefully operating the vehicle within itslimits. Each of these time-scales presents its own researchchallenges. The tactical and operational time scale time-scales are of particular interest to us, as in former, the driverstill has time to react to the feedback from an assistancesystem and in later, the system can intervene appropriatelyto provide autonomous assistance.

In this paper, we report recently accomplished project andits findings including system design, results and demonstra-tions. In particular, we present two components of the sys-tem, called as, 1) Drive Attention Guard and 2) Merge/LaneChange Recommendation system. Attention Guard providesearly detection of driver distraction by continuously monitor-ing driver and surround traffic situation. It further providesproactive countermeasures that maintain the ego-vehicle asafe following distance from lead vehicle and stay withinmarked lane lines. Merge/Lane Change Recommendationsystem monitors full surround of the ego-vehicle using rangesensors to not only detect that a car is in the blind spot, butalso if a vehicle approaching or being approached is enteringthe space in the adjacent lane. The system continuouslyadjusts predictions about the surround vehicles’ trajectoryin order to assist driver during stress inducing situations likelane change and merging to a highway.

2014 IEEE Intelligent Vehicles Symposium (IV)June 8-11, 2014. Dearborn, Michigan, USA

978-1-4799-3637-3/14/$31.00 ©2014 IEEE 115

(a) (b)

Fig. 1: A unique instrumented testbed: (a) looking-in and looking-out sensors and (b) a depiction of the surround sensingcapabilities of the range (lidar and radar) sensors

Each system has been implemented on two prototypevehicles; an instrumented Audi A8 at University of CaliforniaSan Diego, and an instrumented Audi A6 at the VolkswagenGroup of America Electronics Research Laboratory in Bel-mont, California.

II. RELATED WORK AND MOTIVATION

In the driving context, challenges lies in the developmentof a system which is robust, reliable and can operate continu-ously within the desired design specification. Another impor-tant aspect is how to efficiently relay the outcomes and thesuggestions to the driver in order to maximize the utility andgain driver’s trust. A good Human-Machine-Interface (HMI)is an essential design requirement. From commercial pointof view, these systems need to be non-contact, non-invasiveand be cost effective and yet not compromise aesthetics ofthe vehicle. In this section, we discuss some of the recentworks with above design criteria and are relevant to the twoproposed systems. Each system uses several modules suchas head pose tracking, vehicle detection, lane detection etc.Detailed literature review of each module is beyond the scopeof this paper; however, we will point out to relevant surveywhere appropriate for further reading.

Driver’s focus of attention is intrinsically linked to eye-gaze and head pose [5], [6], [7]. Therefore, eye or headtracking technologies have been extensively used for visualdistraction detection. Robust and reliable eye gaze estimationusing non-contact remote systems is very challenging in thereal-world driving conditions. Even though precise eye gazeis a better indicator of driver’s attention, head pose can stillprovide good cues for driver distraction [8] estimation. Itis also argued that gaze away from the road with detectablehead deviation is, with respect to safety concern, more severethan without head deviation [8]. In search of a reliabletechnology, therefore, in recent years, a lot of advancementshave been made in robust, accurate and continues headpose estimation and tracking [9]. Readers are encouraged torefer to a good overview of different head pose estimationalgorithms by Murphy-Chutorian and Trivedi [10].

Ahlstrom et al. [11] investigated a distraction warningsystem which utilizes eye-gaze and when not available, head

pose to detect the distraction event. A rule based system isused to regulate a fixed time length (2 sec) ‘attention buffer’- an indicator of driver’s attentive state and the trigger for awarning. Although the study does not provide generalizableconclusions due to limited number of participants, it shows apositive trend towards improved visual behavior. The system,however, does not take the surround context into considera-tion and uses heuristics (e.g. minimum speed threshold) toavoid false alarms (e.g. in urban areas). A systematic reviewof various drive distraction detection systems can be foundin [12]. In our work, specifically targeted for urban areas,we incorporate holistic features including vehicle state andsurround context to assess driver attentive state.

Next, we focus on the studies related to driver assistancederived from lane position and surround vehicles dynamics.Complex maneuvers such as lane changes and merges, whichrequire the driver to maintain an awareness of the vehiclesand dynamics in multiple lanes, are of particular concern.Until recently, decision-making for driver assistance lanechanges has fit a binary decision paradigm, the systems basedon fundamental on-road perception answering a yes/no ques-tion. Many decision systems for lane changes have focusedon when a lane change is infeasible, with sensors monitoringthe vehicle’s blind spots [13]. When a driver is being assisted,the feedback is delivered as negative HMI, communicatingthat the maneuver is not feasible. Commercially availableactive safety systems like lane departure warning (LDW) orside warning assist (SWA) typically feature negative HMIwarnings.

In this work, we develop predictive driver assistance forlane changes and merges, integrating positive HMI. Thesystem introduced in this paper is intended to communicatethat the maneuver is feasible, as well as when and how toexecute the lane change or merge.

III. TESTBED DESIGN AND ARCHITECTURE

In order to develop the future Urban Intelligent Assistsystem, a complete vehicular context is required. This com-plete vehicular context includes the driver, vehicle surroundand vehicle state. At the backbone of this research is thecreation of a human-centered intelligent vehicle that captures

116

this complete contextual data and interacts with its occupantsto assist the driver in critical (stressful) situations that existduring urban driving.

The challenges in the design of such testbed lie in se-lection of sensor modalities and sensor placements. Fromcommercial point of view, they need to be unobtrusive andshould not compromise aesthetics. From the research point ofview, the data need to be collected from on-road naturalisticdriving conditions. Since they present the actual real-worldscenarios and set realistic requirements for the developmentof a robust and reliable system. Hence the sensor placementshould not alter normal driving behavior, for example, sensorobstructing driver’s view for safe maneuvering.

A. Hardware Architecture

Built on a 2011 Audi A8, the automotive testbed has beenoutfitted with extensive auxiliary sensing for the research anddevelopment of advanced driver assistance technologies. Thegoal of the testbed buildup is to provide a near-panoramicsensing field of view for experimental data capture. Figure 1shows the different sensing modal as detailed below:

1) Internal Vision: Three cameras are placed monitoringdriving head and foot behavior - two cameras on the A-pillar and near the rear view mirror observing drive’shead movements and another camera pointing downobserving foot movements. They capture face view incolor video stream at 30fps and foot view monochromeinfrared stream at 25fps.

2) External Vision: For looking out at the road, the UIAexperimental testbed features a single forward-lookingcamera, captured at 25Hz. In this study we use thecamera for lane marker detection and lane tracking.

3) Radar: For tracking vehicles on the sides of the ego-vehicle, we employ two medium-range radars, whichhave been installed behind the rear-side panels oneither side of the vehicle. The radars are able to detectand track vehicles as they overtake the ego vehicle oneither side.

4) Lidar: The UIA testbed features two lidar sensors, onefacing forward and one facing backward. We use thesesensors for detecting and tracking vehicles, as well asdetecting obstacles such as guardrails and curbs. Thelidars provide high fidelity sensor information, and areable to estimate parameters such as vehicle length,width, and orientation, as well as position and velocity.

5) GPS and Vehicle Dynamic Sensors: Vehicle state -steering and pedal profile are measured through CANbus interface and for the precise vehicle localization aGPS with inertial measurement unit is installed in theback of the vehicle trunk.

Currently, the experimental testbed features robust compu-tation in the form of a dedicated PC for development, whichtaps all the available data from the on-board vehicle systems.Sensor data from the radars and lidars are fused into a singleobject list, with complete object tracking and re-identificationhandled by sensor-fusion.

Fig. 2: ADTF graphical user interface for synchronized datacapture, processing and visualization.

B. Software architecture

Large amount of data from above mentioned multi-modalmulti-sensory network coming through different channelsUSB, CAN, FlexRay and Ethernet bus system, requiresseamless synchronized capture. We utilize EB Assist ADTF1

(Automotive Data Time Triggered Framework) software de-velopment tool. ADTF provides a graphical user interface todeploy and connect various software components as shownin Figure 2. Besides this the framework offers tools forfor real-time data playback, data handling, processing andvisualization in the lab as well in the test car.

IV. LOOKING-IN LOOKING-OUT (LILO) FRAMEWORK

A human-centric UIA system should be capable of main-taining dynamic representations of the external world sur-rounding the vehicle, the state of the vehicle itself, and stateof the driver. The LiLo framework provides the ability tosimultaneously correlate the dynamic contextual informationof the vehicle interior and the vehicle exterior. In followingsections, we present two systems developed in this frame-work using computer vision, signal processing and machine-learning techniques.

A. Driver Attention Guard

We develop a head pose and its dynamic based attentionmonitoring system. The aim is to mitigate driver distractioneffects using active assistance. The holistic integration ofdriver, vehicle and surround object states is proposed in thedesign of the Attention Guard system. There are followingthree components of the system, continuously monitoringinside driver state and outside environment state along withego vehicle parameters:

1) CoHMEt (Continuous Head Movement Estimator): Acalibrated multi-camera setup is used for reliable andcontinuous head monitoring. Each perspective is pro-cessed by a commercially available head pose trackerand their outputs are fused at later stage, as proposedin [9], to get the final estimate with respect to carreference frame. Readers are encourage to refer [9],

1http://automotive.elektrobit.com/home/driver-assistance-software/eb-assist-adtf.html

117

Fig. 3: Drivability map based on surround sensor fusion and probabilistic modeling.

[14] for algorithmic details and optimal camera posi-tion determination procedure.

2) FOVEA (Field of View Estimation for Attention) Mon-itoring: Head pose and dynamics are then used toestimate driver’s field of view [15], in particular, todetermine whether the driver is paying attention tothe front or not. Furthermore, a confidence measureto determine the quality of the estimate, is calculatedbased on the head pose tracking accuracy provided bythe commercial system. This measure allow the systemto determine whether or not to engage the assistance.

3) Attention Buffer Modulation: Attention buffer reflectsdriver’s attentive state in driving task. Number ofstudies have shown that higher percentage of eyes-off-road duration, either due to single long glances oraccumulated glance duration, is directly proportionalto the increased likelihood of the occurrence of safetycritical events [1]. Hence we use the total eyes-off-roadduration (time duration driver not looking road ahead)as a measure of distraction. Based on the suggestionof Alliance of Automobile Manufacturers (AAM) indesigning HMI, 2 sec worth baseline attention buffer isused which is decreased if the driver looks away fromfront road or increased otherwise until it reaches the 2 sbuffer size. Depletion rate is dependent on front vehicle(if any) dynamics in order to never exceed time-to-collision. When the buffer is fully empty the driveris considered distracted and the assistance is providedby maintaining the ego-vehicle position within lanemarkings and following distance to the lead vehicle.

The full system has been implemented in C++ and runsin real-time. It is tested in controlled test track along withanother vehicle to simulate the front vehicle behavior; furthertesting under naturalistic driving condition is ongoing. Hopeis to make the system and the driver response (if alert) non-differentiable. This is in the direction of positive HMI, wheredriver is not overwhelmed by the frequent warnings but onlywhen absolutely needed and/or assisted with a smooth activecontrol.

B. Merge Recommendations for Driver Assistance

In this work, we make recommendations for lane changeand merge maneuvers. The recommendations consist of rec-ommended accelerations and timings to execute the maneu-ver, specifying how and when to change lanes or merge into

traffic, a problem that features a high number of variables.There are myriad combinations of surround vehicles, laneconfigurations, obstacles, and dynamics to consider.

We base our computation on an intermediate, compactprobabilistic representation of the on-road environment, theDynamic Probabilistic Drivability Map [16]. This compactrepresentation allows us to compactly encode spatial, dy-namic, and legal constraints into one probabilistic represen-tation, which we use to compute the timings and accelera-tions necessary to execute a maneuver. For detail algorithmdescriptions, we encourage reader to refer [17]. Here, weonly mention the main characteristics, which are pertinent tothe discussion.

The DPDM consists of an array of cells, each carryinga probability of drivability P (D), as well as the position,dimensions, and dynamics of any tracked vehicles or obsta-cles that lie within the cell’s boundaries. Figure 3 shows anexample of the drivability map around the ego-vehicle.

We solve for the lowest-cost recommendation to getinto the adjacent lane, by formulating the problem as adynamic programming solution over the DPDM. Dynamicprogramming breaks down a large problem into a seriesof inter-dependent smaller problems. We use the DPDM todecompose the task into a discrete number of computations,computing the cost of accelerating to each possible celllocation within 5 car-lengths of the ego-vehicle.

We note that our system tells the driver not just whetherhe can merge, but when and how to merge. The end result isa recommendation to the driver, detailing when and how tomerge into highway traffic. The current implementation com-putes the recommended acceleration, which can be presentedto the driver as a highlighted recommended range of speedsin the instrument cluster and Heads-Up Display (HUD).

The full system has been implemented in C++ and runs inreal-time. Multiple HMI concepts for relaying the maneuverrecommendations to the driver via HUD have also beenprototyped, with on-road and user interactivity studies inprogress. Figure 6 shows some example HUD mock-ups.The goal is to relay information to the driver in an intuitive,non-distracting manner. In the current implementation, thedriver initiates the HUD merge/lane visualization by usingthe blinker, and the system delivers recommendations forthe desired adjacent lane, left or right. Further applicationsand directions for this body of research, such as wirelessconnectivity and cooperative driving, are explored in [18].

118

Fig. 4: Lane change recommendations over time. Heads-Updisplay for relaying recommendations to the driver and dif-ferent recommendations as shown using intuitive and simpleHMI (only the red boxed portion). Blue screen is surroundsensing visualization of tracked objects and vehicles and isnot shown to the driver.

V. EXPERIMENTAL RESULTS AND ON-ROAD TESTING

The Urban Intelligent Assist project held a public show-case in San Francisco, CA on January 9 and 10, 2014.During this event, real on-road systems were demonstratedto members of the public and press, participants given theopportunity to sit behind the wheel and experience prototypeimplementations of lane and merge assist, driver attentionguard, and research-grade prototype routing and parkingapplications.

We showcase some examples of the on-road systemsworking live on the road. Figure 4 shows a screen grab of thelane change assistance system. The blue area of the figure isa bird’s-eye visualization of raw sensor and vehicle trackingdata. The driver has indicated that he would like to changeto the right lane.

In the first frame, the system shows yellow, and shows arearward arrow, to indicate that the driver should slow downto execute the maneuver. In the next frame, a few secondslater, the driver has slowed down to 60mph, and the systemshows green to indicate that the right lane change is nowsafe.

Distributed camera framework, CoHMEt, was critical forthe success of robust and continuous driver monitoringneeded in the Attention Guard system. We reported a thor-ough study [9] analyzing different camera configurationsincluding number of cameras and their positioning. Figure 5shows improved operational range of the system over 140-degree using two cameras with proper positioning. Fromthe head pose, driver’s field of view is inferred next. Forfrontal (driving direction) and non-frontal gaze detection, thesystem shows very high accuracy. For seven zones (over left-shoulder, left mirror, front, rear-view mirror, right mirror,over right-shoulder and center console) glances itself, thesystem reached close to 95% classification accuracy [15].

For testing the full functionality of the Attention Guardsystem, we require driver to be distracted, looking awayfrom the driving direction. For obvious reasons, this is not

Fig. 5: CoHMEt Framework operating range results: qual-ity of head pose estimate Vs head deviation from frontalposition. Quality > 0 corresponds to a good estimate. Asexpected head pose quality deteriorates with larger deviationfrom frontal position on both side. It can be seen the overalloperating range is over 1400.

done on public roads. A pilot test, however, is successfullyconducted in a controlled test track environment equippedwith lane markings and a lead vehicle to simulate frontvehicle behavior. Figure 6 shows a snapshot of testingphase, when driver is not paying attention to the drivingdirection. Figure 7 shows what are seen by the vehiclesensors - the inside (driver) and the outside context (leadvehicle behavior). The Attention Guard system recognizesthe distracted driver, the attention buffer going to zero (redsignal), and activates the lane keep assist and adaptive cruisecontrol system for steering and braking assistance, needed forthe distracted driver.

The experiences of the technology media, present in theevent, about the above two systems are available in onlinenews media forums and blogs [19], [20].

VI. CONCLUSIONS

Urban streets and arteries roads present unique challengeswith congested traffic condition and other road users likepedestrians, bicyclist etc. With fast urbanization around theglobe, Urban Intelligent Assist project’s aim is to makeurban day-to-day driving safer and stress-free. Towards thisgoal, we presented Driver Attention Guard and Merge/LaneChange Recommendation systems. The systems are installedin a novel testbed, based on 2011 Audi-A8 and Audi-A6model, with unique and innovative sensing technology. Thesystems performed real-time and were tested in naturalisticon-road driving conditions as well as in controlled environ-ment. Further testing in different driving conditions, weatherconditions as well as for longer period are needed to make theproposed prototype system, a commercial product. To furtherincrease the scope of such systems in understanding in-vehicle driver activities, we recommend to include other bodyparts analysis e.g. hand in conjunction with head analysis[21]. We also suggest to include driver behavior analysisfor early intent prediction [22] for the future assistance andrecommendation system.

119

(a) (b)

Fig. 6: Controlled testing of Attention Guard system in Candlestick Park, San Francisco, CA. (a) Driver looking away fromthe froward driving direction and (b) ego-vehicle (white) coming to stop slowly while depending upon lead vehicle behavior.

Fig. 7: Attention Guard: A functional illustration in a con-trolled experiment in a driving test track. The driver isinteracting with infotainment system, not looking towardforward driving direction. The blue screen visualizes thesurround context seen by the car - a lead vehicle (in red) andthe ego-vehicle (in black). The depleted attention buffer (redsignal) shows the distracted driver state with high confidence(blue signal).

ACKNOWLEDGMENTThe authors would like to thank the support from UC

Discovery and industry partners especially Audi and VWElectronic Research Laboratory. We thank other universitypartners, USC and California PATH, Berkeley, for the usefuldiscussions and collaboration. We also thank LISA col-leagues for their valuable assistance.

REFERENCES

[1] NHTSA, “2012 motor vehicle crashes: Overview (dot-hs-811-856),”www-nrd.nhtsa.dot.gov/Cats, 2013. [Online]. Available: http://www-nrd.nhtsa.dot.gov/Pubs/811856.pdf

[2] T. Ranney, “Models of driving behavior: A review of their evolution,”Accident Analysis and Prevention, vol. 26, no. 6, pp. 733–750, 1994.

[3] M. Plochl and J. Edelmann, “Driver models in automobile dynamicsapplication,” Vehicle System Dynamics, vol. 45, no. 7, pp. 699–741,2007.

[4] A. Doshi and M. M. Trivedi, “Tactical driver behavior prediction andintent inference: A review,” in 14th International IEEE Conference onIntelligent Transportation Systems. IEEE, 2011, pp. 1892–1897.

[5] E. Murphy-Chutorian and M. Trivedi, “Hyhope: Hybrid head orienta-tion and position estimation for vision-based driver head tracking,” inIntelligent Vehicles Symposium, 2008 IEEE, june 2008, pp. 512 –517.

[6] A. Doshi and M. M. Trivedi, “Investigating the relationships betweengaze patterns, dynamic vehicle surround analysis, and driver inten-tions,” in IEEE Intelligent Vehicles Symposium, 2009, pp. 887–892.

[7] A. Doshi, B. Morris, and M. M. Trivedi, “On-road prediction ofdriver’s intent with multimodal sensory cues,” IEEE Pervasive Com-puting, vol. 10, no. 3, pp. 22–34, July 2011.

[8] H. Zhang, M. Smith, and R. Dufour. A fi-nal report of safety vehicles using adaptive inter-face technology: Visual distraction. [Online]. Available:http://www.volpe.dot.gov/coi/hfrsa/work/roadway/saveit/docs.html

[9] A. Tawari, S. Martin, and M. M. Trivedi, “Continuous head move-ment estimator for driver assistance: Issues, algorithms and on-roadevaluations,” IEEE Transactions on Intelligent Transportation Systems,vol. PP, pp. 1–13, 2014.

[10] E. Murphy-Chutorian and M. Trivedi, “Head pose estimation incomputer vision: A survey,” IEEE Transactions on Pattern Analysisand Machine Intelligence, vol. 31, no. 4, pp. 607 –626, April 2009.

[11] C. Ahlstrom, K. Kircher, and A. Kircher, “A gaze-based driverdistraction warning system and its effect on visual behavior,” IEEETransactions on Intelligent Transportation Systems, vol. 14, no. 2, pp.965–973, June 2013.

[12] Y. Dong, Z. Hu, K. Uchimura, and N. Murayama, “Driver inattentionmonitoring system for intelligent vehicles: A review,” IntelligentTransportation Systems, IEEE Transactions on, vol. 12, no. 2, pp.596–614, 2011.

[13] R. Schubert, K. Schulze, and G. Wanielik, “Situation assessment forautomatic lane-change maneuvers,” Intelligent Transportation Systems,IEEE Transactions on, vol. 11, no. 3, pp. 607 –616, sept. 2010.

[14] S. Martin, A. Tawari, and M. M. Trivedi, “Monitoring head dynamicsfor driver assistance: A multiple perspective approach,” in 16th IEEEIntelligent Transportation Systems Conference, (ITSC). IEEE, Oct.2013.

[15] A. Tawari and M. M. Trivedi, “Robust and continuous estimation ofdriver gaze zone by dynamic analysis of multiple face videos,” in IEEEIntelligent Vehicles Symposium, June 2014.

[16] S. Sivaraman, M. M. Trivedi, M. Tippelhofer, and T. Shannon, “Mergerecommendations for driver assistance: A cross-modal, cost-sensitiveapproach,” in IEEE Intelligent Vehicles Symposium. IEEE, June 2013.

[17] S. Sivaraman and M. M. Trivedi, “Dynamic probabilistic drivabilitymaps for lane change and merge driver assistance,” IEEE Transactionson Intelligent Transportation Systems, vol. PP, pp. 1–11, 2014.

[18] A. T. McCartt, V. S. Northrup, and R. A. Retting, “Types and char-acteristics of ramp-related motor vehicle crashes on urban interstateroadways in northern virginia,” Journal of Safety Research, vol. 35,no. 1, 2004.

[19] A. Goodwin. (2014, Jan.) Test driving the audi erl tech of tomorrow,today. [Online]. Available: http://reviews.cnet.com/8301-13746 7-57617064-48/test-driving-the-audi-erl-tech-of-tomorrow-today/

[20] (2014, Jan.). [Online]. Available:http://sanfrancisco.cbslocal.com/video/9715465-audis-smart-car-is-like-having-a-very-intelligent-backseat-driver/

[21] S. Martin, E. Ohn-Bar, A. Tawari, and M. M. Trivedi, “Understandinghead and hand activities and coordination in naturalistic drivingvideos,” in IEEE Intelligent Vehicles Symposium, June 2014.

[22] E. Ohn-Bar, A. Tawari, S. Martin, and M. M. Trivedi, “Predictingdriver intent to overtake by learning holistic features,” in IEEEIntelligent Vehicles Symposium, June 2014.

120