Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
Research ArticleIntention-Aware Autonomous Driving Decision-Making inan Uncontrolled Intersection
Weilong Song Guangming Xiong and Huiyan Chen
School of Mechanical Engineering Beijing Institute of Technology Beijing 100081 China
Correspondence should be addressed to Guangming Xiong xiongguangmingbiteducn
Received 3 September 2015 Revised 18 February 2016 Accepted 23 March 2016
Academic Editor Dongsuk Kum
Copyright copy 2016 Weilong Song et al This is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited
Autonomous vehicles need to perform social accepted behaviors in complex urban scenarios including human-driven vehicleswith uncertain intentions This leads to many difficult decision-making problems such as deciding a lane change maneuverand generating policies to pass through intersections In this paper we propose an intention-aware decision-making algorithmto solve this challenging problem in an uncontrolled intersection scenario In order to consider uncertain intentions we firstdevelop a continuous hidden Markov model to predict both the high-level motion intention (eg turn right turn left and gostraight) and the low level interaction intentions (eg yield status for related vehicles)Then a partially observable Markov decisionprocess (POMDP) is built to model the general decision-making framework Due to the difficulty in solving POMDP we useproper assumptions and approximations to simplify this problem A human-like policy generation mechanism is used to generatethe possible candidates Human-driven vehiclesrsquo future motion model is proposed to be applied in state transition process andthe intention is updated during each prediction time step The reward function which considers the driving safety traffic lawstime efficiency and so forth is designed to calculate the optimal policy Finally our method is evaluated in simulation withPreScan software and a driving simulator The experiments show that our method could lead autonomous vehicle to pass throughuncontrolled intersections safely and efficiently
1 Introduction
Autonomous driving technology has developed rapidly in thelast decade In DARPA Urban Challenge [1] autonomousvehicles showed their abilities for interacting in some typicalscenarios such as Tee intersections and lane driving In2011 Google released its autonomous driving platformsOver 10000 miles of autonomous driving for each vehiclewas completed under various traffic conditions [2] Besidesmany big automobile companies also plan to launch theirautonomous driving product in the next several years Withthese significant progresses autonomous vehicles have showntheir potential to reduce the number of traffic accidents andsolve the problem of traffic congestions
One key challenge for autonomous vehicles driven in thereal world is how to deal with the uncertainties such as inac-curacy perception and unclear motion intentions With thedevelopment of intelligent transportation system (ITS) the
perception uncertainty could be solved through the vehicle2Xtechnology and the interactions between autonomous vehi-cles can be solved by centralized or decentralized cooperativecontrol algorithms However human-driven vehicles will stillbe predominance in a short time and the uncertainties oftheir driving intentions will still be retained due to the lackof ldquointention sensorrdquo Human drivers anticipate potentialconflicts continuously make decisions and adjust theirdriving behaviors which are often not rational Thereforeautonomous vehicles need to understand human driversrsquodriving intentions and choose proper actions to behavecooperatively
In this paper we focus on solving this problem in anuncontrolled intersection scenario The uncontrolled inter-section is a complex scenario with high accident rate In USstop signs can be used to normalize the vehiclesrsquo passingsequence However this kind of signs is rarely used in Chinaand the right first traffic laws are often broken by some
Hindawi Publishing CorporationMathematical Problems in EngineeringVolume 2016 Article ID 1025349 15 pageshttpdxdoiorg10115520161025349
2 Mathematical Problems in Engineering
O
1
2
3
4
A
B (x0 y0 0 a0 yaw0)
(x1 y1 1 a1 yaw1)
Figure 1 Amotivation example Autonomous vehicle B is going straight while human-driven vehicle A has three potential driving directionsgoing straight turning right or turning left If vehicle A turns right it will not affect the normal driving of autonomous vehicle B But theother maneuvers including turning left and going straight will lead to a passing sequence problem Besides if they have potential conflictautonomous vehicle B will simulate the trajectories of vehicle A in a prediction horizon and gives the best actions in the current scenarioThevehicles drawn by dash lines are the future prediction positions The red dash lines are the virtual lane assumption used in this paper whichmeans that the vehicles are considered to be driven inside the lane The dark blue area is the potential collision region for these two cars
aggressive drivers Perception failures misunderstandingsand wrong decisions are likely to be performed by humandrivers In such cases even with stop signs the ldquofirst comefirst servedrdquo rule is likely to be broken Besides humandriving behaviors are likely to change as time goes on Withthese uncertain situations specific layout and the trafficrules when autonomous vehicles approach an intersectionthey should have potential ability to recognize the behaviorof other vehicles and give a suitable corresponding behaviorconsidering future evolution of the traffic scenario (seeFigure 1)
With these requirements we propose an intention-awaredecision-making algorithm for autonomous driving in anuncontrolled intersection in this paper Specifically we firstuse easily observed features (eg velocity and position) andcontinuous hidden Markov model (HMM) [3] to build theintention prediction model which outputs the lateral inten-tions (eg turn right turn left and go straight) for human-driven vehicles and longitudinal behavior (eg the yield-ing status) for related vehicles Then a generative partiallyobservable Markov decision process (POMDP) frameworkis built to model the autonomous driving decision-makingprocess This framework is able to deal with the uncertain-ties in the environment including human-driven vehiclesrsquodriving intentions However it is intractable to compute theoptimal policy for general POMDP due to its complexityWe make reasonable approximations and assumptions tosolve this problem in a low computational way A human-like policy generation mechanism is used to compute thepotential policy set A scenario predictionmechanism is usedto simulate the future actions of human-driven vehicles basedon their lateral and longitudinal intentions and the properreward functions are designed to evaluate each strategyTraffic time safety and laws are all considered to get thefinal reward equations The proposed method has been well
evaluated during simulation The main contributions of thispaper are as follows
(i) Modeling a generative autonomous driving decision-making framework considering uncertainties (eghuman driverrsquos intention) in the environment
(ii) Building intention prediction model using easilyobserved parameters (eg velocity and position)for recognizing the realistic lateral and longitudinalbehaviors of human-driven vehicles
(iii) Using reasonable approximations and assumption tobuild an efficient solver based on the specific layout inan uncontrolled intersection area
The structure of this paper is as follows Section 2 reviewsthe related work and two-layer HMM-based intention pre-diction algorithm is discussed in Section 3 Section 4 modelsgeneral autonomous driving decision-making process in aPOMDP while the approximations and the simplified solverare described in Section 5 In Section 6 we evaluate ouralgorithm in a simulated uncontrolled intersection scenariowith PreScan software and a driver simulator Finally theconclusion and future work are discussed in Section 7
2 Related Work
The decision-making module is one of the most impor-tant components of autonomous vehicles connecting envi-ronment perception and vehicle control Thus numerousresearch works are performed to handle autonomous driv-ing decision-making problem in the last decade The mostcommon method is to manually define specific drivingrules corresponding to situations Both finite state machines(FSMs) and hierarchical state machines (HSMs) are usedto evaluate situations and decide in their framework [4ndash6] In DARPA Urban Challenge (DUC) the winner Boss
Mathematical Problems in Engineering 3
used a rule-based behavior generation mechanism to obeythe predefined driving rules based on the obstacle vehiclesrsquometrics [1 6] Boss was able to check vehiclersquos accelerationabilities and the spaces to decide whether merging into a newlane or passing intersections is safe Similarly the decision-making system of ldquoJuniorrdquo [7] ranking second in DUC wasbased on a HSM with manually defined 13 states Due to theadvantages including implementing simply and traceabilitythis framework is widely used in many autonomous drivingplatforms However these approaches always use constantvelocity assumptions and lack considering surrounding vehi-cles future reactions to host vehiclersquos actions Without thisability the driving decisions could have potential risks [8]
In order to consider the evolution of future scenario theplanning and utility-based approaches have been proposedfor decision-making Bahram et al proposed a predictionbased reactive strategy to generate autonomous drivingstrategies [9] A Bayesian classifier is used to predict thefuture motion of obstacle vehicles and a tree-based searchingmechanism is designed to find the optimal driving strategyusing multilevel cost functions However the surroundingvehiclesrsquo reactions to autonomous vehiclesrsquo actions are notconsidered in their frameworkWei et al proposed a compre-hensive approach for autonomous driver model by emulatinghuman driving behavior [10] The human-driven vehicles areassumed to follow a proper social behavior model and thebest velocity profiles are generated in autonomous freewaydriving applications Nonetheless their method does notconsider the motion intention of human-driven vehicles andonly targets in-lane driving In their subsequent work Weiet al modeled traffic interactions and realized autonomousvehicle social behavior in highway entrance ramp [11] Thehuman-driven vehiclesrsquo motion intentions are modeled bya Bayesian model and the human-driven vehiclesrsquo futurereactions are introduced which is based on the yieldingnot-yielding intentions at the first prediction step Autonomousvehicles could perform social cooperative behavior usingtheir frameworkHowever they do not consider the intentionuncertainty over prediction time step
POMDPs provide a mathematical framework for solv-ing the decision-making problem with uncertainties Bai etal proposed an intention-aware approach for autonomousdriving in scenarios with many pedestrians (eg in campus)[12] In their framework the hybrid 119860lowast algorithm is usedto generate global path while a POMDP planner is usedto control the velocity of the autonomous vehicle solvingby an online POMDP solver DESPOT [13] Brechtel et alpresented a probabilistic decision-making algorithm usingcontinuous POMDP [14] They focus on dealing with theuncertainties of incomplete and inaccurate perception in theintersection area while our goal is to deal with the uncertainintentions of human-driven vehicles However the onlinePOMDP solver always needs large computation resource andconsumes much time [15 16] which limits its use in realworld autonomous driving platform Ulbrich and Maurerdesigned a two-step decision-making algorithm to reducethe complexity of the POMDP in lane change scenario [17]Eight POMDP states are manually defined to simplify theproblem in their framework Cunningham et al proposed a
multipolicy decision-making method in lane changing andmerging scenarios [18] POMDPs are used to model thedecision-making problem in their paper while multivehiclesimulation mechanism is used to generate the optimal high-level policy for autonomous vehicle to execute However themotion intentions are not considered
Overall the autonomous driving decision-making prob-lem with uncertain driving intention is still a challengingproblem It is necessary to build an effective behavior predic-tion model for human-driven vehicles Besides it is essentialto incorporate human-driven vehiclesrsquo intentions and behav-iors into autonomous vehicle decision-making system andgenerate suitable actions to ensure autonomous vehicles drivesafely and efficiently This work addresses this problem byfirst building aHMM-based intention predictionmodel thenmodeling human-driven vehiclersquos intentions in a POMDPframework and finally solving it in an approximate method
3 HMM-Based Intention Prediction
In order to pass through an uncontrolled intersectionautonomous vehicles should have the ability to predict thedriving intentions of human-driven vehicles Estimatingdriverrsquos behavior is very difficult because the state of a vehicledriver is in some high-dimensional feature space Instead ofusing driver related features (eg gas pedal brake pedal anddriversrsquo vision) easily observed parameters are used to buildthe intention prediction model in this paper
The vehicle motion intention 119868 considered in thispaper is divided into two aspects lateral intention 119868lat isin
119868TR 119868TL 119868GS 119868S (ie turn right turn left go straight andstop) and longitudinal intention 119868lon isin 119868Yield 119868119873Yield Thelateral intention is a high-level driving maneuver which isdetermined by human driversrsquo long term decision-makingprocess This intention is not always changed in the drivingprocess and determines the future trajectory of human-driven vehicles In particular the intention of stop is treatedas a lateral intention in our model because it can be predictedonly using data from human-driven vehicle itself Howeverthe longitudinal intention is a cooperative behavior onlyoccurring when it interacts with other vehicles We willfirst describe the HMM and then formulize our intentionprediction model in this section
31 HMM A HMM consists of a set of 119873 finite ldquohiddenrdquostates and a set of119872 observable symbols per state The statetransition probabilities are defined as Α = 119886
119894119895 where
119886119894119895= 119875 [119902
119905+1= 119895 | 119902
119905= 119894] 1 le 119894 119895 le 119873 (1)
The initial state distribution is denoted as 120587 = 120587119894 where
120587119894= 119875 [119902
1= 119894] 1 le 119894 le 119873 (2)
Because the observation symbols are continuous param-eters we use Gaussian Mixture Model (GMM) [19] torepresent their probability distribution functions (pdf)
119887119895(o) =
119872
sum
119896=1
119888119895119896N (o | 120583
119895119896 Σ119895119896) 1 le 119895 le 119873 (3)
4 Mathematical Problems in Engineering
where 119888119895119896
represents the mixture coefficient in the 119895th statefor the 119896th mixture N is the pdf of a Gaussian distributionwith mean 120583 and covariance Σmeasured from observation oMixture coefficient 119888 satisfies the following constraints
119872
sum
119896=1
119888119895119896= 1 (4)
where 119888119895119896gt 0 1 le 119895 le 119873 1 le 119896 le 119872
And
int
+infin
minusinfin
119887119895(o) 119889o = 1 1 le 119895 le 119873 (5)
Then a HMM could be completely defined by hiddenstates119873 and the probability tuples 120582 = (120587 119860 119862 120583 Σ)
In the training process we use the Baum-Welch method[20] to estimate model parameters for different driver inten-tion 119868 Once themodel parameters corresponding to differentdriver intention have been trained we can perform thedriverrsquos intention estimation in the recognition process Theprediction process for lateral intentions can be seen inFigure 2
32 HMM-Based Intention Prediction Process Given a con-tinuous HMM the intention prediction process is dividedinto two steps The first step focused on the lateral intentionThe training inputs of each vehiclersquos lateral intention modelin time 119905 are defined as 119861lateral = 119871 V 119886 yaw where 119871 isthe distance to the intersection V is the longitudinal velocity119886 is the longitudinal acceleration and yaw is the yaw ratewhile the output of this model is the motion intentions 119868lat isin119868TR 119868TL 119868GS 119868S The corresponding HMMs can be trainedincluding 120582TR 120582TL 120582GS and 120582119878
The next step is about longitudinal intention This prob-ability could be decomposed based on the total probabilityformula
119875 (119868Yield | 119861) = sum119875 (119868Yield | 119868lat 119861) 119875 (119868lat | 119861)
= 119875 (119868Yield | 119868TR 119861) 119875 (119868TR | 119861)
+ 119875 (119868Yield | 119868TL 119861) 119875 (119868TL)
+ 119875 (119868Yield | 119868GS 119861) 119875 (119868GS | 119861)
+ 119875 (119868Yield | 119868S 119861) 119875 (119868S | 119861)
(6)
where 119861 is the behavior data including 119861lateral and 119861lonIn this process we assume that the lateral behavior 119868lat is
predicted correctly by a deterministic HMM in the first stepand therefore 119868lat is determined by the lateral prediction result119868latPredict where 119875(119868lat | 119861 119868lat = 119868latPredict) = 1 and 119875(119868lat | 119861119868lat = 119868latPredict) = 0 And (6) is reformulated by
119875 (119868Yield | 119861) = 119875 (119868Yield | 119861 119868latPredict) 119875 (119868latPredict | 119861)
= 119875 (119868Yield | 119861 119868latPredict) (7)
The problem is changed to model 119875(119868Yield | 119861 119868latPredict)The features used in longitudinal intention prediction are
119861lon = ΔV Δ119886 ΔDTC where ΔV = Vsocial minus Vhost Δ119886 =
119886social minus 119886host and ΔDTC = DTCsocial minusDTChost DTC meansthe distance to the potential collision area The output ofthe longitudinal intention prediction model is longitudinalmotion intention 119868lon isin 119868Yield 119868119873Yield
Instead of building a generative model we use a deter-ministic approach to restrict 119875(119868Yield | 119861 119868latPredict) as 0 or1 Thus two types of HMMs named 120582
119884119868lat 120582119873119868lat
are trainedwhere 119868lat isin 119868TR 119868TL 119868GS 119868S Two test examples for lateraland longitudinal intention prediction are shown in Figures3 and 4 Through these two figures we can find that ourapproach can recognize human-driven vehiclersquos lateral andlongitudinal intention successfully
4 Modeling Autonomous DrivingDecision-Making in a POMDP Framework
For the decision-making process the key problem is howto design a policy to perform the optimal actions withuncertainties This needs to not only obtain traffic lawsbut also consider the driving uncertainties of human-drivenvehicles Facing potential conflicts human-driven vehicleshave uncertain probabilities to yield autonomous vehicles andsome aggressive drivers may violate the traffic laws Suchelements should be implemented into a powerful decision-making framework As a result we model autonomousdriving decision-making problem in a general POMDPframework in this section
41 POMDP Preliminaries A POMDP model can be for-mulized as a tupleSA 119879Z 119874 119877 120574 where S is a setof states A is the action space and Z denotes a set ofobservations The conditional function 119879(1199041015840 119886 119904) = Pr(1199041015840 |119904 119886) models transition probabilities to state 1199041015840 isin S whenthe system takes an action 119886 isin A in the state 119904 isin S Theobservation function 119874(119911 1199041015840 119886) = Pr(119911 | 119904
1015840
119886) models theprobability of observing 119911 isin Z when an action 119886 isin A istaken and the end state is 1199041015840 isin S The reward function 119877(119904 119886)calculates an immediate reward when taking an action 119886 instate 119904 120574 isin [0 1] is the discount factor in order to balance theimmediate and the future rewards
Because the system contains partially observed state suchas intentions a belief 119887 isin B is maintained A belief updatefunction 120591 is defined as 1198871015840 = 120591(119887 119886 119911) If the agent takes action119886 and gets observation 119911 the new belief 1198871015840 is obtained throughthe Bayesrsquo rule
1198871015840
(1199041015840
) = 120578119874 (1199041015840
119886 119911)sum
119904isin119878
119879 (119904 119886 1199041015840
) 119887 (119904) (8)
where 120578 = 1sum1199041015840isin119878119874(1199041015840
119886 119911)sum119904isin119878119879(119904 119886 119904
1015840
)119887(119904) is a normal-izing constant
A key concept in POMDP planning is a policy a mapping120587 that specifies the action 119886 = 120587(119887) at belief 119887 To solvethe POMDP an optimal policy 120587lowast should be designed tomaximize the total reward
120587lowast
= argmax120587
(119864(
infin
sum
119905=0
120574119905
119877 (119904119905 120587 (119887119905)) | 1198870 120587)) (9)
where 1198870is marked as the initial belief
Mathematical Problems in Engineering 5
Distance tointersection
Velocity
Acceleration
Yaw rate
Observe sequence
HMM_TR Forward algorithm
HMM_TL Forward algorithm
HMM_S Forward algorithm
HMM_GS Forward algorithm
DriverintentionO = B1 B2 BM
1205821 = (1205871 A1 C1 1205831 Σ1)
1205822 = (1205872 A2 C2 1205832 Σ2)
1205823 = (1205873 A3 C3 1205833 Σ4)
1205824 = (1205874 A4 C4 1205834 Σ4)
P(O | 1205822)
P(O | 1205821)
P(O | 1205824)
P(O | 1205824)
arg max P(O | 120582i)i
Figure 2 Prediction process for HMMThe observed sequence will be evaluated by four HMMs Forward algorithm is used to calculate theconditional probabilities and the intention corresponding to the largest value will be considered as the vehiclersquos intention
Time (s)1 2 3 4 5 6
024
0
Predict results
(a)
01 2 3 4 5 60
1020
Velocity
v (m
s)
Time (s)
(b)
0
1 2 3 4 5 60
10
minus10
Acceleration
a (m
s2)
Time (s)
(c)
Time (s)0 1 2 3 4 5 6
minus30minus20minus10
010
Yaw rate
Yaw
rate
(deg
s)
(d)
0 1 2 3 4 5 60
1020
Time (s)
Distance to intersection (DTI)
DTI
(m)
(e)
Figure 3 Lateral intention prediction example The true intention of human-driven vehicle is to turn left in this scenario In the first figurethe value 1 of the 119910 label means turn left 2 means turn right 3 represents go straight and 4 corresponds to stop
42 State Space Because of the Markov property sufficientinformation should be contained in the state space S fordecision-making process [14] The state space includes thevehicle pose [119909 119910 120579] velocity V the average yaw rate yawaveand acceleration 119886ave in the last planning period for allthe vehicles For the human-driven vehicles the lateral andlongitudinal intentions [119868lat 119868lon] also need to be containedfor state transition modeling However the road contextknowledge is static reference information so that it will be notadded to the state space
The joint state 119904 isin S could be denoted as 119904 = [119904host 1199041
1199042 119904
119873]119879 where 119904host is the state of host vehicle (auto-
nomous vehicle) 119904119894 119894 isin 1 2 3 119873 is the state of
human-driven vehicles and 119873 is the number of human-driven vehicles involved Let us define metric state 119909 =
[119909 119910 120579 V 119886ave yawave]119879 including the vehicle position head-
ing velocity acceleration and yaw rateThus the state of hostvehicle can be defined as 119904host = 119909host while the human-driven vehicle state 119904
119894is 119904119894= [119909
119894 119868lat119894 119868lon119894]
119879 With theadvanced perception system and V2V communication tech-nology we assume that the metric state 119909 could be observedBecause the sensor noise is small and hardly affects decision-making process we do not model observation noise for themetric state However the intention state cannot be directlyobserved so it is the partially observable variables in ourpaper The intention state should be inferred from obser-vation data and predictive model over time
43 Action Space In our autonomous vehicle system thedecision-making system is used to select the suitable tactical
6 Mathematical Problems in Engineering
Time (s)0 1 2 3 4 5 6 7 8
012
Predict results
(a)
0 1 2 3 4 5 6 7 80
1020
Relative velocity (RV)
RV (m
s)
Time (s)
(b)
0 1 2 3 4 5 6 7 8minus2
02
Relative acceleration (RA)
RA (m
s2)
Time (s)
(c)
0 1 2 3 4 5 6 7
02040
Time (s)
minus20
Relative DTC (RDTC)
RDTC
(m)
(d)
Figure 4 One example of predicting longitudinal intentions This example is based on the scenario of Figure 1 and two vehicles both gostraightThe value 1 of 119910-axis in the first figure denotes the intention of yielding while 2 represents not yielding In the first 28 s the intentionis yielding After that due to the acceleration action and less relative DTC autonomous vehicle could understand human-driven vehiclersquosnot-yielding intention
maneuvers Specifically in the intersection area autonomousvehicles should follow a global reference path generated bypath planning module The decision-making module onlyneeds to generate accelerationdeceleration commands to thecontrol layer As the reference path may not be straight thesteering control module can adjust the front wheel angleto follow the reference path Therefore the action space Acould be defined as a discrete set A = [acc dec con] whichcontains commands including acceleration deceleration andmaintaining current velocity
44 Observation Space Similar to the joint state space theobservation 119911 is denoted as 119911 = [119911host 1199111 1199112 119911119873]
119879 where119911host and 119911119894 are the host vehicle and human-driven vehiclersquosobservations respectively The acceleration and yaw rate canbe approximately calculated by speed and heading in theconsecutive states
45 State Transition Model In state transition process weneed to model transition probability Pr(1199041015840 | 119904 119886) Thisprobability is determined by each targeted element in thescenario So the transition model can be calculated by thefollowing probabilistic equation
Pr (1199041015840 | 119904 119886) = Pr (1199041015840host | 119904host 119886host)119873
prod
119894=1
Pr (1199041015840119894| 119904119894) (10)
In the decision-making layer we do not need to considercomplex vehicle dynamic model Thus the host vehiclersquosmotion Pr(1199041015840host | 119904host 119886host) can be simply represented by thefollowing equations given action 119886
1199091015840
= 119909 + (V +119886Δ119905
2)Δ119905 cos (120579 + Δ120579)
1199101015840
= 119910 + (V +119886Δ119905
2)Δ119905 sin (120579 + Δ120579)
1205791015840
= 120579 + Δ120579
V1015840 = V + 119886Δ119905
yaw1015840ave =Δ120579
Δ119905
1198861015840
ave = 119886
(11)
Thus the key problem is converted to compute Pr(1199041015840119894|
119904119894) the state transition probability of human-driven vehicles
Based on the total probability formula this probability can befactorized as a sum in whole action space
Pr (1199041015840119894| 119904119894) = sum
119886119894
Pr (1199041015840119894| 119904119894 119886119894)Pr (119886
119894| 119904119894) (12)
With this equation we only need to calculate the statetransition probability Pr(1199041015840
119894| 119904119894 119886119894) given a specific action 119886
119894
and the probability of selecting this action Pr(119886119894| 119904119894) under
current state 119904119894
Because the human-driven vehiclesrsquo state 119904119894= [119909119894 119868119894] the
probability Pr(1199041015840119894| 119904119894 119886119894) can be calculated as
Pr (1199041015840119894| 119904119894 119886119894) = Pr (1199091015840
119894 1198681015840
119894| 119909119894 119868119894 119886119894)
= Pr (1199091015840119894| 119909119894 119868119894 119886119894)Pr (1198681015840
119894| 1199091015840
119894 119909119894 119868119894 119886119894)
(13)
With a certain action 119886119894 Pr(1199091015840119894| 119909119894 119868119894 119886119894) is equal to Pr(1199091015840
119894|
119909119894 119868lat119894 119886119894)The lateral behavior 119868lat119894 is considered to be a goal-
directed driving intention which will not be changed in thedriving process So Pr(1199091015840
119894| 119909119894 119868lat119894 119886119894) is equal to Pr(119909
1015840
119894| 119909119894 119886119894)
given a reference path corresponding to the intention of 119868lat119894Using (11) Pr(1199091015840
119894| 119909119894 119886119894) can be well solved
The remaining problem for calculating Pr(1199041015840119894| 119904119894 119886119894) is
to deal with Pr(1198681015840119894| 1199091015840
119894 119909119894 119868119894 119886119894) The lateral intention 1198681015840lat119894
is assumed stable through the above explanation And the
Mathematical Problems in Engineering 7
longitudinal intention 1198681015840
lon119894 is assumed to be not updatedin this process But it will be updated with new inputs inobservation space
Now Pr(1199041015840119894| 119904119894 119886119894) is well modeled and the remaining
problem is to compute the probabilities Pr(119886119894| 119904119894) of human-
driven vehiclesrsquo future actions
Pr (119886119894| 119904119894) = Pr (119886
119894| 119909119894 119868119894)
= sum
1199091015840
host
Pr (119886119894| 1199091015840
host 119909119894 119868119894)Pr (1199091015840
host | 119909119894 119868119894) (14)
Because 1199091015840
host is determined by the designed policyPr(1199091015840host | 119909119894 119868119894) could be calculated by (11) given an action119886host The probability Pr(119886
119894| 1199091015840
host 119909119894 119868119894) means the distribu-tion of human-driven vehiclesrsquo actions given the new state1199091015840
host of host vehicle the current state of itself and itsintentions Instead of building a complex probability modelwe designed a deterministic mechanism to calculate themostpossible action 119886
119894given 1199091015840host 119909119894 and 119868119894
In this prediction process the host vehicle is assumed tobe maintaining the current actions in the next time step andthe action 119886
119894will be leading human-driven vehicle passing
through the potential collision area either in advance of hostvehicle under the intention 119868
119873Yield or behind the host vehicleunder the intention 119868Yield to keep a safe distance 119889safe In thecase with the intention of 119868
119873Yield we can calculate the lowboundary 119886
119894low of 119886119894 through the above process anddeterminethe upper one using the largest comfort value 119886
119894comfort If119886119894comfort lt 119886
119894low 119886119894low will be used as the human-drivenvehiclersquos action If not we consider the targeted 119886
119894following
a normal distribution with mean value 120583119886119894
between 119886119894low and
119886119894comfort To simplify our model we use the mean value ofthese two boundaries to represent human-driven vehiclersquosaction 119886
119894 Similarly the case with the intention of 119868Yield can
be analyzed in the same processAfter these steps the transition probability Pr(1199041015840 | 119904 119886)
is well formulized and the autonomous vehicle could havethe ability to understand the future motion of the scenariothrough this model
46 Observation Model The observation model is built tosimulate the measurement process The motion intentionis updated in this process The measurements of human-driven vehicles are modeled with conditional independentassumption Thus the observation model can be calculatedas
Pr (119911 | 119886 1199041015840) = Pr (119911host | 1199041015840
host)119873
prod
119894=1
Pr (119911119894| 1199041015840
119894) (15)
The host vehiclersquos observation function is denoted as
Pr (119911host | 1199041015840
host) simN (119911host | 1199091015840
host Σ119911host) (16)
But in this paper due to the use of V2V communicationsensor the observation error almost does not affect the deci-sion-making result The variance matrix is set as zero
The human-driven vehiclersquos observation will follow thevehiclersquos motion intentions Because we do not consider
the observation error the value in metric state will be thesame as the state transition results But the longitudinalintention of human-driven vehicles in the state space will beupdated using the new observations andHMMmentioned inSection 3 The new observation space will be confirmed withthe above step
47 Reward Function The candidate policies have to satisfyseveral evaluation criterions Autonomous vehicles should bedriven safely and comfortably At the same time they shouldfollow the traffic rules and reach the destination as soon aspossible As a result we design objective function (17) consid-ering three aspects including safety time efficiency and trafficlaws where 120583
1 1205832 and 120583
3are the weight coefficient
119877 (119904 119886) = 1205831119877safety (119904 119886) + 1205832119877time (119904 119886)
+ 1205833119877law (119904 119886)
(17)
The detailed information will be discussed in the follow-ing subsections In addition the factor of comfort will be con-sidered and discussed in policy generation part (Section 51)
471 Safety Reward The safety reward function 119877safety(119904 119886)is based on the potential conflict status In our strategysafety reward is defined as a penalty If there are no potentialconflicts the safety rewardwill be set as 0 A large penalty willbe assigned due to the risk of collision status
In an uncontrolled intersection the four approachingdirections are defined as 119860
119894isin 1 2 3 4 (Figure 5) The
driverrsquos lateral intentions are defined as 119868lat isin 119868TR 119868TL 119868GS119868S So the driving trajectory for each vehicle in the inter-section can be generally represented by 119860
119894and 119868lat119895 and we
marked it as 119879119860119894119868lat119895
1 le 119894 le 4 1 le 119895 le 4 The function 119865 isused to judge the potential collision status which is denotedas
119865 (119879119909 119879119910) =
1 if potential conflict
0 otherwise(18)
where 119879119909and 119879
119910are vehiclesrsquo maneuver 119879
119860119894119868lat119895
119865(119909 119910) can be calculated through relative direction
between two cars which is shown in Table 1The safety reward is based on the following items(i) If 119865(119909 119910) is equal to 0 then the safety reward is equal
to 0 due to the noncollision status(ii) If potential collision occurs there will be a large
penalty(iii) If |TTC
119894minus TTChost| lt 119905threshod there is a penalty
depending on |TTC119894minus TTChost| and TTChost
472 Traffic Law Reward Autonomous vehicles should fol-low traffic laws to interactwith human-driven vehicles Trafficlaw is modeled as a function Law(119909 119910) for each two vehicles119909 and 119910
Law (119879119909 119879119910) =
1 if 119909 is prior
0 otherwise(19)
8 Mathematical Problems in Engineering
Human-driven vehicle
Autonomous vehicle
1
4
3
2
Figure 5 One typical scenario for calculating safety reward
Table 1 Safe condition judgments in the intersection
Human-driven vehicle Left side Right side Opposite side Same sideDriving direction TL LK TR S TL LK TR S TL LK TR S TL LK TR S
Turn left permil permil I I permil permil I I permil permil permil I I I I ILane keeping permil permil I I permil permil permil I permil I I I I I I ITurn right I permil I I I I I I permil I I I I I I IStop I I I I I I I I I I I I I I I I
permil indicates potential collision I indicates no potential collision
where 119879119909and 119879
119910are vehiclesrsquo maneuver 119879
119860119894119878119895
This functionLaw(119879
119909 119879119910) is formulized as shown in Algorithm 1
If the behavior will break the law a large penalty is appliedand the behavior of obeying traffic laws will get a zero reward
473 Time Reward The time cost is based on the time to thedestination for the targeted vehicles in the intersection area
Costtime =DTGVhost
(20)
DTG is the distance to the driving goal In addition wealso need to consider the speed limit which is discussed inpolicy generation part in Section 5
5 Approximations on SolvingPOMDP Problem
Solving POMDP is quite difficult The complexity of search-ing total brief space is O(|A|119867|Z|
119867
) [12] where 119867 is theprediction horizon In this paper we model the intentionrecognition process as a deterministic model and use com-munication sensors to ignore the perception error and thusthe size of |Z| is reduced to 1 in the simplified problem Tosolve this problem we first generate the suitable potentialpolicies according to the property of driving tasks and then
select the reasonable total predicting interval time and totalhorizon After that the approximate optimal policy canbe calculated through searching all possible policies withmaximum total rewardThe policy selection process is shownin Algorithm 2 and some detailed explanations are discussedin the subsections
51 Policy Generation For autonomous driving near inter-section the desired velocity curves need to satisfy severalconstraints Firstly except for emergency braking the accel-eration constraints are applied to ensure comfort Secondlythe speed limit constraints should be used in this process Weaim to avoid the acceleration commands when autonomousvehicle is reaching maximum speed limit Thirdly for thecomfort purpose the acceleration command should not bealways changed In otherwords we need tominimize the jerk
Similar to [11] the candidate policies are divided intothree time segments The first two segments are like ldquokeepconstant accelerationdeceleration actionsrdquo while keepingconstant velocity in the third segment We use 119905
1 1199052 and 119905
3
to represent the time periods of these three segments Toguarantee comfort the acceleration is limited to the rangefrom minus4ms2 to 2ms2 and we discrete acceleration actioninto a multiple of [minus05 05 0] Then the action space can berepresented by a discretizing acceleration setThenwe can set
Mathematical Problems in Engineering 9
Law(119879119909 119879119910) larr 0
119905119909larr 1198891199092119868V119909
119905119910larr 1198891199102119868V119910
if 119905119909lt 119905119910minus Δ119905 then
Law(119879119909 119879119910) larr 1
else if |119905119909minus 119905119910| lt Δ119905 then
statuslarr 119865(119879119909 119879119910)
if status = 1 thenif 119868lat119909 = lanekeeping and 119868lat119910 ltgt lanekeeping thenLaw(119879
119909 119879119910) larr 1
else if 119868lat119909 119868lat119910 = lanekeeping and 119860119909minus 119860119910= 1 or minus3 then
Law(119879119909 119879119910) larr 1
else if 119868lat119909 = turnleft and 119868lat119910 = turnright thenLaw(119879
119909 119879119910) larr 1
end ifend if
end ifreturn Law(119879
119909 119879119910)
Algorithm 1 Traffic law formulization
InputPredict horizon119867 time step Δ119905Current states 119904host = 119909host 119904human = [119909human 119868human]
(1) 119875 larr genenratepolicyset()(2) for each 120587
119896isin 119875 do
(3) for 119894 = 1 to119867Δ119905 do(4) 119886host larr 120587
119896(119894)
(5) 1199041015840
host larr updatestate(119904host 119886host)(6) 119886human larr predictactions(1199041015840host 119904human 119868human)
(7) 1199091015840
human larr updatestate(119909human 119886human)
(8) 1198681015840
human larr updateintention(1199041015840host 1199091015840
human)
(9) 1199041015840
human larr [1199091015840
human 1198681015840
human]
(10) 119877(119894) = calculatereward(1199041015840host 1199041015840
human)
(11) 119904host larr 1199041015840
host(12) 119904human larr 119904
1015840
human(13) end(14) 119877
total119896
larr sum119894sum(119877(119894))
(15) end(16) 119896lowast larr argmax
119896(119877
total119896
)
(17) 120587lowast larr 120587119896lowast
(18) return 120587lowast
Algorithm 2 Policy selection process
the value of 1199051 1199052 and 119905
3and the prediction period of single
step An example of policy generation is shown in Figure 6
52 Planning Horizon Selection After building policy gener-ation model the next problem is to select a suitable planninghorizon Longer horizon can lead to a better solution but con-suming more computing resources However as our purpose
is to deal with the interaction problem in the uncontrolledintersection we only need to consider the situation beforeautonomous vehicle gets through In our algorithmwe set theprediction horizon as 8 seconds In addition in the process ofupdating the future state of each vehicle using each policy thecar following mode is used after autonomous vehicle passesthrough the intersection area
10 Mathematical Problems in Engineering
0
1
2Candidate policies
minus4
minus3
minus2
minus1
Acce
lera
tion
(ms2)
1 2 3 4 5 6 7 80Time (s)
(a)
Time (s)0 1 2 3 4 5 6 7 8
0
2
4
6
8
10
12
14
16
18
20
Velo
city
(ms
)
Velocity strategies
(b)
Figure 6 An example of policy generation process (a) is the generated policies and (b) is the corresponding speed profiles The interval ofeach prediction step is 05 s current speed is 12ms2 and the speed limit is 20ms2 The bold black line is one policy In the first 3 secondsautonomous vehicles decelerate in minus35ms2 then accelerate at 2ms2 for 4 seconds and finally stop in the last one second In this case 109policies were generated which is suitable for replanning fast
AB
Figure 7 Testing scenario Autonomous vehicle B and human-driven vehicle A are both approaching the uncontrolled intersection To goacross the intersection successfully autonomous vehicle should interact with human-driven vehicle
53 Time Step Selection Another problem is the predic-tion time step The intention prediction algorithm and thePOMDP are computed in each step If the time step is 119905stepthe total computation timeswill be119867119905stepThus smaller timestep leads to more computation time To solve this problemwe use a simple adaptive time step calculation mechanism togive a final value The time step is selected based on the TTCof autonomous vehicle If the host vehicle is far away from theintersection we can use a very large time step But if the TTCis quite small the low 119905step is applied to ensure safety
6 Experiment and Results
61 Settings In this paper we evaluate our approach throughPreScan 710 [21] a simulation tool for autonomous drivingand connected vehicles Using this software we can build thetesting scenarios (Figure 7) and add vehicles with dynamicmodel In order to get a similar scenario considering social
interaction the driver simulator is added in our experiment(Figure 8) The human-driven vehicle is driven by severalpeople during the experiment and the autonomous vehiclemakes decisions based on the human-driven vehiclersquos drivingbehavior The reference trajectory for autonomous vehicleis generated from path planning module and the human-driven vehiclersquos data (eg position velocity and heading)are transferred through V2V communication sensor Thedecision-making module sends desired velocity command tothe PID controlled to follow the reference path All policies inthe experiment part use a planning horizon119867 = 8 s which isdiscretized into the time step of 05 s
62 Results It is difficult to compare different approachesin the same scenario because the environment is dynamicand not exactly the same However we select two typicalsituations and special settings to make it possible The sameinitial conditions including position orientation and velocity
Mathematical Problems in Engineering 11
Figure 8 Logitech G27 driving simulator
T = 0 1 2 6 s T = 5 s
T = 4 s
T = 3 s
T = 2 s
T = 1 s
T = 0 s
T = 6 s
(a)
T = 0 1 2 9 s
T = 4 s
T = 5 s
T = 6 sT = 7 sT = 8 s
T = 9 s
T = 3 s
T = 2 s
T = 1 s
T = 0 s
(b)
Figure 9 The visualized passing sequence (a) is the result of our approach and (b) represents the result of reactive approach withoutconsidering intention The black vehicle is an autonomous vehicle while the red car is the human-driven vehicle Each vehicle representsthe position in a specific time 119879 with an interval of 1 second
for each vehicle are used in different tests Besides twotypical situations including human-driven vehicle gettingthrough before or after autonomous vehicle are comparedin this section With the same initial state different reac-tions will occur based on various methods We compareour approach and reactive-based method [6] in this sec-tion The key difference for these two methods is that ourapproach considers human-driven vehiclersquos driving inten-tion
The first experiment is that human-driven vehicle triesto yield autonomous vehicle in the interaction process Theresults are shown in Figures 9 and 10 Firstly Figure 9 givesus a visual comparison of the different approaches Fromalmost the same initial state (eg position and velocity) ourapproach could lead to autonomous vehicle passing throughthe intersection more quickly and reasonable
Then let us look at Figure 10 for detailed explanationIn the first 12 s in Figures 10(a) and 10(c) autonomousvehicle maintains speed and understands that human-drivenvehicle will not perform yielding actions Then autonomousvehicle gets yielding intention of human-driven vehicle andunderstands that human-driven vehiclersquos lateral intentionis to go straight Based on candidate policies autonomousvehicle selects acceleration strategy with maximum rewardand finally crosses the intersection In this process we canobviously find that autonomous vehicle understands human-driven vehiclersquos yielding intention Figure 10(c) is an exampleof understand human-driven vehiclersquos behavior based on egovehiclersquos future actions in a specific time Our strategy pre-dicts the future actions of human-driven vehicle Althoughthe velocity curves after 1 s do not correspond it does notaffect the performance of our methods The reason is that
12 Mathematical Problems in Engineering
0 1 2 3 4 5 6 7 80
1020
Time (s)
Velo
city
(ms
) Velocity profile
0 1 2 3 4 5 6 7 8024
Time (s)
Inte
ntio
n Longitudinal intention
0 1 2 3 4 5 6 7 8024
Time (s)
Inte
ntio
n Lateral intention
(a)
0 1 2 3 4 5 6 7 8 90
1020
Time (s)
Velo
city
(ms
) Velocity profile
0 1 2 3 4 5 6 7 8 9024
Time (s)
Inte
ntio
n
Longitudinal intention
0 1 2 3 4 5 6 7 8 9024
Time (s)
Inte
ntio
n
Lateral intention
(b)
0 1 2 3 4 5 6 7 8minus60
minus40
minus20
0
20
40
60
80
Time (s)
Distance to collision region
Dist
ance
(m)
(c)
0 1 2 3 4 5 6 7 8 9minus10
0
10
20
30
40
50
60
70
80
Time (s)
Dist
ance
(m)
Distance to collision region
(d)
0 1 2 3 4 5 6 7 87
8
9
10
11
12
13
14
15
Time (s)
Velo
city
(ms
)
(e)
Time (s)0 1 2 3 4 5 6 7 8
0
2
4
6
8
10
12
14
Velo
city
(ms
)
(f)
Figure 10 Case test 1 In this case human-driven vehicle passes through intersection after autonomous vehicle (a) (c) and (e) are theperformance of our method while (b) (d) and (f) are from the strategy without considering the driving intention (a) and (b) are the velocityprofiles and the corresponding driving intention For longitudinal intention label 1 means yielding and label 2 means not yielding In lateralintention 1 means turning left 2 means turning right 3 means going straight and 4 means stop The intentions in (b) are not used in thatmethod but for detailed analysis (c) and (d) are the distance to collision area for autonomous vehicle and human-driven vehicle respectively(e) and (f) are the prediction and true motions of human-driven vehicles in time 15 s with a prediction length of 8 s The red curves in thesesubfigures are from autonomous vehicle while blue lines are from human-driven vehicle The green lines in (e) and (f) are the predictionvelocity curves of human-driven vehicle
Mathematical Problems in Engineering 13
T = 4 s
T = 1 s
T = 2 s
T = 3 s
T = 5 s
T = 0 s
T = 0 1 2 5 s
(a)
T = 4 s
T = 1 s
T = 2 s
T = 3 s
T = 5 s
T = 0 s
T = 0 1 2 5 s
(b)
Figure 11 The visualized passing sequence for the case of human-driven vehicle first getting through (a) is the result of our approach and(b) is the reactive-based approach
we use a deterministic model in the prediction process andthe prediction value is inside two boundaries to ensure safetyBesides the whole actions of autonomous vehicle in thisprocess could also help human-driven vehicle to understandnot-yielding intention of autonomous vehicles In this casecooperative driving behaviors are performed by both vehi-cles
However if the intention is not considered in this processwe can find the results in Figures 10(b) 10(d) and 10(f)After 2 s in Figure 10(b) while the human-driven vehiclegives a yielding intention autonomous vehicle could notunderstand and they find a potential collision based on theconstant velocity assumptions Then it decreases the speedbut the human-driven vehicle also slows down The puzzledbehavior leads both vehicles to slow down near intersectionFinally human-driven vehicle stops at the stop line and thenautonomous vehicle could pass the intersection In this strat-egy the human-driven vehiclersquos future motion is assumed tobe constant (Figure 10(f))Without understanding of human-driven vehiclersquos intentions this strategy can increase conges-tion problem
Another experiment is that human-driven vehicle tries toget through the intersection first The results are shown inFigures 11 and 12This case is quite typical because many traf-fic accidents in real world are happening in this situation Indetail if one vehicle tries to cross an intersection while violat-ing the law another vehicle will be in great danger if it doesnot understand its behavior From the visualized perform-ance in Figure 11 our method is a little more safe than otherapproaches as there is nearly collision situation in Figure11(b) In detail we can see from Figure 12(a) that our strategycould perform deceleration actions after we understand thenot-yielding intention in 08 sHowever without understand-ing human-driven vehiclersquos motion intention the responsetime has a 1-second delay which may be quite dangerous
In addition it is shown that good performance is in thepredictions of human-driven vehiclersquos future motion in ourmethods (Figure 12(e))
The results of these two cases demonstrate that our algo-rithm could deal with typical scenarios and have better per-formance than traditional reactive controller Autonomousvehicle could be driven more safely fast and comfortablythrough our strategy
7 Conclusion and Future Work
In this paper we proposed an autonomous driving decision-making algorithm considering human-driven vehiclersquos uncer-tain intentions in an uncontrolled intersection The lat-eral and longitudinal intentions are recognized by a con-tinuous HMM Based on HMM and POMDP we modelgeneral decision-making process and then use an approxi-mate approach to solve this complex problem Finally weuse PreScan software and a driving simulator to emulatesocial interaction process The experiment results show thatautonomous vehicles with our approach can pass throughuncontrolled intersections more safely and efficiently thanusing the strategy without considering human-driven vehi-clesrsquo driving intentions
In the near future we aim to implement our approach intoa real autonomous vehicle and perform real world experi-ments In additionmore precious intention recognition algo-rithm aims to be figured out Somemethods like probabilisticgraphic model can be used to get a distribution of each inten-tion Finally designing online POMDP planning algorithmsis also valuable
Competing Interests
The authors declare that there are no competing interestsregarding the publication of this paper
14 Mathematical Problems in Engineering
0 05 1 15 2 25 3 35 4 45 510
20
30
Time (s)
Velo
city
(ms
) Velocity profile
0 05 1 15 2 25 3 35 4 45 50
2
4
Time (s)
Inte
ntio
n
Longitudinal intention
0 05 1 15 2 25 3 35 4 45 5024
Inte
ntio
n Lateral intention
Time (s)
(a)
0 05 1 15 2 25 3 35 4 45 5102030
Time (s)
Velo
city
(ms
) Velocity profile
0 05 1 15 2 25 3 35 4 45 5024
Time (s)
Inte
ntio
n
Longitudinal intention
0 05 1 15 2 25 3 35 4 45 5024
Time (s)
Inte
ntio
n
Lateral intention
(b)
0 05 1 15 2 25 3 35 4 45 5
0
20
40
60
80
Dist
ance
(m)
Distance to collision region
Time (s)
minus40
minus20
(c)
0 05 1 15 2 25 3 35 4 45 5
0
20
40
60
80D
istan
ce (m
)
Distance to collision region
Time (s)
minus40
minus20
(d)
0 1 2 3 4 5 6 7 816
17
18
19
20
21
22
23
24
25
Velo
city
(ms
)
Time (s)
(e)
0 1 2 3 4 5 6 7 817
18
19
20
21
22
23
24
25
Time (s)
Velo
city
(ms
)
(f)
Figure 12 Case test 2 In this case human-driven vehicle passes through intersection before autonomous vehicle through different strategiesThe definition of each subfigure is the same as in Figure 10
Mathematical Problems in Engineering 15
Acknowledgments
This study is supported by the National Natural ScienceFoundation of China (no 91420203)
References
[1] C Urmson J Anhalt D Bagnell et al ldquoAutonomous driving inurban environments boss and the urban challengerdquo Journal ofField Robotics vol 25 no 8 pp 425ndash466 2008
[2] J Markoff ldquoGoogle cars drive themselves in trafficrdquo New YorkTimes vol 9 2010
[3] L R Rabiner and B-H Juang ldquoAn introduction to hiddenMarkov modelsrdquo IEEE ASSP Magazine vol 3 no 1 pp 4ndash161986
[4] M Buehler K Iagnemma and S Singh The DARPA UrbanChallenge Autonomous Vehicles in City Traffic vol 56 Springer2009
[5] S Kammel J Ziegler B Pitzer et al ldquoTeam AnnieWAYrsquos auto-nomous system for the 2007 DARPAUrban Challengerdquo Journalof Field Robotics vol 25 no 9 pp 615ndash639 2008
[6] C R Baker and J M Dolan ldquoTraffic interaction in the urbanchallenge putting boss on its best behaviorrdquo in Proceedingsof the IEEERSJ International Conference on Intelligent Robotsand Systems (IROS rsquo08) pp 1752ndash1758 IEEE Nice FranceSeptember 2008
[7] M Montemerlo J Becker S Shat et al ldquoJunior the Stanfordentry in the urban challengerdquo Journal of Field Robotics vol 25no 9 pp 569ndash597 2008
[8] L Fletcher S Teller E Olson et al ldquoTheMIT-Cornell collisionand why it happenedrdquo Journal of Field Robotics vol 25 no 10pp 775ndash807 2008
[9] M Bahram A Wolf M Aeberhard and D Wollherr ldquoAprediction-based reactive driving strategy for highly automateddriving function on freewaysrdquo in Proceedings of the 25th IEEEIntelligent Vehicles Symposium pp 400ndash406 IEEE DearbornMich USA June 2014
[10] J Wei J M Dolan and B Litkouhi ldquoA prediction- and costfunction-based algorithm for robust autonomous freeway driv-ingrdquo in Proceedings of the IEEE Intelligent Vehicles Symposium(IV rsquo10) pp 512ndash517 San Diego Calif USA June 2010
[11] J Wei J M Dolan and B Litkouhi ldquoAutonomous vehiclesocial behavior for highway entrance ramp managementrdquo inProceedings of the IEEE Intelligent Vehicles Symposium (IV rsquo13)pp 201ndash207 IEEE Gold Coast Australia June 2013
[12] H Bai S Cai N Ye D Hsu and W S Lee ldquoIntention-awareonline POMDP planning for autonomous driving in a crowdrdquoin Proceedings of the IEEE International Conference on Roboticsand Automation (ICRA rsquo15) pp 454ndash460 IEEE Seattle WashUSA May 2015
[13] A Somani N Ye D Hsu and W S Lee ldquoDespot onlinepomdp planning with regularizationrdquo in Advances in NeuralInformation Processing Systems pp 1772ndash1780 2013
[14] S Brechtel T Gindele and R Dillmann ldquoProbabilistic deci-sion-making under uncertainty for autonomous driving usingcontinuous POMDPsrdquo in Proceedings of the 17th IEEE Interna-tional Conference on Intelligent Transportation Systems (ITSCrsquo14) pp 392ndash399 Qingdao China October 2014
[15] C H Papadimitriou and J N Tsitsiklis ldquoThe complexity ofMarkov decision processesrdquo Mathematics of Operations Re-search vol 12 no 3 pp 441ndash450 1987
[16] O Madani S Hanks and A Condon ldquoOn the undecidabilityof probabilistic planning and related stochastic optimizationproblemsrdquoArtificial Intelligence vol 147 no 1-2 pp 5ndash34 2003
[17] S Ulbrich and M Maurer ldquoProbabilistic online POMDP deci-sion making for lane changes in fully automated drivingrdquo inProceedings of the 16th International IEEE Conference on Intelli-gent Transportation Systems (ITSC rsquo13) pp 2063ndash2067 TheHague The Netherlands October 2013
[18] A G Cunningham E Galceran R M Eustice and E OlsonldquoMPDM multipolicy decision-making in dynamic uncertainenvironments for autonomous drivingrdquo in Proceedings of theIEEE International Conference on Robotics and Automation(ICRA rsquo15) pp 1670ndash1677 Seattle Wash USA May 2015
[19] D A Reynolds and R C Rose ldquoRobust text-independentspeaker identification using Gaussian mixture speaker modelsrdquoIEEE Transactions on Speech and Audio Processing vol 3 no 1pp 72ndash83 1995
[20] L E Baum and T Petrie ldquoStatistical inference for probabilisticfunctions of finite state Markov chainsrdquo The Annals of Mathe-matical Statistics vol 37 pp 1554ndash1563 1966
[21] M Tideman and M Van Noort ldquoA simulation tool suite fordeveloping connected vehicle systemsrdquo in Proceedings of theIEEE Intelligent Vehicles Symposium (IEEE IV rsquo13) pp 713ndash718Queensland Australia June 2013
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
2 Mathematical Problems in Engineering
O
1
2
3
4
A
B (x0 y0 0 a0 yaw0)
(x1 y1 1 a1 yaw1)
Figure 1 Amotivation example Autonomous vehicle B is going straight while human-driven vehicle A has three potential driving directionsgoing straight turning right or turning left If vehicle A turns right it will not affect the normal driving of autonomous vehicle B But theother maneuvers including turning left and going straight will lead to a passing sequence problem Besides if they have potential conflictautonomous vehicle B will simulate the trajectories of vehicle A in a prediction horizon and gives the best actions in the current scenarioThevehicles drawn by dash lines are the future prediction positions The red dash lines are the virtual lane assumption used in this paper whichmeans that the vehicles are considered to be driven inside the lane The dark blue area is the potential collision region for these two cars
aggressive drivers Perception failures misunderstandingsand wrong decisions are likely to be performed by humandrivers In such cases even with stop signs the ldquofirst comefirst servedrdquo rule is likely to be broken Besides humandriving behaviors are likely to change as time goes on Withthese uncertain situations specific layout and the trafficrules when autonomous vehicles approach an intersectionthey should have potential ability to recognize the behaviorof other vehicles and give a suitable corresponding behaviorconsidering future evolution of the traffic scenario (seeFigure 1)
With these requirements we propose an intention-awaredecision-making algorithm for autonomous driving in anuncontrolled intersection in this paper Specifically we firstuse easily observed features (eg velocity and position) andcontinuous hidden Markov model (HMM) [3] to build theintention prediction model which outputs the lateral inten-tions (eg turn right turn left and go straight) for human-driven vehicles and longitudinal behavior (eg the yield-ing status) for related vehicles Then a generative partiallyobservable Markov decision process (POMDP) frameworkis built to model the autonomous driving decision-makingprocess This framework is able to deal with the uncertain-ties in the environment including human-driven vehiclesrsquodriving intentions However it is intractable to compute theoptimal policy for general POMDP due to its complexityWe make reasonable approximations and assumptions tosolve this problem in a low computational way A human-like policy generation mechanism is used to compute thepotential policy set A scenario predictionmechanism is usedto simulate the future actions of human-driven vehicles basedon their lateral and longitudinal intentions and the properreward functions are designed to evaluate each strategyTraffic time safety and laws are all considered to get thefinal reward equations The proposed method has been well
evaluated during simulation The main contributions of thispaper are as follows
(i) Modeling a generative autonomous driving decision-making framework considering uncertainties (eghuman driverrsquos intention) in the environment
(ii) Building intention prediction model using easilyobserved parameters (eg velocity and position)for recognizing the realistic lateral and longitudinalbehaviors of human-driven vehicles
(iii) Using reasonable approximations and assumption tobuild an efficient solver based on the specific layout inan uncontrolled intersection area
The structure of this paper is as follows Section 2 reviewsthe related work and two-layer HMM-based intention pre-diction algorithm is discussed in Section 3 Section 4 modelsgeneral autonomous driving decision-making process in aPOMDP while the approximations and the simplified solverare described in Section 5 In Section 6 we evaluate ouralgorithm in a simulated uncontrolled intersection scenariowith PreScan software and a driver simulator Finally theconclusion and future work are discussed in Section 7
2 Related Work
The decision-making module is one of the most impor-tant components of autonomous vehicles connecting envi-ronment perception and vehicle control Thus numerousresearch works are performed to handle autonomous driv-ing decision-making problem in the last decade The mostcommon method is to manually define specific drivingrules corresponding to situations Both finite state machines(FSMs) and hierarchical state machines (HSMs) are usedto evaluate situations and decide in their framework [4ndash6] In DARPA Urban Challenge (DUC) the winner Boss
Mathematical Problems in Engineering 3
used a rule-based behavior generation mechanism to obeythe predefined driving rules based on the obstacle vehiclesrsquometrics [1 6] Boss was able to check vehiclersquos accelerationabilities and the spaces to decide whether merging into a newlane or passing intersections is safe Similarly the decision-making system of ldquoJuniorrdquo [7] ranking second in DUC wasbased on a HSM with manually defined 13 states Due to theadvantages including implementing simply and traceabilitythis framework is widely used in many autonomous drivingplatforms However these approaches always use constantvelocity assumptions and lack considering surrounding vehi-cles future reactions to host vehiclersquos actions Without thisability the driving decisions could have potential risks [8]
In order to consider the evolution of future scenario theplanning and utility-based approaches have been proposedfor decision-making Bahram et al proposed a predictionbased reactive strategy to generate autonomous drivingstrategies [9] A Bayesian classifier is used to predict thefuture motion of obstacle vehicles and a tree-based searchingmechanism is designed to find the optimal driving strategyusing multilevel cost functions However the surroundingvehiclesrsquo reactions to autonomous vehiclesrsquo actions are notconsidered in their frameworkWei et al proposed a compre-hensive approach for autonomous driver model by emulatinghuman driving behavior [10] The human-driven vehicles areassumed to follow a proper social behavior model and thebest velocity profiles are generated in autonomous freewaydriving applications Nonetheless their method does notconsider the motion intention of human-driven vehicles andonly targets in-lane driving In their subsequent work Weiet al modeled traffic interactions and realized autonomousvehicle social behavior in highway entrance ramp [11] Thehuman-driven vehiclesrsquo motion intentions are modeled bya Bayesian model and the human-driven vehiclesrsquo futurereactions are introduced which is based on the yieldingnot-yielding intentions at the first prediction step Autonomousvehicles could perform social cooperative behavior usingtheir frameworkHowever they do not consider the intentionuncertainty over prediction time step
POMDPs provide a mathematical framework for solv-ing the decision-making problem with uncertainties Bai etal proposed an intention-aware approach for autonomousdriving in scenarios with many pedestrians (eg in campus)[12] In their framework the hybrid 119860lowast algorithm is usedto generate global path while a POMDP planner is usedto control the velocity of the autonomous vehicle solvingby an online POMDP solver DESPOT [13] Brechtel et alpresented a probabilistic decision-making algorithm usingcontinuous POMDP [14] They focus on dealing with theuncertainties of incomplete and inaccurate perception in theintersection area while our goal is to deal with the uncertainintentions of human-driven vehicles However the onlinePOMDP solver always needs large computation resource andconsumes much time [15 16] which limits its use in realworld autonomous driving platform Ulbrich and Maurerdesigned a two-step decision-making algorithm to reducethe complexity of the POMDP in lane change scenario [17]Eight POMDP states are manually defined to simplify theproblem in their framework Cunningham et al proposed a
multipolicy decision-making method in lane changing andmerging scenarios [18] POMDPs are used to model thedecision-making problem in their paper while multivehiclesimulation mechanism is used to generate the optimal high-level policy for autonomous vehicle to execute However themotion intentions are not considered
Overall the autonomous driving decision-making prob-lem with uncertain driving intention is still a challengingproblem It is necessary to build an effective behavior predic-tion model for human-driven vehicles Besides it is essentialto incorporate human-driven vehiclesrsquo intentions and behav-iors into autonomous vehicle decision-making system andgenerate suitable actions to ensure autonomous vehicles drivesafely and efficiently This work addresses this problem byfirst building aHMM-based intention predictionmodel thenmodeling human-driven vehiclersquos intentions in a POMDPframework and finally solving it in an approximate method
3 HMM-Based Intention Prediction
In order to pass through an uncontrolled intersectionautonomous vehicles should have the ability to predict thedriving intentions of human-driven vehicles Estimatingdriverrsquos behavior is very difficult because the state of a vehicledriver is in some high-dimensional feature space Instead ofusing driver related features (eg gas pedal brake pedal anddriversrsquo vision) easily observed parameters are used to buildthe intention prediction model in this paper
The vehicle motion intention 119868 considered in thispaper is divided into two aspects lateral intention 119868lat isin
119868TR 119868TL 119868GS 119868S (ie turn right turn left go straight andstop) and longitudinal intention 119868lon isin 119868Yield 119868119873Yield Thelateral intention is a high-level driving maneuver which isdetermined by human driversrsquo long term decision-makingprocess This intention is not always changed in the drivingprocess and determines the future trajectory of human-driven vehicles In particular the intention of stop is treatedas a lateral intention in our model because it can be predictedonly using data from human-driven vehicle itself Howeverthe longitudinal intention is a cooperative behavior onlyoccurring when it interacts with other vehicles We willfirst describe the HMM and then formulize our intentionprediction model in this section
31 HMM A HMM consists of a set of 119873 finite ldquohiddenrdquostates and a set of119872 observable symbols per state The statetransition probabilities are defined as Α = 119886
119894119895 where
119886119894119895= 119875 [119902
119905+1= 119895 | 119902
119905= 119894] 1 le 119894 119895 le 119873 (1)
The initial state distribution is denoted as 120587 = 120587119894 where
120587119894= 119875 [119902
1= 119894] 1 le 119894 le 119873 (2)
Because the observation symbols are continuous param-eters we use Gaussian Mixture Model (GMM) [19] torepresent their probability distribution functions (pdf)
119887119895(o) =
119872
sum
119896=1
119888119895119896N (o | 120583
119895119896 Σ119895119896) 1 le 119895 le 119873 (3)
4 Mathematical Problems in Engineering
where 119888119895119896
represents the mixture coefficient in the 119895th statefor the 119896th mixture N is the pdf of a Gaussian distributionwith mean 120583 and covariance Σmeasured from observation oMixture coefficient 119888 satisfies the following constraints
119872
sum
119896=1
119888119895119896= 1 (4)
where 119888119895119896gt 0 1 le 119895 le 119873 1 le 119896 le 119872
And
int
+infin
minusinfin
119887119895(o) 119889o = 1 1 le 119895 le 119873 (5)
Then a HMM could be completely defined by hiddenstates119873 and the probability tuples 120582 = (120587 119860 119862 120583 Σ)
In the training process we use the Baum-Welch method[20] to estimate model parameters for different driver inten-tion 119868 Once themodel parameters corresponding to differentdriver intention have been trained we can perform thedriverrsquos intention estimation in the recognition process Theprediction process for lateral intentions can be seen inFigure 2
32 HMM-Based Intention Prediction Process Given a con-tinuous HMM the intention prediction process is dividedinto two steps The first step focused on the lateral intentionThe training inputs of each vehiclersquos lateral intention modelin time 119905 are defined as 119861lateral = 119871 V 119886 yaw where 119871 isthe distance to the intersection V is the longitudinal velocity119886 is the longitudinal acceleration and yaw is the yaw ratewhile the output of this model is the motion intentions 119868lat isin119868TR 119868TL 119868GS 119868S The corresponding HMMs can be trainedincluding 120582TR 120582TL 120582GS and 120582119878
The next step is about longitudinal intention This prob-ability could be decomposed based on the total probabilityformula
119875 (119868Yield | 119861) = sum119875 (119868Yield | 119868lat 119861) 119875 (119868lat | 119861)
= 119875 (119868Yield | 119868TR 119861) 119875 (119868TR | 119861)
+ 119875 (119868Yield | 119868TL 119861) 119875 (119868TL)
+ 119875 (119868Yield | 119868GS 119861) 119875 (119868GS | 119861)
+ 119875 (119868Yield | 119868S 119861) 119875 (119868S | 119861)
(6)
where 119861 is the behavior data including 119861lateral and 119861lonIn this process we assume that the lateral behavior 119868lat is
predicted correctly by a deterministic HMM in the first stepand therefore 119868lat is determined by the lateral prediction result119868latPredict where 119875(119868lat | 119861 119868lat = 119868latPredict) = 1 and 119875(119868lat | 119861119868lat = 119868latPredict) = 0 And (6) is reformulated by
119875 (119868Yield | 119861) = 119875 (119868Yield | 119861 119868latPredict) 119875 (119868latPredict | 119861)
= 119875 (119868Yield | 119861 119868latPredict) (7)
The problem is changed to model 119875(119868Yield | 119861 119868latPredict)The features used in longitudinal intention prediction are
119861lon = ΔV Δ119886 ΔDTC where ΔV = Vsocial minus Vhost Δ119886 =
119886social minus 119886host and ΔDTC = DTCsocial minusDTChost DTC meansthe distance to the potential collision area The output ofthe longitudinal intention prediction model is longitudinalmotion intention 119868lon isin 119868Yield 119868119873Yield
Instead of building a generative model we use a deter-ministic approach to restrict 119875(119868Yield | 119861 119868latPredict) as 0 or1 Thus two types of HMMs named 120582
119884119868lat 120582119873119868lat
are trainedwhere 119868lat isin 119868TR 119868TL 119868GS 119868S Two test examples for lateraland longitudinal intention prediction are shown in Figures3 and 4 Through these two figures we can find that ourapproach can recognize human-driven vehiclersquos lateral andlongitudinal intention successfully
4 Modeling Autonomous DrivingDecision-Making in a POMDP Framework
For the decision-making process the key problem is howto design a policy to perform the optimal actions withuncertainties This needs to not only obtain traffic lawsbut also consider the driving uncertainties of human-drivenvehicles Facing potential conflicts human-driven vehicleshave uncertain probabilities to yield autonomous vehicles andsome aggressive drivers may violate the traffic laws Suchelements should be implemented into a powerful decision-making framework As a result we model autonomousdriving decision-making problem in a general POMDPframework in this section
41 POMDP Preliminaries A POMDP model can be for-mulized as a tupleSA 119879Z 119874 119877 120574 where S is a setof states A is the action space and Z denotes a set ofobservations The conditional function 119879(1199041015840 119886 119904) = Pr(1199041015840 |119904 119886) models transition probabilities to state 1199041015840 isin S whenthe system takes an action 119886 isin A in the state 119904 isin S Theobservation function 119874(119911 1199041015840 119886) = Pr(119911 | 119904
1015840
119886) models theprobability of observing 119911 isin Z when an action 119886 isin A istaken and the end state is 1199041015840 isin S The reward function 119877(119904 119886)calculates an immediate reward when taking an action 119886 instate 119904 120574 isin [0 1] is the discount factor in order to balance theimmediate and the future rewards
Because the system contains partially observed state suchas intentions a belief 119887 isin B is maintained A belief updatefunction 120591 is defined as 1198871015840 = 120591(119887 119886 119911) If the agent takes action119886 and gets observation 119911 the new belief 1198871015840 is obtained throughthe Bayesrsquo rule
1198871015840
(1199041015840
) = 120578119874 (1199041015840
119886 119911)sum
119904isin119878
119879 (119904 119886 1199041015840
) 119887 (119904) (8)
where 120578 = 1sum1199041015840isin119878119874(1199041015840
119886 119911)sum119904isin119878119879(119904 119886 119904
1015840
)119887(119904) is a normal-izing constant
A key concept in POMDP planning is a policy a mapping120587 that specifies the action 119886 = 120587(119887) at belief 119887 To solvethe POMDP an optimal policy 120587lowast should be designed tomaximize the total reward
120587lowast
= argmax120587
(119864(
infin
sum
119905=0
120574119905
119877 (119904119905 120587 (119887119905)) | 1198870 120587)) (9)
where 1198870is marked as the initial belief
Mathematical Problems in Engineering 5
Distance tointersection
Velocity
Acceleration
Yaw rate
Observe sequence
HMM_TR Forward algorithm
HMM_TL Forward algorithm
HMM_S Forward algorithm
HMM_GS Forward algorithm
DriverintentionO = B1 B2 BM
1205821 = (1205871 A1 C1 1205831 Σ1)
1205822 = (1205872 A2 C2 1205832 Σ2)
1205823 = (1205873 A3 C3 1205833 Σ4)
1205824 = (1205874 A4 C4 1205834 Σ4)
P(O | 1205822)
P(O | 1205821)
P(O | 1205824)
P(O | 1205824)
arg max P(O | 120582i)i
Figure 2 Prediction process for HMMThe observed sequence will be evaluated by four HMMs Forward algorithm is used to calculate theconditional probabilities and the intention corresponding to the largest value will be considered as the vehiclersquos intention
Time (s)1 2 3 4 5 6
024
0
Predict results
(a)
01 2 3 4 5 60
1020
Velocity
v (m
s)
Time (s)
(b)
0
1 2 3 4 5 60
10
minus10
Acceleration
a (m
s2)
Time (s)
(c)
Time (s)0 1 2 3 4 5 6
minus30minus20minus10
010
Yaw rate
Yaw
rate
(deg
s)
(d)
0 1 2 3 4 5 60
1020
Time (s)
Distance to intersection (DTI)
DTI
(m)
(e)
Figure 3 Lateral intention prediction example The true intention of human-driven vehicle is to turn left in this scenario In the first figurethe value 1 of the 119910 label means turn left 2 means turn right 3 represents go straight and 4 corresponds to stop
42 State Space Because of the Markov property sufficientinformation should be contained in the state space S fordecision-making process [14] The state space includes thevehicle pose [119909 119910 120579] velocity V the average yaw rate yawaveand acceleration 119886ave in the last planning period for allthe vehicles For the human-driven vehicles the lateral andlongitudinal intentions [119868lat 119868lon] also need to be containedfor state transition modeling However the road contextknowledge is static reference information so that it will be notadded to the state space
The joint state 119904 isin S could be denoted as 119904 = [119904host 1199041
1199042 119904
119873]119879 where 119904host is the state of host vehicle (auto-
nomous vehicle) 119904119894 119894 isin 1 2 3 119873 is the state of
human-driven vehicles and 119873 is the number of human-driven vehicles involved Let us define metric state 119909 =
[119909 119910 120579 V 119886ave yawave]119879 including the vehicle position head-
ing velocity acceleration and yaw rateThus the state of hostvehicle can be defined as 119904host = 119909host while the human-driven vehicle state 119904
119894is 119904119894= [119909
119894 119868lat119894 119868lon119894]
119879 With theadvanced perception system and V2V communication tech-nology we assume that the metric state 119909 could be observedBecause the sensor noise is small and hardly affects decision-making process we do not model observation noise for themetric state However the intention state cannot be directlyobserved so it is the partially observable variables in ourpaper The intention state should be inferred from obser-vation data and predictive model over time
43 Action Space In our autonomous vehicle system thedecision-making system is used to select the suitable tactical
6 Mathematical Problems in Engineering
Time (s)0 1 2 3 4 5 6 7 8
012
Predict results
(a)
0 1 2 3 4 5 6 7 80
1020
Relative velocity (RV)
RV (m
s)
Time (s)
(b)
0 1 2 3 4 5 6 7 8minus2
02
Relative acceleration (RA)
RA (m
s2)
Time (s)
(c)
0 1 2 3 4 5 6 7
02040
Time (s)
minus20
Relative DTC (RDTC)
RDTC
(m)
(d)
Figure 4 One example of predicting longitudinal intentions This example is based on the scenario of Figure 1 and two vehicles both gostraightThe value 1 of 119910-axis in the first figure denotes the intention of yielding while 2 represents not yielding In the first 28 s the intentionis yielding After that due to the acceleration action and less relative DTC autonomous vehicle could understand human-driven vehiclersquosnot-yielding intention
maneuvers Specifically in the intersection area autonomousvehicles should follow a global reference path generated bypath planning module The decision-making module onlyneeds to generate accelerationdeceleration commands to thecontrol layer As the reference path may not be straight thesteering control module can adjust the front wheel angleto follow the reference path Therefore the action space Acould be defined as a discrete set A = [acc dec con] whichcontains commands including acceleration deceleration andmaintaining current velocity
44 Observation Space Similar to the joint state space theobservation 119911 is denoted as 119911 = [119911host 1199111 1199112 119911119873]
119879 where119911host and 119911119894 are the host vehicle and human-driven vehiclersquosobservations respectively The acceleration and yaw rate canbe approximately calculated by speed and heading in theconsecutive states
45 State Transition Model In state transition process weneed to model transition probability Pr(1199041015840 | 119904 119886) Thisprobability is determined by each targeted element in thescenario So the transition model can be calculated by thefollowing probabilistic equation
Pr (1199041015840 | 119904 119886) = Pr (1199041015840host | 119904host 119886host)119873
prod
119894=1
Pr (1199041015840119894| 119904119894) (10)
In the decision-making layer we do not need to considercomplex vehicle dynamic model Thus the host vehiclersquosmotion Pr(1199041015840host | 119904host 119886host) can be simply represented by thefollowing equations given action 119886
1199091015840
= 119909 + (V +119886Δ119905
2)Δ119905 cos (120579 + Δ120579)
1199101015840
= 119910 + (V +119886Δ119905
2)Δ119905 sin (120579 + Δ120579)
1205791015840
= 120579 + Δ120579
V1015840 = V + 119886Δ119905
yaw1015840ave =Δ120579
Δ119905
1198861015840
ave = 119886
(11)
Thus the key problem is converted to compute Pr(1199041015840119894|
119904119894) the state transition probability of human-driven vehicles
Based on the total probability formula this probability can befactorized as a sum in whole action space
Pr (1199041015840119894| 119904119894) = sum
119886119894
Pr (1199041015840119894| 119904119894 119886119894)Pr (119886
119894| 119904119894) (12)
With this equation we only need to calculate the statetransition probability Pr(1199041015840
119894| 119904119894 119886119894) given a specific action 119886
119894
and the probability of selecting this action Pr(119886119894| 119904119894) under
current state 119904119894
Because the human-driven vehiclesrsquo state 119904119894= [119909119894 119868119894] the
probability Pr(1199041015840119894| 119904119894 119886119894) can be calculated as
Pr (1199041015840119894| 119904119894 119886119894) = Pr (1199091015840
119894 1198681015840
119894| 119909119894 119868119894 119886119894)
= Pr (1199091015840119894| 119909119894 119868119894 119886119894)Pr (1198681015840
119894| 1199091015840
119894 119909119894 119868119894 119886119894)
(13)
With a certain action 119886119894 Pr(1199091015840119894| 119909119894 119868119894 119886119894) is equal to Pr(1199091015840
119894|
119909119894 119868lat119894 119886119894)The lateral behavior 119868lat119894 is considered to be a goal-
directed driving intention which will not be changed in thedriving process So Pr(1199091015840
119894| 119909119894 119868lat119894 119886119894) is equal to Pr(119909
1015840
119894| 119909119894 119886119894)
given a reference path corresponding to the intention of 119868lat119894Using (11) Pr(1199091015840
119894| 119909119894 119886119894) can be well solved
The remaining problem for calculating Pr(1199041015840119894| 119904119894 119886119894) is
to deal with Pr(1198681015840119894| 1199091015840
119894 119909119894 119868119894 119886119894) The lateral intention 1198681015840lat119894
is assumed stable through the above explanation And the
Mathematical Problems in Engineering 7
longitudinal intention 1198681015840
lon119894 is assumed to be not updatedin this process But it will be updated with new inputs inobservation space
Now Pr(1199041015840119894| 119904119894 119886119894) is well modeled and the remaining
problem is to compute the probabilities Pr(119886119894| 119904119894) of human-
driven vehiclesrsquo future actions
Pr (119886119894| 119904119894) = Pr (119886
119894| 119909119894 119868119894)
= sum
1199091015840
host
Pr (119886119894| 1199091015840
host 119909119894 119868119894)Pr (1199091015840
host | 119909119894 119868119894) (14)
Because 1199091015840
host is determined by the designed policyPr(1199091015840host | 119909119894 119868119894) could be calculated by (11) given an action119886host The probability Pr(119886
119894| 1199091015840
host 119909119894 119868119894) means the distribu-tion of human-driven vehiclesrsquo actions given the new state1199091015840
host of host vehicle the current state of itself and itsintentions Instead of building a complex probability modelwe designed a deterministic mechanism to calculate themostpossible action 119886
119894given 1199091015840host 119909119894 and 119868119894
In this prediction process the host vehicle is assumed tobe maintaining the current actions in the next time step andthe action 119886
119894will be leading human-driven vehicle passing
through the potential collision area either in advance of hostvehicle under the intention 119868
119873Yield or behind the host vehicleunder the intention 119868Yield to keep a safe distance 119889safe In thecase with the intention of 119868
119873Yield we can calculate the lowboundary 119886
119894low of 119886119894 through the above process anddeterminethe upper one using the largest comfort value 119886
119894comfort If119886119894comfort lt 119886
119894low 119886119894low will be used as the human-drivenvehiclersquos action If not we consider the targeted 119886
119894following
a normal distribution with mean value 120583119886119894
between 119886119894low and
119886119894comfort To simplify our model we use the mean value ofthese two boundaries to represent human-driven vehiclersquosaction 119886
119894 Similarly the case with the intention of 119868Yield can
be analyzed in the same processAfter these steps the transition probability Pr(1199041015840 | 119904 119886)
is well formulized and the autonomous vehicle could havethe ability to understand the future motion of the scenariothrough this model
46 Observation Model The observation model is built tosimulate the measurement process The motion intentionis updated in this process The measurements of human-driven vehicles are modeled with conditional independentassumption Thus the observation model can be calculatedas
Pr (119911 | 119886 1199041015840) = Pr (119911host | 1199041015840
host)119873
prod
119894=1
Pr (119911119894| 1199041015840
119894) (15)
The host vehiclersquos observation function is denoted as
Pr (119911host | 1199041015840
host) simN (119911host | 1199091015840
host Σ119911host) (16)
But in this paper due to the use of V2V communicationsensor the observation error almost does not affect the deci-sion-making result The variance matrix is set as zero
The human-driven vehiclersquos observation will follow thevehiclersquos motion intentions Because we do not consider
the observation error the value in metric state will be thesame as the state transition results But the longitudinalintention of human-driven vehicles in the state space will beupdated using the new observations andHMMmentioned inSection 3 The new observation space will be confirmed withthe above step
47 Reward Function The candidate policies have to satisfyseveral evaluation criterions Autonomous vehicles should bedriven safely and comfortably At the same time they shouldfollow the traffic rules and reach the destination as soon aspossible As a result we design objective function (17) consid-ering three aspects including safety time efficiency and trafficlaws where 120583
1 1205832 and 120583
3are the weight coefficient
119877 (119904 119886) = 1205831119877safety (119904 119886) + 1205832119877time (119904 119886)
+ 1205833119877law (119904 119886)
(17)
The detailed information will be discussed in the follow-ing subsections In addition the factor of comfort will be con-sidered and discussed in policy generation part (Section 51)
471 Safety Reward The safety reward function 119877safety(119904 119886)is based on the potential conflict status In our strategysafety reward is defined as a penalty If there are no potentialconflicts the safety rewardwill be set as 0 A large penalty willbe assigned due to the risk of collision status
In an uncontrolled intersection the four approachingdirections are defined as 119860
119894isin 1 2 3 4 (Figure 5) The
driverrsquos lateral intentions are defined as 119868lat isin 119868TR 119868TL 119868GS119868S So the driving trajectory for each vehicle in the inter-section can be generally represented by 119860
119894and 119868lat119895 and we
marked it as 119879119860119894119868lat119895
1 le 119894 le 4 1 le 119895 le 4 The function 119865 isused to judge the potential collision status which is denotedas
119865 (119879119909 119879119910) =
1 if potential conflict
0 otherwise(18)
where 119879119909and 119879
119910are vehiclesrsquo maneuver 119879
119860119894119868lat119895
119865(119909 119910) can be calculated through relative direction
between two cars which is shown in Table 1The safety reward is based on the following items(i) If 119865(119909 119910) is equal to 0 then the safety reward is equal
to 0 due to the noncollision status(ii) If potential collision occurs there will be a large
penalty(iii) If |TTC
119894minus TTChost| lt 119905threshod there is a penalty
depending on |TTC119894minus TTChost| and TTChost
472 Traffic Law Reward Autonomous vehicles should fol-low traffic laws to interactwith human-driven vehicles Trafficlaw is modeled as a function Law(119909 119910) for each two vehicles119909 and 119910
Law (119879119909 119879119910) =
1 if 119909 is prior
0 otherwise(19)
8 Mathematical Problems in Engineering
Human-driven vehicle
Autonomous vehicle
1
4
3
2
Figure 5 One typical scenario for calculating safety reward
Table 1 Safe condition judgments in the intersection
Human-driven vehicle Left side Right side Opposite side Same sideDriving direction TL LK TR S TL LK TR S TL LK TR S TL LK TR S
Turn left permil permil I I permil permil I I permil permil permil I I I I ILane keeping permil permil I I permil permil permil I permil I I I I I I ITurn right I permil I I I I I I permil I I I I I I IStop I I I I I I I I I I I I I I I I
permil indicates potential collision I indicates no potential collision
where 119879119909and 119879
119910are vehiclesrsquo maneuver 119879
119860119894119878119895
This functionLaw(119879
119909 119879119910) is formulized as shown in Algorithm 1
If the behavior will break the law a large penalty is appliedand the behavior of obeying traffic laws will get a zero reward
473 Time Reward The time cost is based on the time to thedestination for the targeted vehicles in the intersection area
Costtime =DTGVhost
(20)
DTG is the distance to the driving goal In addition wealso need to consider the speed limit which is discussed inpolicy generation part in Section 5
5 Approximations on SolvingPOMDP Problem
Solving POMDP is quite difficult The complexity of search-ing total brief space is O(|A|119867|Z|
119867
) [12] where 119867 is theprediction horizon In this paper we model the intentionrecognition process as a deterministic model and use com-munication sensors to ignore the perception error and thusthe size of |Z| is reduced to 1 in the simplified problem Tosolve this problem we first generate the suitable potentialpolicies according to the property of driving tasks and then
select the reasonable total predicting interval time and totalhorizon After that the approximate optimal policy canbe calculated through searching all possible policies withmaximum total rewardThe policy selection process is shownin Algorithm 2 and some detailed explanations are discussedin the subsections
51 Policy Generation For autonomous driving near inter-section the desired velocity curves need to satisfy severalconstraints Firstly except for emergency braking the accel-eration constraints are applied to ensure comfort Secondlythe speed limit constraints should be used in this process Weaim to avoid the acceleration commands when autonomousvehicle is reaching maximum speed limit Thirdly for thecomfort purpose the acceleration command should not bealways changed In otherwords we need tominimize the jerk
Similar to [11] the candidate policies are divided intothree time segments The first two segments are like ldquokeepconstant accelerationdeceleration actionsrdquo while keepingconstant velocity in the third segment We use 119905
1 1199052 and 119905
3
to represent the time periods of these three segments Toguarantee comfort the acceleration is limited to the rangefrom minus4ms2 to 2ms2 and we discrete acceleration actioninto a multiple of [minus05 05 0] Then the action space can berepresented by a discretizing acceleration setThenwe can set
Mathematical Problems in Engineering 9
Law(119879119909 119879119910) larr 0
119905119909larr 1198891199092119868V119909
119905119910larr 1198891199102119868V119910
if 119905119909lt 119905119910minus Δ119905 then
Law(119879119909 119879119910) larr 1
else if |119905119909minus 119905119910| lt Δ119905 then
statuslarr 119865(119879119909 119879119910)
if status = 1 thenif 119868lat119909 = lanekeeping and 119868lat119910 ltgt lanekeeping thenLaw(119879
119909 119879119910) larr 1
else if 119868lat119909 119868lat119910 = lanekeeping and 119860119909minus 119860119910= 1 or minus3 then
Law(119879119909 119879119910) larr 1
else if 119868lat119909 = turnleft and 119868lat119910 = turnright thenLaw(119879
119909 119879119910) larr 1
end ifend if
end ifreturn Law(119879
119909 119879119910)
Algorithm 1 Traffic law formulization
InputPredict horizon119867 time step Δ119905Current states 119904host = 119909host 119904human = [119909human 119868human]
(1) 119875 larr genenratepolicyset()(2) for each 120587
119896isin 119875 do
(3) for 119894 = 1 to119867Δ119905 do(4) 119886host larr 120587
119896(119894)
(5) 1199041015840
host larr updatestate(119904host 119886host)(6) 119886human larr predictactions(1199041015840host 119904human 119868human)
(7) 1199091015840
human larr updatestate(119909human 119886human)
(8) 1198681015840
human larr updateintention(1199041015840host 1199091015840
human)
(9) 1199041015840
human larr [1199091015840
human 1198681015840
human]
(10) 119877(119894) = calculatereward(1199041015840host 1199041015840
human)
(11) 119904host larr 1199041015840
host(12) 119904human larr 119904
1015840
human(13) end(14) 119877
total119896
larr sum119894sum(119877(119894))
(15) end(16) 119896lowast larr argmax
119896(119877
total119896
)
(17) 120587lowast larr 120587119896lowast
(18) return 120587lowast
Algorithm 2 Policy selection process
the value of 1199051 1199052 and 119905
3and the prediction period of single
step An example of policy generation is shown in Figure 6
52 Planning Horizon Selection After building policy gener-ation model the next problem is to select a suitable planninghorizon Longer horizon can lead to a better solution but con-suming more computing resources However as our purpose
is to deal with the interaction problem in the uncontrolledintersection we only need to consider the situation beforeautonomous vehicle gets through In our algorithmwe set theprediction horizon as 8 seconds In addition in the process ofupdating the future state of each vehicle using each policy thecar following mode is used after autonomous vehicle passesthrough the intersection area
10 Mathematical Problems in Engineering
0
1
2Candidate policies
minus4
minus3
minus2
minus1
Acce
lera
tion
(ms2)
1 2 3 4 5 6 7 80Time (s)
(a)
Time (s)0 1 2 3 4 5 6 7 8
0
2
4
6
8
10
12
14
16
18
20
Velo
city
(ms
)
Velocity strategies
(b)
Figure 6 An example of policy generation process (a) is the generated policies and (b) is the corresponding speed profiles The interval ofeach prediction step is 05 s current speed is 12ms2 and the speed limit is 20ms2 The bold black line is one policy In the first 3 secondsautonomous vehicles decelerate in minus35ms2 then accelerate at 2ms2 for 4 seconds and finally stop in the last one second In this case 109policies were generated which is suitable for replanning fast
AB
Figure 7 Testing scenario Autonomous vehicle B and human-driven vehicle A are both approaching the uncontrolled intersection To goacross the intersection successfully autonomous vehicle should interact with human-driven vehicle
53 Time Step Selection Another problem is the predic-tion time step The intention prediction algorithm and thePOMDP are computed in each step If the time step is 119905stepthe total computation timeswill be119867119905stepThus smaller timestep leads to more computation time To solve this problemwe use a simple adaptive time step calculation mechanism togive a final value The time step is selected based on the TTCof autonomous vehicle If the host vehicle is far away from theintersection we can use a very large time step But if the TTCis quite small the low 119905step is applied to ensure safety
6 Experiment and Results
61 Settings In this paper we evaluate our approach throughPreScan 710 [21] a simulation tool for autonomous drivingand connected vehicles Using this software we can build thetesting scenarios (Figure 7) and add vehicles with dynamicmodel In order to get a similar scenario considering social
interaction the driver simulator is added in our experiment(Figure 8) The human-driven vehicle is driven by severalpeople during the experiment and the autonomous vehiclemakes decisions based on the human-driven vehiclersquos drivingbehavior The reference trajectory for autonomous vehicleis generated from path planning module and the human-driven vehiclersquos data (eg position velocity and heading)are transferred through V2V communication sensor Thedecision-making module sends desired velocity command tothe PID controlled to follow the reference path All policies inthe experiment part use a planning horizon119867 = 8 s which isdiscretized into the time step of 05 s
62 Results It is difficult to compare different approachesin the same scenario because the environment is dynamicand not exactly the same However we select two typicalsituations and special settings to make it possible The sameinitial conditions including position orientation and velocity
Mathematical Problems in Engineering 11
Figure 8 Logitech G27 driving simulator
T = 0 1 2 6 s T = 5 s
T = 4 s
T = 3 s
T = 2 s
T = 1 s
T = 0 s
T = 6 s
(a)
T = 0 1 2 9 s
T = 4 s
T = 5 s
T = 6 sT = 7 sT = 8 s
T = 9 s
T = 3 s
T = 2 s
T = 1 s
T = 0 s
(b)
Figure 9 The visualized passing sequence (a) is the result of our approach and (b) represents the result of reactive approach withoutconsidering intention The black vehicle is an autonomous vehicle while the red car is the human-driven vehicle Each vehicle representsthe position in a specific time 119879 with an interval of 1 second
for each vehicle are used in different tests Besides twotypical situations including human-driven vehicle gettingthrough before or after autonomous vehicle are comparedin this section With the same initial state different reac-tions will occur based on various methods We compareour approach and reactive-based method [6] in this sec-tion The key difference for these two methods is that ourapproach considers human-driven vehiclersquos driving inten-tion
The first experiment is that human-driven vehicle triesto yield autonomous vehicle in the interaction process Theresults are shown in Figures 9 and 10 Firstly Figure 9 givesus a visual comparison of the different approaches Fromalmost the same initial state (eg position and velocity) ourapproach could lead to autonomous vehicle passing throughthe intersection more quickly and reasonable
Then let us look at Figure 10 for detailed explanationIn the first 12 s in Figures 10(a) and 10(c) autonomousvehicle maintains speed and understands that human-drivenvehicle will not perform yielding actions Then autonomousvehicle gets yielding intention of human-driven vehicle andunderstands that human-driven vehiclersquos lateral intentionis to go straight Based on candidate policies autonomousvehicle selects acceleration strategy with maximum rewardand finally crosses the intersection In this process we canobviously find that autonomous vehicle understands human-driven vehiclersquos yielding intention Figure 10(c) is an exampleof understand human-driven vehiclersquos behavior based on egovehiclersquos future actions in a specific time Our strategy pre-dicts the future actions of human-driven vehicle Althoughthe velocity curves after 1 s do not correspond it does notaffect the performance of our methods The reason is that
12 Mathematical Problems in Engineering
0 1 2 3 4 5 6 7 80
1020
Time (s)
Velo
city
(ms
) Velocity profile
0 1 2 3 4 5 6 7 8024
Time (s)
Inte
ntio
n Longitudinal intention
0 1 2 3 4 5 6 7 8024
Time (s)
Inte
ntio
n Lateral intention
(a)
0 1 2 3 4 5 6 7 8 90
1020
Time (s)
Velo
city
(ms
) Velocity profile
0 1 2 3 4 5 6 7 8 9024
Time (s)
Inte
ntio
n
Longitudinal intention
0 1 2 3 4 5 6 7 8 9024
Time (s)
Inte
ntio
n
Lateral intention
(b)
0 1 2 3 4 5 6 7 8minus60
minus40
minus20
0
20
40
60
80
Time (s)
Distance to collision region
Dist
ance
(m)
(c)
0 1 2 3 4 5 6 7 8 9minus10
0
10
20
30
40
50
60
70
80
Time (s)
Dist
ance
(m)
Distance to collision region
(d)
0 1 2 3 4 5 6 7 87
8
9
10
11
12
13
14
15
Time (s)
Velo
city
(ms
)
(e)
Time (s)0 1 2 3 4 5 6 7 8
0
2
4
6
8
10
12
14
Velo
city
(ms
)
(f)
Figure 10 Case test 1 In this case human-driven vehicle passes through intersection after autonomous vehicle (a) (c) and (e) are theperformance of our method while (b) (d) and (f) are from the strategy without considering the driving intention (a) and (b) are the velocityprofiles and the corresponding driving intention For longitudinal intention label 1 means yielding and label 2 means not yielding In lateralintention 1 means turning left 2 means turning right 3 means going straight and 4 means stop The intentions in (b) are not used in thatmethod but for detailed analysis (c) and (d) are the distance to collision area for autonomous vehicle and human-driven vehicle respectively(e) and (f) are the prediction and true motions of human-driven vehicles in time 15 s with a prediction length of 8 s The red curves in thesesubfigures are from autonomous vehicle while blue lines are from human-driven vehicle The green lines in (e) and (f) are the predictionvelocity curves of human-driven vehicle
Mathematical Problems in Engineering 13
T = 4 s
T = 1 s
T = 2 s
T = 3 s
T = 5 s
T = 0 s
T = 0 1 2 5 s
(a)
T = 4 s
T = 1 s
T = 2 s
T = 3 s
T = 5 s
T = 0 s
T = 0 1 2 5 s
(b)
Figure 11 The visualized passing sequence for the case of human-driven vehicle first getting through (a) is the result of our approach and(b) is the reactive-based approach
we use a deterministic model in the prediction process andthe prediction value is inside two boundaries to ensure safetyBesides the whole actions of autonomous vehicle in thisprocess could also help human-driven vehicle to understandnot-yielding intention of autonomous vehicles In this casecooperative driving behaviors are performed by both vehi-cles
However if the intention is not considered in this processwe can find the results in Figures 10(b) 10(d) and 10(f)After 2 s in Figure 10(b) while the human-driven vehiclegives a yielding intention autonomous vehicle could notunderstand and they find a potential collision based on theconstant velocity assumptions Then it decreases the speedbut the human-driven vehicle also slows down The puzzledbehavior leads both vehicles to slow down near intersectionFinally human-driven vehicle stops at the stop line and thenautonomous vehicle could pass the intersection In this strat-egy the human-driven vehiclersquos future motion is assumed tobe constant (Figure 10(f))Without understanding of human-driven vehiclersquos intentions this strategy can increase conges-tion problem
Another experiment is that human-driven vehicle tries toget through the intersection first The results are shown inFigures 11 and 12This case is quite typical because many traf-fic accidents in real world are happening in this situation Indetail if one vehicle tries to cross an intersection while violat-ing the law another vehicle will be in great danger if it doesnot understand its behavior From the visualized perform-ance in Figure 11 our method is a little more safe than otherapproaches as there is nearly collision situation in Figure11(b) In detail we can see from Figure 12(a) that our strategycould perform deceleration actions after we understand thenot-yielding intention in 08 sHowever without understand-ing human-driven vehiclersquos motion intention the responsetime has a 1-second delay which may be quite dangerous
In addition it is shown that good performance is in thepredictions of human-driven vehiclersquos future motion in ourmethods (Figure 12(e))
The results of these two cases demonstrate that our algo-rithm could deal with typical scenarios and have better per-formance than traditional reactive controller Autonomousvehicle could be driven more safely fast and comfortablythrough our strategy
7 Conclusion and Future Work
In this paper we proposed an autonomous driving decision-making algorithm considering human-driven vehiclersquos uncer-tain intentions in an uncontrolled intersection The lat-eral and longitudinal intentions are recognized by a con-tinuous HMM Based on HMM and POMDP we modelgeneral decision-making process and then use an approxi-mate approach to solve this complex problem Finally weuse PreScan software and a driving simulator to emulatesocial interaction process The experiment results show thatautonomous vehicles with our approach can pass throughuncontrolled intersections more safely and efficiently thanusing the strategy without considering human-driven vehi-clesrsquo driving intentions
In the near future we aim to implement our approach intoa real autonomous vehicle and perform real world experi-ments In additionmore precious intention recognition algo-rithm aims to be figured out Somemethods like probabilisticgraphic model can be used to get a distribution of each inten-tion Finally designing online POMDP planning algorithmsis also valuable
Competing Interests
The authors declare that there are no competing interestsregarding the publication of this paper
14 Mathematical Problems in Engineering
0 05 1 15 2 25 3 35 4 45 510
20
30
Time (s)
Velo
city
(ms
) Velocity profile
0 05 1 15 2 25 3 35 4 45 50
2
4
Time (s)
Inte
ntio
n
Longitudinal intention
0 05 1 15 2 25 3 35 4 45 5024
Inte
ntio
n Lateral intention
Time (s)
(a)
0 05 1 15 2 25 3 35 4 45 5102030
Time (s)
Velo
city
(ms
) Velocity profile
0 05 1 15 2 25 3 35 4 45 5024
Time (s)
Inte
ntio
n
Longitudinal intention
0 05 1 15 2 25 3 35 4 45 5024
Time (s)
Inte
ntio
n
Lateral intention
(b)
0 05 1 15 2 25 3 35 4 45 5
0
20
40
60
80
Dist
ance
(m)
Distance to collision region
Time (s)
minus40
minus20
(c)
0 05 1 15 2 25 3 35 4 45 5
0
20
40
60
80D
istan
ce (m
)
Distance to collision region
Time (s)
minus40
minus20
(d)
0 1 2 3 4 5 6 7 816
17
18
19
20
21
22
23
24
25
Velo
city
(ms
)
Time (s)
(e)
0 1 2 3 4 5 6 7 817
18
19
20
21
22
23
24
25
Time (s)
Velo
city
(ms
)
(f)
Figure 12 Case test 2 In this case human-driven vehicle passes through intersection before autonomous vehicle through different strategiesThe definition of each subfigure is the same as in Figure 10
Mathematical Problems in Engineering 15
Acknowledgments
This study is supported by the National Natural ScienceFoundation of China (no 91420203)
References
[1] C Urmson J Anhalt D Bagnell et al ldquoAutonomous driving inurban environments boss and the urban challengerdquo Journal ofField Robotics vol 25 no 8 pp 425ndash466 2008
[2] J Markoff ldquoGoogle cars drive themselves in trafficrdquo New YorkTimes vol 9 2010
[3] L R Rabiner and B-H Juang ldquoAn introduction to hiddenMarkov modelsrdquo IEEE ASSP Magazine vol 3 no 1 pp 4ndash161986
[4] M Buehler K Iagnemma and S Singh The DARPA UrbanChallenge Autonomous Vehicles in City Traffic vol 56 Springer2009
[5] S Kammel J Ziegler B Pitzer et al ldquoTeam AnnieWAYrsquos auto-nomous system for the 2007 DARPAUrban Challengerdquo Journalof Field Robotics vol 25 no 9 pp 615ndash639 2008
[6] C R Baker and J M Dolan ldquoTraffic interaction in the urbanchallenge putting boss on its best behaviorrdquo in Proceedingsof the IEEERSJ International Conference on Intelligent Robotsand Systems (IROS rsquo08) pp 1752ndash1758 IEEE Nice FranceSeptember 2008
[7] M Montemerlo J Becker S Shat et al ldquoJunior the Stanfordentry in the urban challengerdquo Journal of Field Robotics vol 25no 9 pp 569ndash597 2008
[8] L Fletcher S Teller E Olson et al ldquoTheMIT-Cornell collisionand why it happenedrdquo Journal of Field Robotics vol 25 no 10pp 775ndash807 2008
[9] M Bahram A Wolf M Aeberhard and D Wollherr ldquoAprediction-based reactive driving strategy for highly automateddriving function on freewaysrdquo in Proceedings of the 25th IEEEIntelligent Vehicles Symposium pp 400ndash406 IEEE DearbornMich USA June 2014
[10] J Wei J M Dolan and B Litkouhi ldquoA prediction- and costfunction-based algorithm for robust autonomous freeway driv-ingrdquo in Proceedings of the IEEE Intelligent Vehicles Symposium(IV rsquo10) pp 512ndash517 San Diego Calif USA June 2010
[11] J Wei J M Dolan and B Litkouhi ldquoAutonomous vehiclesocial behavior for highway entrance ramp managementrdquo inProceedings of the IEEE Intelligent Vehicles Symposium (IV rsquo13)pp 201ndash207 IEEE Gold Coast Australia June 2013
[12] H Bai S Cai N Ye D Hsu and W S Lee ldquoIntention-awareonline POMDP planning for autonomous driving in a crowdrdquoin Proceedings of the IEEE International Conference on Roboticsand Automation (ICRA rsquo15) pp 454ndash460 IEEE Seattle WashUSA May 2015
[13] A Somani N Ye D Hsu and W S Lee ldquoDespot onlinepomdp planning with regularizationrdquo in Advances in NeuralInformation Processing Systems pp 1772ndash1780 2013
[14] S Brechtel T Gindele and R Dillmann ldquoProbabilistic deci-sion-making under uncertainty for autonomous driving usingcontinuous POMDPsrdquo in Proceedings of the 17th IEEE Interna-tional Conference on Intelligent Transportation Systems (ITSCrsquo14) pp 392ndash399 Qingdao China October 2014
[15] C H Papadimitriou and J N Tsitsiklis ldquoThe complexity ofMarkov decision processesrdquo Mathematics of Operations Re-search vol 12 no 3 pp 441ndash450 1987
[16] O Madani S Hanks and A Condon ldquoOn the undecidabilityof probabilistic planning and related stochastic optimizationproblemsrdquoArtificial Intelligence vol 147 no 1-2 pp 5ndash34 2003
[17] S Ulbrich and M Maurer ldquoProbabilistic online POMDP deci-sion making for lane changes in fully automated drivingrdquo inProceedings of the 16th International IEEE Conference on Intelli-gent Transportation Systems (ITSC rsquo13) pp 2063ndash2067 TheHague The Netherlands October 2013
[18] A G Cunningham E Galceran R M Eustice and E OlsonldquoMPDM multipolicy decision-making in dynamic uncertainenvironments for autonomous drivingrdquo in Proceedings of theIEEE International Conference on Robotics and Automation(ICRA rsquo15) pp 1670ndash1677 Seattle Wash USA May 2015
[19] D A Reynolds and R C Rose ldquoRobust text-independentspeaker identification using Gaussian mixture speaker modelsrdquoIEEE Transactions on Speech and Audio Processing vol 3 no 1pp 72ndash83 1995
[20] L E Baum and T Petrie ldquoStatistical inference for probabilisticfunctions of finite state Markov chainsrdquo The Annals of Mathe-matical Statistics vol 37 pp 1554ndash1563 1966
[21] M Tideman and M Van Noort ldquoA simulation tool suite fordeveloping connected vehicle systemsrdquo in Proceedings of theIEEE Intelligent Vehicles Symposium (IEEE IV rsquo13) pp 713ndash718Queensland Australia June 2013
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Mathematical Problems in Engineering 3
used a rule-based behavior generation mechanism to obeythe predefined driving rules based on the obstacle vehiclesrsquometrics [1 6] Boss was able to check vehiclersquos accelerationabilities and the spaces to decide whether merging into a newlane or passing intersections is safe Similarly the decision-making system of ldquoJuniorrdquo [7] ranking second in DUC wasbased on a HSM with manually defined 13 states Due to theadvantages including implementing simply and traceabilitythis framework is widely used in many autonomous drivingplatforms However these approaches always use constantvelocity assumptions and lack considering surrounding vehi-cles future reactions to host vehiclersquos actions Without thisability the driving decisions could have potential risks [8]
In order to consider the evolution of future scenario theplanning and utility-based approaches have been proposedfor decision-making Bahram et al proposed a predictionbased reactive strategy to generate autonomous drivingstrategies [9] A Bayesian classifier is used to predict thefuture motion of obstacle vehicles and a tree-based searchingmechanism is designed to find the optimal driving strategyusing multilevel cost functions However the surroundingvehiclesrsquo reactions to autonomous vehiclesrsquo actions are notconsidered in their frameworkWei et al proposed a compre-hensive approach for autonomous driver model by emulatinghuman driving behavior [10] The human-driven vehicles areassumed to follow a proper social behavior model and thebest velocity profiles are generated in autonomous freewaydriving applications Nonetheless their method does notconsider the motion intention of human-driven vehicles andonly targets in-lane driving In their subsequent work Weiet al modeled traffic interactions and realized autonomousvehicle social behavior in highway entrance ramp [11] Thehuman-driven vehiclesrsquo motion intentions are modeled bya Bayesian model and the human-driven vehiclesrsquo futurereactions are introduced which is based on the yieldingnot-yielding intentions at the first prediction step Autonomousvehicles could perform social cooperative behavior usingtheir frameworkHowever they do not consider the intentionuncertainty over prediction time step
POMDPs provide a mathematical framework for solv-ing the decision-making problem with uncertainties Bai etal proposed an intention-aware approach for autonomousdriving in scenarios with many pedestrians (eg in campus)[12] In their framework the hybrid 119860lowast algorithm is usedto generate global path while a POMDP planner is usedto control the velocity of the autonomous vehicle solvingby an online POMDP solver DESPOT [13] Brechtel et alpresented a probabilistic decision-making algorithm usingcontinuous POMDP [14] They focus on dealing with theuncertainties of incomplete and inaccurate perception in theintersection area while our goal is to deal with the uncertainintentions of human-driven vehicles However the onlinePOMDP solver always needs large computation resource andconsumes much time [15 16] which limits its use in realworld autonomous driving platform Ulbrich and Maurerdesigned a two-step decision-making algorithm to reducethe complexity of the POMDP in lane change scenario [17]Eight POMDP states are manually defined to simplify theproblem in their framework Cunningham et al proposed a
multipolicy decision-making method in lane changing andmerging scenarios [18] POMDPs are used to model thedecision-making problem in their paper while multivehiclesimulation mechanism is used to generate the optimal high-level policy for autonomous vehicle to execute However themotion intentions are not considered
Overall the autonomous driving decision-making prob-lem with uncertain driving intention is still a challengingproblem It is necessary to build an effective behavior predic-tion model for human-driven vehicles Besides it is essentialto incorporate human-driven vehiclesrsquo intentions and behav-iors into autonomous vehicle decision-making system andgenerate suitable actions to ensure autonomous vehicles drivesafely and efficiently This work addresses this problem byfirst building aHMM-based intention predictionmodel thenmodeling human-driven vehiclersquos intentions in a POMDPframework and finally solving it in an approximate method
3 HMM-Based Intention Prediction
In order to pass through an uncontrolled intersectionautonomous vehicles should have the ability to predict thedriving intentions of human-driven vehicles Estimatingdriverrsquos behavior is very difficult because the state of a vehicledriver is in some high-dimensional feature space Instead ofusing driver related features (eg gas pedal brake pedal anddriversrsquo vision) easily observed parameters are used to buildthe intention prediction model in this paper
The vehicle motion intention 119868 considered in thispaper is divided into two aspects lateral intention 119868lat isin
119868TR 119868TL 119868GS 119868S (ie turn right turn left go straight andstop) and longitudinal intention 119868lon isin 119868Yield 119868119873Yield Thelateral intention is a high-level driving maneuver which isdetermined by human driversrsquo long term decision-makingprocess This intention is not always changed in the drivingprocess and determines the future trajectory of human-driven vehicles In particular the intention of stop is treatedas a lateral intention in our model because it can be predictedonly using data from human-driven vehicle itself Howeverthe longitudinal intention is a cooperative behavior onlyoccurring when it interacts with other vehicles We willfirst describe the HMM and then formulize our intentionprediction model in this section
31 HMM A HMM consists of a set of 119873 finite ldquohiddenrdquostates and a set of119872 observable symbols per state The statetransition probabilities are defined as Α = 119886
119894119895 where
119886119894119895= 119875 [119902
119905+1= 119895 | 119902
119905= 119894] 1 le 119894 119895 le 119873 (1)
The initial state distribution is denoted as 120587 = 120587119894 where
120587119894= 119875 [119902
1= 119894] 1 le 119894 le 119873 (2)
Because the observation symbols are continuous param-eters we use Gaussian Mixture Model (GMM) [19] torepresent their probability distribution functions (pdf)
119887119895(o) =
119872
sum
119896=1
119888119895119896N (o | 120583
119895119896 Σ119895119896) 1 le 119895 le 119873 (3)
4 Mathematical Problems in Engineering
where 119888119895119896
represents the mixture coefficient in the 119895th statefor the 119896th mixture N is the pdf of a Gaussian distributionwith mean 120583 and covariance Σmeasured from observation oMixture coefficient 119888 satisfies the following constraints
119872
sum
119896=1
119888119895119896= 1 (4)
where 119888119895119896gt 0 1 le 119895 le 119873 1 le 119896 le 119872
And
int
+infin
minusinfin
119887119895(o) 119889o = 1 1 le 119895 le 119873 (5)
Then a HMM could be completely defined by hiddenstates119873 and the probability tuples 120582 = (120587 119860 119862 120583 Σ)
In the training process we use the Baum-Welch method[20] to estimate model parameters for different driver inten-tion 119868 Once themodel parameters corresponding to differentdriver intention have been trained we can perform thedriverrsquos intention estimation in the recognition process Theprediction process for lateral intentions can be seen inFigure 2
32 HMM-Based Intention Prediction Process Given a con-tinuous HMM the intention prediction process is dividedinto two steps The first step focused on the lateral intentionThe training inputs of each vehiclersquos lateral intention modelin time 119905 are defined as 119861lateral = 119871 V 119886 yaw where 119871 isthe distance to the intersection V is the longitudinal velocity119886 is the longitudinal acceleration and yaw is the yaw ratewhile the output of this model is the motion intentions 119868lat isin119868TR 119868TL 119868GS 119868S The corresponding HMMs can be trainedincluding 120582TR 120582TL 120582GS and 120582119878
The next step is about longitudinal intention This prob-ability could be decomposed based on the total probabilityformula
119875 (119868Yield | 119861) = sum119875 (119868Yield | 119868lat 119861) 119875 (119868lat | 119861)
= 119875 (119868Yield | 119868TR 119861) 119875 (119868TR | 119861)
+ 119875 (119868Yield | 119868TL 119861) 119875 (119868TL)
+ 119875 (119868Yield | 119868GS 119861) 119875 (119868GS | 119861)
+ 119875 (119868Yield | 119868S 119861) 119875 (119868S | 119861)
(6)
where 119861 is the behavior data including 119861lateral and 119861lonIn this process we assume that the lateral behavior 119868lat is
predicted correctly by a deterministic HMM in the first stepand therefore 119868lat is determined by the lateral prediction result119868latPredict where 119875(119868lat | 119861 119868lat = 119868latPredict) = 1 and 119875(119868lat | 119861119868lat = 119868latPredict) = 0 And (6) is reformulated by
119875 (119868Yield | 119861) = 119875 (119868Yield | 119861 119868latPredict) 119875 (119868latPredict | 119861)
= 119875 (119868Yield | 119861 119868latPredict) (7)
The problem is changed to model 119875(119868Yield | 119861 119868latPredict)The features used in longitudinal intention prediction are
119861lon = ΔV Δ119886 ΔDTC where ΔV = Vsocial minus Vhost Δ119886 =
119886social minus 119886host and ΔDTC = DTCsocial minusDTChost DTC meansthe distance to the potential collision area The output ofthe longitudinal intention prediction model is longitudinalmotion intention 119868lon isin 119868Yield 119868119873Yield
Instead of building a generative model we use a deter-ministic approach to restrict 119875(119868Yield | 119861 119868latPredict) as 0 or1 Thus two types of HMMs named 120582
119884119868lat 120582119873119868lat
are trainedwhere 119868lat isin 119868TR 119868TL 119868GS 119868S Two test examples for lateraland longitudinal intention prediction are shown in Figures3 and 4 Through these two figures we can find that ourapproach can recognize human-driven vehiclersquos lateral andlongitudinal intention successfully
4 Modeling Autonomous DrivingDecision-Making in a POMDP Framework
For the decision-making process the key problem is howto design a policy to perform the optimal actions withuncertainties This needs to not only obtain traffic lawsbut also consider the driving uncertainties of human-drivenvehicles Facing potential conflicts human-driven vehicleshave uncertain probabilities to yield autonomous vehicles andsome aggressive drivers may violate the traffic laws Suchelements should be implemented into a powerful decision-making framework As a result we model autonomousdriving decision-making problem in a general POMDPframework in this section
41 POMDP Preliminaries A POMDP model can be for-mulized as a tupleSA 119879Z 119874 119877 120574 where S is a setof states A is the action space and Z denotes a set ofobservations The conditional function 119879(1199041015840 119886 119904) = Pr(1199041015840 |119904 119886) models transition probabilities to state 1199041015840 isin S whenthe system takes an action 119886 isin A in the state 119904 isin S Theobservation function 119874(119911 1199041015840 119886) = Pr(119911 | 119904
1015840
119886) models theprobability of observing 119911 isin Z when an action 119886 isin A istaken and the end state is 1199041015840 isin S The reward function 119877(119904 119886)calculates an immediate reward when taking an action 119886 instate 119904 120574 isin [0 1] is the discount factor in order to balance theimmediate and the future rewards
Because the system contains partially observed state suchas intentions a belief 119887 isin B is maintained A belief updatefunction 120591 is defined as 1198871015840 = 120591(119887 119886 119911) If the agent takes action119886 and gets observation 119911 the new belief 1198871015840 is obtained throughthe Bayesrsquo rule
1198871015840
(1199041015840
) = 120578119874 (1199041015840
119886 119911)sum
119904isin119878
119879 (119904 119886 1199041015840
) 119887 (119904) (8)
where 120578 = 1sum1199041015840isin119878119874(1199041015840
119886 119911)sum119904isin119878119879(119904 119886 119904
1015840
)119887(119904) is a normal-izing constant
A key concept in POMDP planning is a policy a mapping120587 that specifies the action 119886 = 120587(119887) at belief 119887 To solvethe POMDP an optimal policy 120587lowast should be designed tomaximize the total reward
120587lowast
= argmax120587
(119864(
infin
sum
119905=0
120574119905
119877 (119904119905 120587 (119887119905)) | 1198870 120587)) (9)
where 1198870is marked as the initial belief
Mathematical Problems in Engineering 5
Distance tointersection
Velocity
Acceleration
Yaw rate
Observe sequence
HMM_TR Forward algorithm
HMM_TL Forward algorithm
HMM_S Forward algorithm
HMM_GS Forward algorithm
DriverintentionO = B1 B2 BM
1205821 = (1205871 A1 C1 1205831 Σ1)
1205822 = (1205872 A2 C2 1205832 Σ2)
1205823 = (1205873 A3 C3 1205833 Σ4)
1205824 = (1205874 A4 C4 1205834 Σ4)
P(O | 1205822)
P(O | 1205821)
P(O | 1205824)
P(O | 1205824)
arg max P(O | 120582i)i
Figure 2 Prediction process for HMMThe observed sequence will be evaluated by four HMMs Forward algorithm is used to calculate theconditional probabilities and the intention corresponding to the largest value will be considered as the vehiclersquos intention
Time (s)1 2 3 4 5 6
024
0
Predict results
(a)
01 2 3 4 5 60
1020
Velocity
v (m
s)
Time (s)
(b)
0
1 2 3 4 5 60
10
minus10
Acceleration
a (m
s2)
Time (s)
(c)
Time (s)0 1 2 3 4 5 6
minus30minus20minus10
010
Yaw rate
Yaw
rate
(deg
s)
(d)
0 1 2 3 4 5 60
1020
Time (s)
Distance to intersection (DTI)
DTI
(m)
(e)
Figure 3 Lateral intention prediction example The true intention of human-driven vehicle is to turn left in this scenario In the first figurethe value 1 of the 119910 label means turn left 2 means turn right 3 represents go straight and 4 corresponds to stop
42 State Space Because of the Markov property sufficientinformation should be contained in the state space S fordecision-making process [14] The state space includes thevehicle pose [119909 119910 120579] velocity V the average yaw rate yawaveand acceleration 119886ave in the last planning period for allthe vehicles For the human-driven vehicles the lateral andlongitudinal intentions [119868lat 119868lon] also need to be containedfor state transition modeling However the road contextknowledge is static reference information so that it will be notadded to the state space
The joint state 119904 isin S could be denoted as 119904 = [119904host 1199041
1199042 119904
119873]119879 where 119904host is the state of host vehicle (auto-
nomous vehicle) 119904119894 119894 isin 1 2 3 119873 is the state of
human-driven vehicles and 119873 is the number of human-driven vehicles involved Let us define metric state 119909 =
[119909 119910 120579 V 119886ave yawave]119879 including the vehicle position head-
ing velocity acceleration and yaw rateThus the state of hostvehicle can be defined as 119904host = 119909host while the human-driven vehicle state 119904
119894is 119904119894= [119909
119894 119868lat119894 119868lon119894]
119879 With theadvanced perception system and V2V communication tech-nology we assume that the metric state 119909 could be observedBecause the sensor noise is small and hardly affects decision-making process we do not model observation noise for themetric state However the intention state cannot be directlyobserved so it is the partially observable variables in ourpaper The intention state should be inferred from obser-vation data and predictive model over time
43 Action Space In our autonomous vehicle system thedecision-making system is used to select the suitable tactical
6 Mathematical Problems in Engineering
Time (s)0 1 2 3 4 5 6 7 8
012
Predict results
(a)
0 1 2 3 4 5 6 7 80
1020
Relative velocity (RV)
RV (m
s)
Time (s)
(b)
0 1 2 3 4 5 6 7 8minus2
02
Relative acceleration (RA)
RA (m
s2)
Time (s)
(c)
0 1 2 3 4 5 6 7
02040
Time (s)
minus20
Relative DTC (RDTC)
RDTC
(m)
(d)
Figure 4 One example of predicting longitudinal intentions This example is based on the scenario of Figure 1 and two vehicles both gostraightThe value 1 of 119910-axis in the first figure denotes the intention of yielding while 2 represents not yielding In the first 28 s the intentionis yielding After that due to the acceleration action and less relative DTC autonomous vehicle could understand human-driven vehiclersquosnot-yielding intention
maneuvers Specifically in the intersection area autonomousvehicles should follow a global reference path generated bypath planning module The decision-making module onlyneeds to generate accelerationdeceleration commands to thecontrol layer As the reference path may not be straight thesteering control module can adjust the front wheel angleto follow the reference path Therefore the action space Acould be defined as a discrete set A = [acc dec con] whichcontains commands including acceleration deceleration andmaintaining current velocity
44 Observation Space Similar to the joint state space theobservation 119911 is denoted as 119911 = [119911host 1199111 1199112 119911119873]
119879 where119911host and 119911119894 are the host vehicle and human-driven vehiclersquosobservations respectively The acceleration and yaw rate canbe approximately calculated by speed and heading in theconsecutive states
45 State Transition Model In state transition process weneed to model transition probability Pr(1199041015840 | 119904 119886) Thisprobability is determined by each targeted element in thescenario So the transition model can be calculated by thefollowing probabilistic equation
Pr (1199041015840 | 119904 119886) = Pr (1199041015840host | 119904host 119886host)119873
prod
119894=1
Pr (1199041015840119894| 119904119894) (10)
In the decision-making layer we do not need to considercomplex vehicle dynamic model Thus the host vehiclersquosmotion Pr(1199041015840host | 119904host 119886host) can be simply represented by thefollowing equations given action 119886
1199091015840
= 119909 + (V +119886Δ119905
2)Δ119905 cos (120579 + Δ120579)
1199101015840
= 119910 + (V +119886Δ119905
2)Δ119905 sin (120579 + Δ120579)
1205791015840
= 120579 + Δ120579
V1015840 = V + 119886Δ119905
yaw1015840ave =Δ120579
Δ119905
1198861015840
ave = 119886
(11)
Thus the key problem is converted to compute Pr(1199041015840119894|
119904119894) the state transition probability of human-driven vehicles
Based on the total probability formula this probability can befactorized as a sum in whole action space
Pr (1199041015840119894| 119904119894) = sum
119886119894
Pr (1199041015840119894| 119904119894 119886119894)Pr (119886
119894| 119904119894) (12)
With this equation we only need to calculate the statetransition probability Pr(1199041015840
119894| 119904119894 119886119894) given a specific action 119886
119894
and the probability of selecting this action Pr(119886119894| 119904119894) under
current state 119904119894
Because the human-driven vehiclesrsquo state 119904119894= [119909119894 119868119894] the
probability Pr(1199041015840119894| 119904119894 119886119894) can be calculated as
Pr (1199041015840119894| 119904119894 119886119894) = Pr (1199091015840
119894 1198681015840
119894| 119909119894 119868119894 119886119894)
= Pr (1199091015840119894| 119909119894 119868119894 119886119894)Pr (1198681015840
119894| 1199091015840
119894 119909119894 119868119894 119886119894)
(13)
With a certain action 119886119894 Pr(1199091015840119894| 119909119894 119868119894 119886119894) is equal to Pr(1199091015840
119894|
119909119894 119868lat119894 119886119894)The lateral behavior 119868lat119894 is considered to be a goal-
directed driving intention which will not be changed in thedriving process So Pr(1199091015840
119894| 119909119894 119868lat119894 119886119894) is equal to Pr(119909
1015840
119894| 119909119894 119886119894)
given a reference path corresponding to the intention of 119868lat119894Using (11) Pr(1199091015840
119894| 119909119894 119886119894) can be well solved
The remaining problem for calculating Pr(1199041015840119894| 119904119894 119886119894) is
to deal with Pr(1198681015840119894| 1199091015840
119894 119909119894 119868119894 119886119894) The lateral intention 1198681015840lat119894
is assumed stable through the above explanation And the
Mathematical Problems in Engineering 7
longitudinal intention 1198681015840
lon119894 is assumed to be not updatedin this process But it will be updated with new inputs inobservation space
Now Pr(1199041015840119894| 119904119894 119886119894) is well modeled and the remaining
problem is to compute the probabilities Pr(119886119894| 119904119894) of human-
driven vehiclesrsquo future actions
Pr (119886119894| 119904119894) = Pr (119886
119894| 119909119894 119868119894)
= sum
1199091015840
host
Pr (119886119894| 1199091015840
host 119909119894 119868119894)Pr (1199091015840
host | 119909119894 119868119894) (14)
Because 1199091015840
host is determined by the designed policyPr(1199091015840host | 119909119894 119868119894) could be calculated by (11) given an action119886host The probability Pr(119886
119894| 1199091015840
host 119909119894 119868119894) means the distribu-tion of human-driven vehiclesrsquo actions given the new state1199091015840
host of host vehicle the current state of itself and itsintentions Instead of building a complex probability modelwe designed a deterministic mechanism to calculate themostpossible action 119886
119894given 1199091015840host 119909119894 and 119868119894
In this prediction process the host vehicle is assumed tobe maintaining the current actions in the next time step andthe action 119886
119894will be leading human-driven vehicle passing
through the potential collision area either in advance of hostvehicle under the intention 119868
119873Yield or behind the host vehicleunder the intention 119868Yield to keep a safe distance 119889safe In thecase with the intention of 119868
119873Yield we can calculate the lowboundary 119886
119894low of 119886119894 through the above process anddeterminethe upper one using the largest comfort value 119886
119894comfort If119886119894comfort lt 119886
119894low 119886119894low will be used as the human-drivenvehiclersquos action If not we consider the targeted 119886
119894following
a normal distribution with mean value 120583119886119894
between 119886119894low and
119886119894comfort To simplify our model we use the mean value ofthese two boundaries to represent human-driven vehiclersquosaction 119886
119894 Similarly the case with the intention of 119868Yield can
be analyzed in the same processAfter these steps the transition probability Pr(1199041015840 | 119904 119886)
is well formulized and the autonomous vehicle could havethe ability to understand the future motion of the scenariothrough this model
46 Observation Model The observation model is built tosimulate the measurement process The motion intentionis updated in this process The measurements of human-driven vehicles are modeled with conditional independentassumption Thus the observation model can be calculatedas
Pr (119911 | 119886 1199041015840) = Pr (119911host | 1199041015840
host)119873
prod
119894=1
Pr (119911119894| 1199041015840
119894) (15)
The host vehiclersquos observation function is denoted as
Pr (119911host | 1199041015840
host) simN (119911host | 1199091015840
host Σ119911host) (16)
But in this paper due to the use of V2V communicationsensor the observation error almost does not affect the deci-sion-making result The variance matrix is set as zero
The human-driven vehiclersquos observation will follow thevehiclersquos motion intentions Because we do not consider
the observation error the value in metric state will be thesame as the state transition results But the longitudinalintention of human-driven vehicles in the state space will beupdated using the new observations andHMMmentioned inSection 3 The new observation space will be confirmed withthe above step
47 Reward Function The candidate policies have to satisfyseveral evaluation criterions Autonomous vehicles should bedriven safely and comfortably At the same time they shouldfollow the traffic rules and reach the destination as soon aspossible As a result we design objective function (17) consid-ering three aspects including safety time efficiency and trafficlaws where 120583
1 1205832 and 120583
3are the weight coefficient
119877 (119904 119886) = 1205831119877safety (119904 119886) + 1205832119877time (119904 119886)
+ 1205833119877law (119904 119886)
(17)
The detailed information will be discussed in the follow-ing subsections In addition the factor of comfort will be con-sidered and discussed in policy generation part (Section 51)
471 Safety Reward The safety reward function 119877safety(119904 119886)is based on the potential conflict status In our strategysafety reward is defined as a penalty If there are no potentialconflicts the safety rewardwill be set as 0 A large penalty willbe assigned due to the risk of collision status
In an uncontrolled intersection the four approachingdirections are defined as 119860
119894isin 1 2 3 4 (Figure 5) The
driverrsquos lateral intentions are defined as 119868lat isin 119868TR 119868TL 119868GS119868S So the driving trajectory for each vehicle in the inter-section can be generally represented by 119860
119894and 119868lat119895 and we
marked it as 119879119860119894119868lat119895
1 le 119894 le 4 1 le 119895 le 4 The function 119865 isused to judge the potential collision status which is denotedas
119865 (119879119909 119879119910) =
1 if potential conflict
0 otherwise(18)
where 119879119909and 119879
119910are vehiclesrsquo maneuver 119879
119860119894119868lat119895
119865(119909 119910) can be calculated through relative direction
between two cars which is shown in Table 1The safety reward is based on the following items(i) If 119865(119909 119910) is equal to 0 then the safety reward is equal
to 0 due to the noncollision status(ii) If potential collision occurs there will be a large
penalty(iii) If |TTC
119894minus TTChost| lt 119905threshod there is a penalty
depending on |TTC119894minus TTChost| and TTChost
472 Traffic Law Reward Autonomous vehicles should fol-low traffic laws to interactwith human-driven vehicles Trafficlaw is modeled as a function Law(119909 119910) for each two vehicles119909 and 119910
Law (119879119909 119879119910) =
1 if 119909 is prior
0 otherwise(19)
8 Mathematical Problems in Engineering
Human-driven vehicle
Autonomous vehicle
1
4
3
2
Figure 5 One typical scenario for calculating safety reward
Table 1 Safe condition judgments in the intersection
Human-driven vehicle Left side Right side Opposite side Same sideDriving direction TL LK TR S TL LK TR S TL LK TR S TL LK TR S
Turn left permil permil I I permil permil I I permil permil permil I I I I ILane keeping permil permil I I permil permil permil I permil I I I I I I ITurn right I permil I I I I I I permil I I I I I I IStop I I I I I I I I I I I I I I I I
permil indicates potential collision I indicates no potential collision
where 119879119909and 119879
119910are vehiclesrsquo maneuver 119879
119860119894119878119895
This functionLaw(119879
119909 119879119910) is formulized as shown in Algorithm 1
If the behavior will break the law a large penalty is appliedand the behavior of obeying traffic laws will get a zero reward
473 Time Reward The time cost is based on the time to thedestination for the targeted vehicles in the intersection area
Costtime =DTGVhost
(20)
DTG is the distance to the driving goal In addition wealso need to consider the speed limit which is discussed inpolicy generation part in Section 5
5 Approximations on SolvingPOMDP Problem
Solving POMDP is quite difficult The complexity of search-ing total brief space is O(|A|119867|Z|
119867
) [12] where 119867 is theprediction horizon In this paper we model the intentionrecognition process as a deterministic model and use com-munication sensors to ignore the perception error and thusthe size of |Z| is reduced to 1 in the simplified problem Tosolve this problem we first generate the suitable potentialpolicies according to the property of driving tasks and then
select the reasonable total predicting interval time and totalhorizon After that the approximate optimal policy canbe calculated through searching all possible policies withmaximum total rewardThe policy selection process is shownin Algorithm 2 and some detailed explanations are discussedin the subsections
51 Policy Generation For autonomous driving near inter-section the desired velocity curves need to satisfy severalconstraints Firstly except for emergency braking the accel-eration constraints are applied to ensure comfort Secondlythe speed limit constraints should be used in this process Weaim to avoid the acceleration commands when autonomousvehicle is reaching maximum speed limit Thirdly for thecomfort purpose the acceleration command should not bealways changed In otherwords we need tominimize the jerk
Similar to [11] the candidate policies are divided intothree time segments The first two segments are like ldquokeepconstant accelerationdeceleration actionsrdquo while keepingconstant velocity in the third segment We use 119905
1 1199052 and 119905
3
to represent the time periods of these three segments Toguarantee comfort the acceleration is limited to the rangefrom minus4ms2 to 2ms2 and we discrete acceleration actioninto a multiple of [minus05 05 0] Then the action space can berepresented by a discretizing acceleration setThenwe can set
Mathematical Problems in Engineering 9
Law(119879119909 119879119910) larr 0
119905119909larr 1198891199092119868V119909
119905119910larr 1198891199102119868V119910
if 119905119909lt 119905119910minus Δ119905 then
Law(119879119909 119879119910) larr 1
else if |119905119909minus 119905119910| lt Δ119905 then
statuslarr 119865(119879119909 119879119910)
if status = 1 thenif 119868lat119909 = lanekeeping and 119868lat119910 ltgt lanekeeping thenLaw(119879
119909 119879119910) larr 1
else if 119868lat119909 119868lat119910 = lanekeeping and 119860119909minus 119860119910= 1 or minus3 then
Law(119879119909 119879119910) larr 1
else if 119868lat119909 = turnleft and 119868lat119910 = turnright thenLaw(119879
119909 119879119910) larr 1
end ifend if
end ifreturn Law(119879
119909 119879119910)
Algorithm 1 Traffic law formulization
InputPredict horizon119867 time step Δ119905Current states 119904host = 119909host 119904human = [119909human 119868human]
(1) 119875 larr genenratepolicyset()(2) for each 120587
119896isin 119875 do
(3) for 119894 = 1 to119867Δ119905 do(4) 119886host larr 120587
119896(119894)
(5) 1199041015840
host larr updatestate(119904host 119886host)(6) 119886human larr predictactions(1199041015840host 119904human 119868human)
(7) 1199091015840
human larr updatestate(119909human 119886human)
(8) 1198681015840
human larr updateintention(1199041015840host 1199091015840
human)
(9) 1199041015840
human larr [1199091015840
human 1198681015840
human]
(10) 119877(119894) = calculatereward(1199041015840host 1199041015840
human)
(11) 119904host larr 1199041015840
host(12) 119904human larr 119904
1015840
human(13) end(14) 119877
total119896
larr sum119894sum(119877(119894))
(15) end(16) 119896lowast larr argmax
119896(119877
total119896
)
(17) 120587lowast larr 120587119896lowast
(18) return 120587lowast
Algorithm 2 Policy selection process
the value of 1199051 1199052 and 119905
3and the prediction period of single
step An example of policy generation is shown in Figure 6
52 Planning Horizon Selection After building policy gener-ation model the next problem is to select a suitable planninghorizon Longer horizon can lead to a better solution but con-suming more computing resources However as our purpose
is to deal with the interaction problem in the uncontrolledintersection we only need to consider the situation beforeautonomous vehicle gets through In our algorithmwe set theprediction horizon as 8 seconds In addition in the process ofupdating the future state of each vehicle using each policy thecar following mode is used after autonomous vehicle passesthrough the intersection area
10 Mathematical Problems in Engineering
0
1
2Candidate policies
minus4
minus3
minus2
minus1
Acce
lera
tion
(ms2)
1 2 3 4 5 6 7 80Time (s)
(a)
Time (s)0 1 2 3 4 5 6 7 8
0
2
4
6
8
10
12
14
16
18
20
Velo
city
(ms
)
Velocity strategies
(b)
Figure 6 An example of policy generation process (a) is the generated policies and (b) is the corresponding speed profiles The interval ofeach prediction step is 05 s current speed is 12ms2 and the speed limit is 20ms2 The bold black line is one policy In the first 3 secondsautonomous vehicles decelerate in minus35ms2 then accelerate at 2ms2 for 4 seconds and finally stop in the last one second In this case 109policies were generated which is suitable for replanning fast
AB
Figure 7 Testing scenario Autonomous vehicle B and human-driven vehicle A are both approaching the uncontrolled intersection To goacross the intersection successfully autonomous vehicle should interact with human-driven vehicle
53 Time Step Selection Another problem is the predic-tion time step The intention prediction algorithm and thePOMDP are computed in each step If the time step is 119905stepthe total computation timeswill be119867119905stepThus smaller timestep leads to more computation time To solve this problemwe use a simple adaptive time step calculation mechanism togive a final value The time step is selected based on the TTCof autonomous vehicle If the host vehicle is far away from theintersection we can use a very large time step But if the TTCis quite small the low 119905step is applied to ensure safety
6 Experiment and Results
61 Settings In this paper we evaluate our approach throughPreScan 710 [21] a simulation tool for autonomous drivingand connected vehicles Using this software we can build thetesting scenarios (Figure 7) and add vehicles with dynamicmodel In order to get a similar scenario considering social
interaction the driver simulator is added in our experiment(Figure 8) The human-driven vehicle is driven by severalpeople during the experiment and the autonomous vehiclemakes decisions based on the human-driven vehiclersquos drivingbehavior The reference trajectory for autonomous vehicleis generated from path planning module and the human-driven vehiclersquos data (eg position velocity and heading)are transferred through V2V communication sensor Thedecision-making module sends desired velocity command tothe PID controlled to follow the reference path All policies inthe experiment part use a planning horizon119867 = 8 s which isdiscretized into the time step of 05 s
62 Results It is difficult to compare different approachesin the same scenario because the environment is dynamicand not exactly the same However we select two typicalsituations and special settings to make it possible The sameinitial conditions including position orientation and velocity
Mathematical Problems in Engineering 11
Figure 8 Logitech G27 driving simulator
T = 0 1 2 6 s T = 5 s
T = 4 s
T = 3 s
T = 2 s
T = 1 s
T = 0 s
T = 6 s
(a)
T = 0 1 2 9 s
T = 4 s
T = 5 s
T = 6 sT = 7 sT = 8 s
T = 9 s
T = 3 s
T = 2 s
T = 1 s
T = 0 s
(b)
Figure 9 The visualized passing sequence (a) is the result of our approach and (b) represents the result of reactive approach withoutconsidering intention The black vehicle is an autonomous vehicle while the red car is the human-driven vehicle Each vehicle representsthe position in a specific time 119879 with an interval of 1 second
for each vehicle are used in different tests Besides twotypical situations including human-driven vehicle gettingthrough before or after autonomous vehicle are comparedin this section With the same initial state different reac-tions will occur based on various methods We compareour approach and reactive-based method [6] in this sec-tion The key difference for these two methods is that ourapproach considers human-driven vehiclersquos driving inten-tion
The first experiment is that human-driven vehicle triesto yield autonomous vehicle in the interaction process Theresults are shown in Figures 9 and 10 Firstly Figure 9 givesus a visual comparison of the different approaches Fromalmost the same initial state (eg position and velocity) ourapproach could lead to autonomous vehicle passing throughthe intersection more quickly and reasonable
Then let us look at Figure 10 for detailed explanationIn the first 12 s in Figures 10(a) and 10(c) autonomousvehicle maintains speed and understands that human-drivenvehicle will not perform yielding actions Then autonomousvehicle gets yielding intention of human-driven vehicle andunderstands that human-driven vehiclersquos lateral intentionis to go straight Based on candidate policies autonomousvehicle selects acceleration strategy with maximum rewardand finally crosses the intersection In this process we canobviously find that autonomous vehicle understands human-driven vehiclersquos yielding intention Figure 10(c) is an exampleof understand human-driven vehiclersquos behavior based on egovehiclersquos future actions in a specific time Our strategy pre-dicts the future actions of human-driven vehicle Althoughthe velocity curves after 1 s do not correspond it does notaffect the performance of our methods The reason is that
12 Mathematical Problems in Engineering
0 1 2 3 4 5 6 7 80
1020
Time (s)
Velo
city
(ms
) Velocity profile
0 1 2 3 4 5 6 7 8024
Time (s)
Inte
ntio
n Longitudinal intention
0 1 2 3 4 5 6 7 8024
Time (s)
Inte
ntio
n Lateral intention
(a)
0 1 2 3 4 5 6 7 8 90
1020
Time (s)
Velo
city
(ms
) Velocity profile
0 1 2 3 4 5 6 7 8 9024
Time (s)
Inte
ntio
n
Longitudinal intention
0 1 2 3 4 5 6 7 8 9024
Time (s)
Inte
ntio
n
Lateral intention
(b)
0 1 2 3 4 5 6 7 8minus60
minus40
minus20
0
20
40
60
80
Time (s)
Distance to collision region
Dist
ance
(m)
(c)
0 1 2 3 4 5 6 7 8 9minus10
0
10
20
30
40
50
60
70
80
Time (s)
Dist
ance
(m)
Distance to collision region
(d)
0 1 2 3 4 5 6 7 87
8
9
10
11
12
13
14
15
Time (s)
Velo
city
(ms
)
(e)
Time (s)0 1 2 3 4 5 6 7 8
0
2
4
6
8
10
12
14
Velo
city
(ms
)
(f)
Figure 10 Case test 1 In this case human-driven vehicle passes through intersection after autonomous vehicle (a) (c) and (e) are theperformance of our method while (b) (d) and (f) are from the strategy without considering the driving intention (a) and (b) are the velocityprofiles and the corresponding driving intention For longitudinal intention label 1 means yielding and label 2 means not yielding In lateralintention 1 means turning left 2 means turning right 3 means going straight and 4 means stop The intentions in (b) are not used in thatmethod but for detailed analysis (c) and (d) are the distance to collision area for autonomous vehicle and human-driven vehicle respectively(e) and (f) are the prediction and true motions of human-driven vehicles in time 15 s with a prediction length of 8 s The red curves in thesesubfigures are from autonomous vehicle while blue lines are from human-driven vehicle The green lines in (e) and (f) are the predictionvelocity curves of human-driven vehicle
Mathematical Problems in Engineering 13
T = 4 s
T = 1 s
T = 2 s
T = 3 s
T = 5 s
T = 0 s
T = 0 1 2 5 s
(a)
T = 4 s
T = 1 s
T = 2 s
T = 3 s
T = 5 s
T = 0 s
T = 0 1 2 5 s
(b)
Figure 11 The visualized passing sequence for the case of human-driven vehicle first getting through (a) is the result of our approach and(b) is the reactive-based approach
we use a deterministic model in the prediction process andthe prediction value is inside two boundaries to ensure safetyBesides the whole actions of autonomous vehicle in thisprocess could also help human-driven vehicle to understandnot-yielding intention of autonomous vehicles In this casecooperative driving behaviors are performed by both vehi-cles
However if the intention is not considered in this processwe can find the results in Figures 10(b) 10(d) and 10(f)After 2 s in Figure 10(b) while the human-driven vehiclegives a yielding intention autonomous vehicle could notunderstand and they find a potential collision based on theconstant velocity assumptions Then it decreases the speedbut the human-driven vehicle also slows down The puzzledbehavior leads both vehicles to slow down near intersectionFinally human-driven vehicle stops at the stop line and thenautonomous vehicle could pass the intersection In this strat-egy the human-driven vehiclersquos future motion is assumed tobe constant (Figure 10(f))Without understanding of human-driven vehiclersquos intentions this strategy can increase conges-tion problem
Another experiment is that human-driven vehicle tries toget through the intersection first The results are shown inFigures 11 and 12This case is quite typical because many traf-fic accidents in real world are happening in this situation Indetail if one vehicle tries to cross an intersection while violat-ing the law another vehicle will be in great danger if it doesnot understand its behavior From the visualized perform-ance in Figure 11 our method is a little more safe than otherapproaches as there is nearly collision situation in Figure11(b) In detail we can see from Figure 12(a) that our strategycould perform deceleration actions after we understand thenot-yielding intention in 08 sHowever without understand-ing human-driven vehiclersquos motion intention the responsetime has a 1-second delay which may be quite dangerous
In addition it is shown that good performance is in thepredictions of human-driven vehiclersquos future motion in ourmethods (Figure 12(e))
The results of these two cases demonstrate that our algo-rithm could deal with typical scenarios and have better per-formance than traditional reactive controller Autonomousvehicle could be driven more safely fast and comfortablythrough our strategy
7 Conclusion and Future Work
In this paper we proposed an autonomous driving decision-making algorithm considering human-driven vehiclersquos uncer-tain intentions in an uncontrolled intersection The lat-eral and longitudinal intentions are recognized by a con-tinuous HMM Based on HMM and POMDP we modelgeneral decision-making process and then use an approxi-mate approach to solve this complex problem Finally weuse PreScan software and a driving simulator to emulatesocial interaction process The experiment results show thatautonomous vehicles with our approach can pass throughuncontrolled intersections more safely and efficiently thanusing the strategy without considering human-driven vehi-clesrsquo driving intentions
In the near future we aim to implement our approach intoa real autonomous vehicle and perform real world experi-ments In additionmore precious intention recognition algo-rithm aims to be figured out Somemethods like probabilisticgraphic model can be used to get a distribution of each inten-tion Finally designing online POMDP planning algorithmsis also valuable
Competing Interests
The authors declare that there are no competing interestsregarding the publication of this paper
14 Mathematical Problems in Engineering
0 05 1 15 2 25 3 35 4 45 510
20
30
Time (s)
Velo
city
(ms
) Velocity profile
0 05 1 15 2 25 3 35 4 45 50
2
4
Time (s)
Inte
ntio
n
Longitudinal intention
0 05 1 15 2 25 3 35 4 45 5024
Inte
ntio
n Lateral intention
Time (s)
(a)
0 05 1 15 2 25 3 35 4 45 5102030
Time (s)
Velo
city
(ms
) Velocity profile
0 05 1 15 2 25 3 35 4 45 5024
Time (s)
Inte
ntio
n
Longitudinal intention
0 05 1 15 2 25 3 35 4 45 5024
Time (s)
Inte
ntio
n
Lateral intention
(b)
0 05 1 15 2 25 3 35 4 45 5
0
20
40
60
80
Dist
ance
(m)
Distance to collision region
Time (s)
minus40
minus20
(c)
0 05 1 15 2 25 3 35 4 45 5
0
20
40
60
80D
istan
ce (m
)
Distance to collision region
Time (s)
minus40
minus20
(d)
0 1 2 3 4 5 6 7 816
17
18
19
20
21
22
23
24
25
Velo
city
(ms
)
Time (s)
(e)
0 1 2 3 4 5 6 7 817
18
19
20
21
22
23
24
25
Time (s)
Velo
city
(ms
)
(f)
Figure 12 Case test 2 In this case human-driven vehicle passes through intersection before autonomous vehicle through different strategiesThe definition of each subfigure is the same as in Figure 10
Mathematical Problems in Engineering 15
Acknowledgments
This study is supported by the National Natural ScienceFoundation of China (no 91420203)
References
[1] C Urmson J Anhalt D Bagnell et al ldquoAutonomous driving inurban environments boss and the urban challengerdquo Journal ofField Robotics vol 25 no 8 pp 425ndash466 2008
[2] J Markoff ldquoGoogle cars drive themselves in trafficrdquo New YorkTimes vol 9 2010
[3] L R Rabiner and B-H Juang ldquoAn introduction to hiddenMarkov modelsrdquo IEEE ASSP Magazine vol 3 no 1 pp 4ndash161986
[4] M Buehler K Iagnemma and S Singh The DARPA UrbanChallenge Autonomous Vehicles in City Traffic vol 56 Springer2009
[5] S Kammel J Ziegler B Pitzer et al ldquoTeam AnnieWAYrsquos auto-nomous system for the 2007 DARPAUrban Challengerdquo Journalof Field Robotics vol 25 no 9 pp 615ndash639 2008
[6] C R Baker and J M Dolan ldquoTraffic interaction in the urbanchallenge putting boss on its best behaviorrdquo in Proceedingsof the IEEERSJ International Conference on Intelligent Robotsand Systems (IROS rsquo08) pp 1752ndash1758 IEEE Nice FranceSeptember 2008
[7] M Montemerlo J Becker S Shat et al ldquoJunior the Stanfordentry in the urban challengerdquo Journal of Field Robotics vol 25no 9 pp 569ndash597 2008
[8] L Fletcher S Teller E Olson et al ldquoTheMIT-Cornell collisionand why it happenedrdquo Journal of Field Robotics vol 25 no 10pp 775ndash807 2008
[9] M Bahram A Wolf M Aeberhard and D Wollherr ldquoAprediction-based reactive driving strategy for highly automateddriving function on freewaysrdquo in Proceedings of the 25th IEEEIntelligent Vehicles Symposium pp 400ndash406 IEEE DearbornMich USA June 2014
[10] J Wei J M Dolan and B Litkouhi ldquoA prediction- and costfunction-based algorithm for robust autonomous freeway driv-ingrdquo in Proceedings of the IEEE Intelligent Vehicles Symposium(IV rsquo10) pp 512ndash517 San Diego Calif USA June 2010
[11] J Wei J M Dolan and B Litkouhi ldquoAutonomous vehiclesocial behavior for highway entrance ramp managementrdquo inProceedings of the IEEE Intelligent Vehicles Symposium (IV rsquo13)pp 201ndash207 IEEE Gold Coast Australia June 2013
[12] H Bai S Cai N Ye D Hsu and W S Lee ldquoIntention-awareonline POMDP planning for autonomous driving in a crowdrdquoin Proceedings of the IEEE International Conference on Roboticsand Automation (ICRA rsquo15) pp 454ndash460 IEEE Seattle WashUSA May 2015
[13] A Somani N Ye D Hsu and W S Lee ldquoDespot onlinepomdp planning with regularizationrdquo in Advances in NeuralInformation Processing Systems pp 1772ndash1780 2013
[14] S Brechtel T Gindele and R Dillmann ldquoProbabilistic deci-sion-making under uncertainty for autonomous driving usingcontinuous POMDPsrdquo in Proceedings of the 17th IEEE Interna-tional Conference on Intelligent Transportation Systems (ITSCrsquo14) pp 392ndash399 Qingdao China October 2014
[15] C H Papadimitriou and J N Tsitsiklis ldquoThe complexity ofMarkov decision processesrdquo Mathematics of Operations Re-search vol 12 no 3 pp 441ndash450 1987
[16] O Madani S Hanks and A Condon ldquoOn the undecidabilityof probabilistic planning and related stochastic optimizationproblemsrdquoArtificial Intelligence vol 147 no 1-2 pp 5ndash34 2003
[17] S Ulbrich and M Maurer ldquoProbabilistic online POMDP deci-sion making for lane changes in fully automated drivingrdquo inProceedings of the 16th International IEEE Conference on Intelli-gent Transportation Systems (ITSC rsquo13) pp 2063ndash2067 TheHague The Netherlands October 2013
[18] A G Cunningham E Galceran R M Eustice and E OlsonldquoMPDM multipolicy decision-making in dynamic uncertainenvironments for autonomous drivingrdquo in Proceedings of theIEEE International Conference on Robotics and Automation(ICRA rsquo15) pp 1670ndash1677 Seattle Wash USA May 2015
[19] D A Reynolds and R C Rose ldquoRobust text-independentspeaker identification using Gaussian mixture speaker modelsrdquoIEEE Transactions on Speech and Audio Processing vol 3 no 1pp 72ndash83 1995
[20] L E Baum and T Petrie ldquoStatistical inference for probabilisticfunctions of finite state Markov chainsrdquo The Annals of Mathe-matical Statistics vol 37 pp 1554ndash1563 1966
[21] M Tideman and M Van Noort ldquoA simulation tool suite fordeveloping connected vehicle systemsrdquo in Proceedings of theIEEE Intelligent Vehicles Symposium (IEEE IV rsquo13) pp 713ndash718Queensland Australia June 2013
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
4 Mathematical Problems in Engineering
where 119888119895119896
represents the mixture coefficient in the 119895th statefor the 119896th mixture N is the pdf of a Gaussian distributionwith mean 120583 and covariance Σmeasured from observation oMixture coefficient 119888 satisfies the following constraints
119872
sum
119896=1
119888119895119896= 1 (4)
where 119888119895119896gt 0 1 le 119895 le 119873 1 le 119896 le 119872
And
int
+infin
minusinfin
119887119895(o) 119889o = 1 1 le 119895 le 119873 (5)
Then a HMM could be completely defined by hiddenstates119873 and the probability tuples 120582 = (120587 119860 119862 120583 Σ)
In the training process we use the Baum-Welch method[20] to estimate model parameters for different driver inten-tion 119868 Once themodel parameters corresponding to differentdriver intention have been trained we can perform thedriverrsquos intention estimation in the recognition process Theprediction process for lateral intentions can be seen inFigure 2
32 HMM-Based Intention Prediction Process Given a con-tinuous HMM the intention prediction process is dividedinto two steps The first step focused on the lateral intentionThe training inputs of each vehiclersquos lateral intention modelin time 119905 are defined as 119861lateral = 119871 V 119886 yaw where 119871 isthe distance to the intersection V is the longitudinal velocity119886 is the longitudinal acceleration and yaw is the yaw ratewhile the output of this model is the motion intentions 119868lat isin119868TR 119868TL 119868GS 119868S The corresponding HMMs can be trainedincluding 120582TR 120582TL 120582GS and 120582119878
The next step is about longitudinal intention This prob-ability could be decomposed based on the total probabilityformula
119875 (119868Yield | 119861) = sum119875 (119868Yield | 119868lat 119861) 119875 (119868lat | 119861)
= 119875 (119868Yield | 119868TR 119861) 119875 (119868TR | 119861)
+ 119875 (119868Yield | 119868TL 119861) 119875 (119868TL)
+ 119875 (119868Yield | 119868GS 119861) 119875 (119868GS | 119861)
+ 119875 (119868Yield | 119868S 119861) 119875 (119868S | 119861)
(6)
where 119861 is the behavior data including 119861lateral and 119861lonIn this process we assume that the lateral behavior 119868lat is
predicted correctly by a deterministic HMM in the first stepand therefore 119868lat is determined by the lateral prediction result119868latPredict where 119875(119868lat | 119861 119868lat = 119868latPredict) = 1 and 119875(119868lat | 119861119868lat = 119868latPredict) = 0 And (6) is reformulated by
119875 (119868Yield | 119861) = 119875 (119868Yield | 119861 119868latPredict) 119875 (119868latPredict | 119861)
= 119875 (119868Yield | 119861 119868latPredict) (7)
The problem is changed to model 119875(119868Yield | 119861 119868latPredict)The features used in longitudinal intention prediction are
119861lon = ΔV Δ119886 ΔDTC where ΔV = Vsocial minus Vhost Δ119886 =
119886social minus 119886host and ΔDTC = DTCsocial minusDTChost DTC meansthe distance to the potential collision area The output ofthe longitudinal intention prediction model is longitudinalmotion intention 119868lon isin 119868Yield 119868119873Yield
Instead of building a generative model we use a deter-ministic approach to restrict 119875(119868Yield | 119861 119868latPredict) as 0 or1 Thus two types of HMMs named 120582
119884119868lat 120582119873119868lat
are trainedwhere 119868lat isin 119868TR 119868TL 119868GS 119868S Two test examples for lateraland longitudinal intention prediction are shown in Figures3 and 4 Through these two figures we can find that ourapproach can recognize human-driven vehiclersquos lateral andlongitudinal intention successfully
4 Modeling Autonomous DrivingDecision-Making in a POMDP Framework
For the decision-making process the key problem is howto design a policy to perform the optimal actions withuncertainties This needs to not only obtain traffic lawsbut also consider the driving uncertainties of human-drivenvehicles Facing potential conflicts human-driven vehicleshave uncertain probabilities to yield autonomous vehicles andsome aggressive drivers may violate the traffic laws Suchelements should be implemented into a powerful decision-making framework As a result we model autonomousdriving decision-making problem in a general POMDPframework in this section
41 POMDP Preliminaries A POMDP model can be for-mulized as a tupleSA 119879Z 119874 119877 120574 where S is a setof states A is the action space and Z denotes a set ofobservations The conditional function 119879(1199041015840 119886 119904) = Pr(1199041015840 |119904 119886) models transition probabilities to state 1199041015840 isin S whenthe system takes an action 119886 isin A in the state 119904 isin S Theobservation function 119874(119911 1199041015840 119886) = Pr(119911 | 119904
1015840
119886) models theprobability of observing 119911 isin Z when an action 119886 isin A istaken and the end state is 1199041015840 isin S The reward function 119877(119904 119886)calculates an immediate reward when taking an action 119886 instate 119904 120574 isin [0 1] is the discount factor in order to balance theimmediate and the future rewards
Because the system contains partially observed state suchas intentions a belief 119887 isin B is maintained A belief updatefunction 120591 is defined as 1198871015840 = 120591(119887 119886 119911) If the agent takes action119886 and gets observation 119911 the new belief 1198871015840 is obtained throughthe Bayesrsquo rule
1198871015840
(1199041015840
) = 120578119874 (1199041015840
119886 119911)sum
119904isin119878
119879 (119904 119886 1199041015840
) 119887 (119904) (8)
where 120578 = 1sum1199041015840isin119878119874(1199041015840
119886 119911)sum119904isin119878119879(119904 119886 119904
1015840
)119887(119904) is a normal-izing constant
A key concept in POMDP planning is a policy a mapping120587 that specifies the action 119886 = 120587(119887) at belief 119887 To solvethe POMDP an optimal policy 120587lowast should be designed tomaximize the total reward
120587lowast
= argmax120587
(119864(
infin
sum
119905=0
120574119905
119877 (119904119905 120587 (119887119905)) | 1198870 120587)) (9)
where 1198870is marked as the initial belief
Mathematical Problems in Engineering 5
Distance tointersection
Velocity
Acceleration
Yaw rate
Observe sequence
HMM_TR Forward algorithm
HMM_TL Forward algorithm
HMM_S Forward algorithm
HMM_GS Forward algorithm
DriverintentionO = B1 B2 BM
1205821 = (1205871 A1 C1 1205831 Σ1)
1205822 = (1205872 A2 C2 1205832 Σ2)
1205823 = (1205873 A3 C3 1205833 Σ4)
1205824 = (1205874 A4 C4 1205834 Σ4)
P(O | 1205822)
P(O | 1205821)
P(O | 1205824)
P(O | 1205824)
arg max P(O | 120582i)i
Figure 2 Prediction process for HMMThe observed sequence will be evaluated by four HMMs Forward algorithm is used to calculate theconditional probabilities and the intention corresponding to the largest value will be considered as the vehiclersquos intention
Time (s)1 2 3 4 5 6
024
0
Predict results
(a)
01 2 3 4 5 60
1020
Velocity
v (m
s)
Time (s)
(b)
0
1 2 3 4 5 60
10
minus10
Acceleration
a (m
s2)
Time (s)
(c)
Time (s)0 1 2 3 4 5 6
minus30minus20minus10
010
Yaw rate
Yaw
rate
(deg
s)
(d)
0 1 2 3 4 5 60
1020
Time (s)
Distance to intersection (DTI)
DTI
(m)
(e)
Figure 3 Lateral intention prediction example The true intention of human-driven vehicle is to turn left in this scenario In the first figurethe value 1 of the 119910 label means turn left 2 means turn right 3 represents go straight and 4 corresponds to stop
42 State Space Because of the Markov property sufficientinformation should be contained in the state space S fordecision-making process [14] The state space includes thevehicle pose [119909 119910 120579] velocity V the average yaw rate yawaveand acceleration 119886ave in the last planning period for allthe vehicles For the human-driven vehicles the lateral andlongitudinal intentions [119868lat 119868lon] also need to be containedfor state transition modeling However the road contextknowledge is static reference information so that it will be notadded to the state space
The joint state 119904 isin S could be denoted as 119904 = [119904host 1199041
1199042 119904
119873]119879 where 119904host is the state of host vehicle (auto-
nomous vehicle) 119904119894 119894 isin 1 2 3 119873 is the state of
human-driven vehicles and 119873 is the number of human-driven vehicles involved Let us define metric state 119909 =
[119909 119910 120579 V 119886ave yawave]119879 including the vehicle position head-
ing velocity acceleration and yaw rateThus the state of hostvehicle can be defined as 119904host = 119909host while the human-driven vehicle state 119904
119894is 119904119894= [119909
119894 119868lat119894 119868lon119894]
119879 With theadvanced perception system and V2V communication tech-nology we assume that the metric state 119909 could be observedBecause the sensor noise is small and hardly affects decision-making process we do not model observation noise for themetric state However the intention state cannot be directlyobserved so it is the partially observable variables in ourpaper The intention state should be inferred from obser-vation data and predictive model over time
43 Action Space In our autonomous vehicle system thedecision-making system is used to select the suitable tactical
6 Mathematical Problems in Engineering
Time (s)0 1 2 3 4 5 6 7 8
012
Predict results
(a)
0 1 2 3 4 5 6 7 80
1020
Relative velocity (RV)
RV (m
s)
Time (s)
(b)
0 1 2 3 4 5 6 7 8minus2
02
Relative acceleration (RA)
RA (m
s2)
Time (s)
(c)
0 1 2 3 4 5 6 7
02040
Time (s)
minus20
Relative DTC (RDTC)
RDTC
(m)
(d)
Figure 4 One example of predicting longitudinal intentions This example is based on the scenario of Figure 1 and two vehicles both gostraightThe value 1 of 119910-axis in the first figure denotes the intention of yielding while 2 represents not yielding In the first 28 s the intentionis yielding After that due to the acceleration action and less relative DTC autonomous vehicle could understand human-driven vehiclersquosnot-yielding intention
maneuvers Specifically in the intersection area autonomousvehicles should follow a global reference path generated bypath planning module The decision-making module onlyneeds to generate accelerationdeceleration commands to thecontrol layer As the reference path may not be straight thesteering control module can adjust the front wheel angleto follow the reference path Therefore the action space Acould be defined as a discrete set A = [acc dec con] whichcontains commands including acceleration deceleration andmaintaining current velocity
44 Observation Space Similar to the joint state space theobservation 119911 is denoted as 119911 = [119911host 1199111 1199112 119911119873]
119879 where119911host and 119911119894 are the host vehicle and human-driven vehiclersquosobservations respectively The acceleration and yaw rate canbe approximately calculated by speed and heading in theconsecutive states
45 State Transition Model In state transition process weneed to model transition probability Pr(1199041015840 | 119904 119886) Thisprobability is determined by each targeted element in thescenario So the transition model can be calculated by thefollowing probabilistic equation
Pr (1199041015840 | 119904 119886) = Pr (1199041015840host | 119904host 119886host)119873
prod
119894=1
Pr (1199041015840119894| 119904119894) (10)
In the decision-making layer we do not need to considercomplex vehicle dynamic model Thus the host vehiclersquosmotion Pr(1199041015840host | 119904host 119886host) can be simply represented by thefollowing equations given action 119886
1199091015840
= 119909 + (V +119886Δ119905
2)Δ119905 cos (120579 + Δ120579)
1199101015840
= 119910 + (V +119886Δ119905
2)Δ119905 sin (120579 + Δ120579)
1205791015840
= 120579 + Δ120579
V1015840 = V + 119886Δ119905
yaw1015840ave =Δ120579
Δ119905
1198861015840
ave = 119886
(11)
Thus the key problem is converted to compute Pr(1199041015840119894|
119904119894) the state transition probability of human-driven vehicles
Based on the total probability formula this probability can befactorized as a sum in whole action space
Pr (1199041015840119894| 119904119894) = sum
119886119894
Pr (1199041015840119894| 119904119894 119886119894)Pr (119886
119894| 119904119894) (12)
With this equation we only need to calculate the statetransition probability Pr(1199041015840
119894| 119904119894 119886119894) given a specific action 119886
119894
and the probability of selecting this action Pr(119886119894| 119904119894) under
current state 119904119894
Because the human-driven vehiclesrsquo state 119904119894= [119909119894 119868119894] the
probability Pr(1199041015840119894| 119904119894 119886119894) can be calculated as
Pr (1199041015840119894| 119904119894 119886119894) = Pr (1199091015840
119894 1198681015840
119894| 119909119894 119868119894 119886119894)
= Pr (1199091015840119894| 119909119894 119868119894 119886119894)Pr (1198681015840
119894| 1199091015840
119894 119909119894 119868119894 119886119894)
(13)
With a certain action 119886119894 Pr(1199091015840119894| 119909119894 119868119894 119886119894) is equal to Pr(1199091015840
119894|
119909119894 119868lat119894 119886119894)The lateral behavior 119868lat119894 is considered to be a goal-
directed driving intention which will not be changed in thedriving process So Pr(1199091015840
119894| 119909119894 119868lat119894 119886119894) is equal to Pr(119909
1015840
119894| 119909119894 119886119894)
given a reference path corresponding to the intention of 119868lat119894Using (11) Pr(1199091015840
119894| 119909119894 119886119894) can be well solved
The remaining problem for calculating Pr(1199041015840119894| 119904119894 119886119894) is
to deal with Pr(1198681015840119894| 1199091015840
119894 119909119894 119868119894 119886119894) The lateral intention 1198681015840lat119894
is assumed stable through the above explanation And the
Mathematical Problems in Engineering 7
longitudinal intention 1198681015840
lon119894 is assumed to be not updatedin this process But it will be updated with new inputs inobservation space
Now Pr(1199041015840119894| 119904119894 119886119894) is well modeled and the remaining
problem is to compute the probabilities Pr(119886119894| 119904119894) of human-
driven vehiclesrsquo future actions
Pr (119886119894| 119904119894) = Pr (119886
119894| 119909119894 119868119894)
= sum
1199091015840
host
Pr (119886119894| 1199091015840
host 119909119894 119868119894)Pr (1199091015840
host | 119909119894 119868119894) (14)
Because 1199091015840
host is determined by the designed policyPr(1199091015840host | 119909119894 119868119894) could be calculated by (11) given an action119886host The probability Pr(119886
119894| 1199091015840
host 119909119894 119868119894) means the distribu-tion of human-driven vehiclesrsquo actions given the new state1199091015840
host of host vehicle the current state of itself and itsintentions Instead of building a complex probability modelwe designed a deterministic mechanism to calculate themostpossible action 119886
119894given 1199091015840host 119909119894 and 119868119894
In this prediction process the host vehicle is assumed tobe maintaining the current actions in the next time step andthe action 119886
119894will be leading human-driven vehicle passing
through the potential collision area either in advance of hostvehicle under the intention 119868
119873Yield or behind the host vehicleunder the intention 119868Yield to keep a safe distance 119889safe In thecase with the intention of 119868
119873Yield we can calculate the lowboundary 119886
119894low of 119886119894 through the above process anddeterminethe upper one using the largest comfort value 119886
119894comfort If119886119894comfort lt 119886
119894low 119886119894low will be used as the human-drivenvehiclersquos action If not we consider the targeted 119886
119894following
a normal distribution with mean value 120583119886119894
between 119886119894low and
119886119894comfort To simplify our model we use the mean value ofthese two boundaries to represent human-driven vehiclersquosaction 119886
119894 Similarly the case with the intention of 119868Yield can
be analyzed in the same processAfter these steps the transition probability Pr(1199041015840 | 119904 119886)
is well formulized and the autonomous vehicle could havethe ability to understand the future motion of the scenariothrough this model
46 Observation Model The observation model is built tosimulate the measurement process The motion intentionis updated in this process The measurements of human-driven vehicles are modeled with conditional independentassumption Thus the observation model can be calculatedas
Pr (119911 | 119886 1199041015840) = Pr (119911host | 1199041015840
host)119873
prod
119894=1
Pr (119911119894| 1199041015840
119894) (15)
The host vehiclersquos observation function is denoted as
Pr (119911host | 1199041015840
host) simN (119911host | 1199091015840
host Σ119911host) (16)
But in this paper due to the use of V2V communicationsensor the observation error almost does not affect the deci-sion-making result The variance matrix is set as zero
The human-driven vehiclersquos observation will follow thevehiclersquos motion intentions Because we do not consider
the observation error the value in metric state will be thesame as the state transition results But the longitudinalintention of human-driven vehicles in the state space will beupdated using the new observations andHMMmentioned inSection 3 The new observation space will be confirmed withthe above step
47 Reward Function The candidate policies have to satisfyseveral evaluation criterions Autonomous vehicles should bedriven safely and comfortably At the same time they shouldfollow the traffic rules and reach the destination as soon aspossible As a result we design objective function (17) consid-ering three aspects including safety time efficiency and trafficlaws where 120583
1 1205832 and 120583
3are the weight coefficient
119877 (119904 119886) = 1205831119877safety (119904 119886) + 1205832119877time (119904 119886)
+ 1205833119877law (119904 119886)
(17)
The detailed information will be discussed in the follow-ing subsections In addition the factor of comfort will be con-sidered and discussed in policy generation part (Section 51)
471 Safety Reward The safety reward function 119877safety(119904 119886)is based on the potential conflict status In our strategysafety reward is defined as a penalty If there are no potentialconflicts the safety rewardwill be set as 0 A large penalty willbe assigned due to the risk of collision status
In an uncontrolled intersection the four approachingdirections are defined as 119860
119894isin 1 2 3 4 (Figure 5) The
driverrsquos lateral intentions are defined as 119868lat isin 119868TR 119868TL 119868GS119868S So the driving trajectory for each vehicle in the inter-section can be generally represented by 119860
119894and 119868lat119895 and we
marked it as 119879119860119894119868lat119895
1 le 119894 le 4 1 le 119895 le 4 The function 119865 isused to judge the potential collision status which is denotedas
119865 (119879119909 119879119910) =
1 if potential conflict
0 otherwise(18)
where 119879119909and 119879
119910are vehiclesrsquo maneuver 119879
119860119894119868lat119895
119865(119909 119910) can be calculated through relative direction
between two cars which is shown in Table 1The safety reward is based on the following items(i) If 119865(119909 119910) is equal to 0 then the safety reward is equal
to 0 due to the noncollision status(ii) If potential collision occurs there will be a large
penalty(iii) If |TTC
119894minus TTChost| lt 119905threshod there is a penalty
depending on |TTC119894minus TTChost| and TTChost
472 Traffic Law Reward Autonomous vehicles should fol-low traffic laws to interactwith human-driven vehicles Trafficlaw is modeled as a function Law(119909 119910) for each two vehicles119909 and 119910
Law (119879119909 119879119910) =
1 if 119909 is prior
0 otherwise(19)
8 Mathematical Problems in Engineering
Human-driven vehicle
Autonomous vehicle
1
4
3
2
Figure 5 One typical scenario for calculating safety reward
Table 1 Safe condition judgments in the intersection
Human-driven vehicle Left side Right side Opposite side Same sideDriving direction TL LK TR S TL LK TR S TL LK TR S TL LK TR S
Turn left permil permil I I permil permil I I permil permil permil I I I I ILane keeping permil permil I I permil permil permil I permil I I I I I I ITurn right I permil I I I I I I permil I I I I I I IStop I I I I I I I I I I I I I I I I
permil indicates potential collision I indicates no potential collision
where 119879119909and 119879
119910are vehiclesrsquo maneuver 119879
119860119894119878119895
This functionLaw(119879
119909 119879119910) is formulized as shown in Algorithm 1
If the behavior will break the law a large penalty is appliedand the behavior of obeying traffic laws will get a zero reward
473 Time Reward The time cost is based on the time to thedestination for the targeted vehicles in the intersection area
Costtime =DTGVhost
(20)
DTG is the distance to the driving goal In addition wealso need to consider the speed limit which is discussed inpolicy generation part in Section 5
5 Approximations on SolvingPOMDP Problem
Solving POMDP is quite difficult The complexity of search-ing total brief space is O(|A|119867|Z|
119867
) [12] where 119867 is theprediction horizon In this paper we model the intentionrecognition process as a deterministic model and use com-munication sensors to ignore the perception error and thusthe size of |Z| is reduced to 1 in the simplified problem Tosolve this problem we first generate the suitable potentialpolicies according to the property of driving tasks and then
select the reasonable total predicting interval time and totalhorizon After that the approximate optimal policy canbe calculated through searching all possible policies withmaximum total rewardThe policy selection process is shownin Algorithm 2 and some detailed explanations are discussedin the subsections
51 Policy Generation For autonomous driving near inter-section the desired velocity curves need to satisfy severalconstraints Firstly except for emergency braking the accel-eration constraints are applied to ensure comfort Secondlythe speed limit constraints should be used in this process Weaim to avoid the acceleration commands when autonomousvehicle is reaching maximum speed limit Thirdly for thecomfort purpose the acceleration command should not bealways changed In otherwords we need tominimize the jerk
Similar to [11] the candidate policies are divided intothree time segments The first two segments are like ldquokeepconstant accelerationdeceleration actionsrdquo while keepingconstant velocity in the third segment We use 119905
1 1199052 and 119905
3
to represent the time periods of these three segments Toguarantee comfort the acceleration is limited to the rangefrom minus4ms2 to 2ms2 and we discrete acceleration actioninto a multiple of [minus05 05 0] Then the action space can berepresented by a discretizing acceleration setThenwe can set
Mathematical Problems in Engineering 9
Law(119879119909 119879119910) larr 0
119905119909larr 1198891199092119868V119909
119905119910larr 1198891199102119868V119910
if 119905119909lt 119905119910minus Δ119905 then
Law(119879119909 119879119910) larr 1
else if |119905119909minus 119905119910| lt Δ119905 then
statuslarr 119865(119879119909 119879119910)
if status = 1 thenif 119868lat119909 = lanekeeping and 119868lat119910 ltgt lanekeeping thenLaw(119879
119909 119879119910) larr 1
else if 119868lat119909 119868lat119910 = lanekeeping and 119860119909minus 119860119910= 1 or minus3 then
Law(119879119909 119879119910) larr 1
else if 119868lat119909 = turnleft and 119868lat119910 = turnright thenLaw(119879
119909 119879119910) larr 1
end ifend if
end ifreturn Law(119879
119909 119879119910)
Algorithm 1 Traffic law formulization
InputPredict horizon119867 time step Δ119905Current states 119904host = 119909host 119904human = [119909human 119868human]
(1) 119875 larr genenratepolicyset()(2) for each 120587
119896isin 119875 do
(3) for 119894 = 1 to119867Δ119905 do(4) 119886host larr 120587
119896(119894)
(5) 1199041015840
host larr updatestate(119904host 119886host)(6) 119886human larr predictactions(1199041015840host 119904human 119868human)
(7) 1199091015840
human larr updatestate(119909human 119886human)
(8) 1198681015840
human larr updateintention(1199041015840host 1199091015840
human)
(9) 1199041015840
human larr [1199091015840
human 1198681015840
human]
(10) 119877(119894) = calculatereward(1199041015840host 1199041015840
human)
(11) 119904host larr 1199041015840
host(12) 119904human larr 119904
1015840
human(13) end(14) 119877
total119896
larr sum119894sum(119877(119894))
(15) end(16) 119896lowast larr argmax
119896(119877
total119896
)
(17) 120587lowast larr 120587119896lowast
(18) return 120587lowast
Algorithm 2 Policy selection process
the value of 1199051 1199052 and 119905
3and the prediction period of single
step An example of policy generation is shown in Figure 6
52 Planning Horizon Selection After building policy gener-ation model the next problem is to select a suitable planninghorizon Longer horizon can lead to a better solution but con-suming more computing resources However as our purpose
is to deal with the interaction problem in the uncontrolledintersection we only need to consider the situation beforeautonomous vehicle gets through In our algorithmwe set theprediction horizon as 8 seconds In addition in the process ofupdating the future state of each vehicle using each policy thecar following mode is used after autonomous vehicle passesthrough the intersection area
10 Mathematical Problems in Engineering
0
1
2Candidate policies
minus4
minus3
minus2
minus1
Acce
lera
tion
(ms2)
1 2 3 4 5 6 7 80Time (s)
(a)
Time (s)0 1 2 3 4 5 6 7 8
0
2
4
6
8
10
12
14
16
18
20
Velo
city
(ms
)
Velocity strategies
(b)
Figure 6 An example of policy generation process (a) is the generated policies and (b) is the corresponding speed profiles The interval ofeach prediction step is 05 s current speed is 12ms2 and the speed limit is 20ms2 The bold black line is one policy In the first 3 secondsautonomous vehicles decelerate in minus35ms2 then accelerate at 2ms2 for 4 seconds and finally stop in the last one second In this case 109policies were generated which is suitable for replanning fast
AB
Figure 7 Testing scenario Autonomous vehicle B and human-driven vehicle A are both approaching the uncontrolled intersection To goacross the intersection successfully autonomous vehicle should interact with human-driven vehicle
53 Time Step Selection Another problem is the predic-tion time step The intention prediction algorithm and thePOMDP are computed in each step If the time step is 119905stepthe total computation timeswill be119867119905stepThus smaller timestep leads to more computation time To solve this problemwe use a simple adaptive time step calculation mechanism togive a final value The time step is selected based on the TTCof autonomous vehicle If the host vehicle is far away from theintersection we can use a very large time step But if the TTCis quite small the low 119905step is applied to ensure safety
6 Experiment and Results
61 Settings In this paper we evaluate our approach throughPreScan 710 [21] a simulation tool for autonomous drivingand connected vehicles Using this software we can build thetesting scenarios (Figure 7) and add vehicles with dynamicmodel In order to get a similar scenario considering social
interaction the driver simulator is added in our experiment(Figure 8) The human-driven vehicle is driven by severalpeople during the experiment and the autonomous vehiclemakes decisions based on the human-driven vehiclersquos drivingbehavior The reference trajectory for autonomous vehicleis generated from path planning module and the human-driven vehiclersquos data (eg position velocity and heading)are transferred through V2V communication sensor Thedecision-making module sends desired velocity command tothe PID controlled to follow the reference path All policies inthe experiment part use a planning horizon119867 = 8 s which isdiscretized into the time step of 05 s
62 Results It is difficult to compare different approachesin the same scenario because the environment is dynamicand not exactly the same However we select two typicalsituations and special settings to make it possible The sameinitial conditions including position orientation and velocity
Mathematical Problems in Engineering 11
Figure 8 Logitech G27 driving simulator
T = 0 1 2 6 s T = 5 s
T = 4 s
T = 3 s
T = 2 s
T = 1 s
T = 0 s
T = 6 s
(a)
T = 0 1 2 9 s
T = 4 s
T = 5 s
T = 6 sT = 7 sT = 8 s
T = 9 s
T = 3 s
T = 2 s
T = 1 s
T = 0 s
(b)
Figure 9 The visualized passing sequence (a) is the result of our approach and (b) represents the result of reactive approach withoutconsidering intention The black vehicle is an autonomous vehicle while the red car is the human-driven vehicle Each vehicle representsthe position in a specific time 119879 with an interval of 1 second
for each vehicle are used in different tests Besides twotypical situations including human-driven vehicle gettingthrough before or after autonomous vehicle are comparedin this section With the same initial state different reac-tions will occur based on various methods We compareour approach and reactive-based method [6] in this sec-tion The key difference for these two methods is that ourapproach considers human-driven vehiclersquos driving inten-tion
The first experiment is that human-driven vehicle triesto yield autonomous vehicle in the interaction process Theresults are shown in Figures 9 and 10 Firstly Figure 9 givesus a visual comparison of the different approaches Fromalmost the same initial state (eg position and velocity) ourapproach could lead to autonomous vehicle passing throughthe intersection more quickly and reasonable
Then let us look at Figure 10 for detailed explanationIn the first 12 s in Figures 10(a) and 10(c) autonomousvehicle maintains speed and understands that human-drivenvehicle will not perform yielding actions Then autonomousvehicle gets yielding intention of human-driven vehicle andunderstands that human-driven vehiclersquos lateral intentionis to go straight Based on candidate policies autonomousvehicle selects acceleration strategy with maximum rewardand finally crosses the intersection In this process we canobviously find that autonomous vehicle understands human-driven vehiclersquos yielding intention Figure 10(c) is an exampleof understand human-driven vehiclersquos behavior based on egovehiclersquos future actions in a specific time Our strategy pre-dicts the future actions of human-driven vehicle Althoughthe velocity curves after 1 s do not correspond it does notaffect the performance of our methods The reason is that
12 Mathematical Problems in Engineering
0 1 2 3 4 5 6 7 80
1020
Time (s)
Velo
city
(ms
) Velocity profile
0 1 2 3 4 5 6 7 8024
Time (s)
Inte
ntio
n Longitudinal intention
0 1 2 3 4 5 6 7 8024
Time (s)
Inte
ntio
n Lateral intention
(a)
0 1 2 3 4 5 6 7 8 90
1020
Time (s)
Velo
city
(ms
) Velocity profile
0 1 2 3 4 5 6 7 8 9024
Time (s)
Inte
ntio
n
Longitudinal intention
0 1 2 3 4 5 6 7 8 9024
Time (s)
Inte
ntio
n
Lateral intention
(b)
0 1 2 3 4 5 6 7 8minus60
minus40
minus20
0
20
40
60
80
Time (s)
Distance to collision region
Dist
ance
(m)
(c)
0 1 2 3 4 5 6 7 8 9minus10
0
10
20
30
40
50
60
70
80
Time (s)
Dist
ance
(m)
Distance to collision region
(d)
0 1 2 3 4 5 6 7 87
8
9
10
11
12
13
14
15
Time (s)
Velo
city
(ms
)
(e)
Time (s)0 1 2 3 4 5 6 7 8
0
2
4
6
8
10
12
14
Velo
city
(ms
)
(f)
Figure 10 Case test 1 In this case human-driven vehicle passes through intersection after autonomous vehicle (a) (c) and (e) are theperformance of our method while (b) (d) and (f) are from the strategy without considering the driving intention (a) and (b) are the velocityprofiles and the corresponding driving intention For longitudinal intention label 1 means yielding and label 2 means not yielding In lateralintention 1 means turning left 2 means turning right 3 means going straight and 4 means stop The intentions in (b) are not used in thatmethod but for detailed analysis (c) and (d) are the distance to collision area for autonomous vehicle and human-driven vehicle respectively(e) and (f) are the prediction and true motions of human-driven vehicles in time 15 s with a prediction length of 8 s The red curves in thesesubfigures are from autonomous vehicle while blue lines are from human-driven vehicle The green lines in (e) and (f) are the predictionvelocity curves of human-driven vehicle
Mathematical Problems in Engineering 13
T = 4 s
T = 1 s
T = 2 s
T = 3 s
T = 5 s
T = 0 s
T = 0 1 2 5 s
(a)
T = 4 s
T = 1 s
T = 2 s
T = 3 s
T = 5 s
T = 0 s
T = 0 1 2 5 s
(b)
Figure 11 The visualized passing sequence for the case of human-driven vehicle first getting through (a) is the result of our approach and(b) is the reactive-based approach
we use a deterministic model in the prediction process andthe prediction value is inside two boundaries to ensure safetyBesides the whole actions of autonomous vehicle in thisprocess could also help human-driven vehicle to understandnot-yielding intention of autonomous vehicles In this casecooperative driving behaviors are performed by both vehi-cles
However if the intention is not considered in this processwe can find the results in Figures 10(b) 10(d) and 10(f)After 2 s in Figure 10(b) while the human-driven vehiclegives a yielding intention autonomous vehicle could notunderstand and they find a potential collision based on theconstant velocity assumptions Then it decreases the speedbut the human-driven vehicle also slows down The puzzledbehavior leads both vehicles to slow down near intersectionFinally human-driven vehicle stops at the stop line and thenautonomous vehicle could pass the intersection In this strat-egy the human-driven vehiclersquos future motion is assumed tobe constant (Figure 10(f))Without understanding of human-driven vehiclersquos intentions this strategy can increase conges-tion problem
Another experiment is that human-driven vehicle tries toget through the intersection first The results are shown inFigures 11 and 12This case is quite typical because many traf-fic accidents in real world are happening in this situation Indetail if one vehicle tries to cross an intersection while violat-ing the law another vehicle will be in great danger if it doesnot understand its behavior From the visualized perform-ance in Figure 11 our method is a little more safe than otherapproaches as there is nearly collision situation in Figure11(b) In detail we can see from Figure 12(a) that our strategycould perform deceleration actions after we understand thenot-yielding intention in 08 sHowever without understand-ing human-driven vehiclersquos motion intention the responsetime has a 1-second delay which may be quite dangerous
In addition it is shown that good performance is in thepredictions of human-driven vehiclersquos future motion in ourmethods (Figure 12(e))
The results of these two cases demonstrate that our algo-rithm could deal with typical scenarios and have better per-formance than traditional reactive controller Autonomousvehicle could be driven more safely fast and comfortablythrough our strategy
7 Conclusion and Future Work
In this paper we proposed an autonomous driving decision-making algorithm considering human-driven vehiclersquos uncer-tain intentions in an uncontrolled intersection The lat-eral and longitudinal intentions are recognized by a con-tinuous HMM Based on HMM and POMDP we modelgeneral decision-making process and then use an approxi-mate approach to solve this complex problem Finally weuse PreScan software and a driving simulator to emulatesocial interaction process The experiment results show thatautonomous vehicles with our approach can pass throughuncontrolled intersections more safely and efficiently thanusing the strategy without considering human-driven vehi-clesrsquo driving intentions
In the near future we aim to implement our approach intoa real autonomous vehicle and perform real world experi-ments In additionmore precious intention recognition algo-rithm aims to be figured out Somemethods like probabilisticgraphic model can be used to get a distribution of each inten-tion Finally designing online POMDP planning algorithmsis also valuable
Competing Interests
The authors declare that there are no competing interestsregarding the publication of this paper
14 Mathematical Problems in Engineering
0 05 1 15 2 25 3 35 4 45 510
20
30
Time (s)
Velo
city
(ms
) Velocity profile
0 05 1 15 2 25 3 35 4 45 50
2
4
Time (s)
Inte
ntio
n
Longitudinal intention
0 05 1 15 2 25 3 35 4 45 5024
Inte
ntio
n Lateral intention
Time (s)
(a)
0 05 1 15 2 25 3 35 4 45 5102030
Time (s)
Velo
city
(ms
) Velocity profile
0 05 1 15 2 25 3 35 4 45 5024
Time (s)
Inte
ntio
n
Longitudinal intention
0 05 1 15 2 25 3 35 4 45 5024
Time (s)
Inte
ntio
n
Lateral intention
(b)
0 05 1 15 2 25 3 35 4 45 5
0
20
40
60
80
Dist
ance
(m)
Distance to collision region
Time (s)
minus40
minus20
(c)
0 05 1 15 2 25 3 35 4 45 5
0
20
40
60
80D
istan
ce (m
)
Distance to collision region
Time (s)
minus40
minus20
(d)
0 1 2 3 4 5 6 7 816
17
18
19
20
21
22
23
24
25
Velo
city
(ms
)
Time (s)
(e)
0 1 2 3 4 5 6 7 817
18
19
20
21
22
23
24
25
Time (s)
Velo
city
(ms
)
(f)
Figure 12 Case test 2 In this case human-driven vehicle passes through intersection before autonomous vehicle through different strategiesThe definition of each subfigure is the same as in Figure 10
Mathematical Problems in Engineering 15
Acknowledgments
This study is supported by the National Natural ScienceFoundation of China (no 91420203)
References
[1] C Urmson J Anhalt D Bagnell et al ldquoAutonomous driving inurban environments boss and the urban challengerdquo Journal ofField Robotics vol 25 no 8 pp 425ndash466 2008
[2] J Markoff ldquoGoogle cars drive themselves in trafficrdquo New YorkTimes vol 9 2010
[3] L R Rabiner and B-H Juang ldquoAn introduction to hiddenMarkov modelsrdquo IEEE ASSP Magazine vol 3 no 1 pp 4ndash161986
[4] M Buehler K Iagnemma and S Singh The DARPA UrbanChallenge Autonomous Vehicles in City Traffic vol 56 Springer2009
[5] S Kammel J Ziegler B Pitzer et al ldquoTeam AnnieWAYrsquos auto-nomous system for the 2007 DARPAUrban Challengerdquo Journalof Field Robotics vol 25 no 9 pp 615ndash639 2008
[6] C R Baker and J M Dolan ldquoTraffic interaction in the urbanchallenge putting boss on its best behaviorrdquo in Proceedingsof the IEEERSJ International Conference on Intelligent Robotsand Systems (IROS rsquo08) pp 1752ndash1758 IEEE Nice FranceSeptember 2008
[7] M Montemerlo J Becker S Shat et al ldquoJunior the Stanfordentry in the urban challengerdquo Journal of Field Robotics vol 25no 9 pp 569ndash597 2008
[8] L Fletcher S Teller E Olson et al ldquoTheMIT-Cornell collisionand why it happenedrdquo Journal of Field Robotics vol 25 no 10pp 775ndash807 2008
[9] M Bahram A Wolf M Aeberhard and D Wollherr ldquoAprediction-based reactive driving strategy for highly automateddriving function on freewaysrdquo in Proceedings of the 25th IEEEIntelligent Vehicles Symposium pp 400ndash406 IEEE DearbornMich USA June 2014
[10] J Wei J M Dolan and B Litkouhi ldquoA prediction- and costfunction-based algorithm for robust autonomous freeway driv-ingrdquo in Proceedings of the IEEE Intelligent Vehicles Symposium(IV rsquo10) pp 512ndash517 San Diego Calif USA June 2010
[11] J Wei J M Dolan and B Litkouhi ldquoAutonomous vehiclesocial behavior for highway entrance ramp managementrdquo inProceedings of the IEEE Intelligent Vehicles Symposium (IV rsquo13)pp 201ndash207 IEEE Gold Coast Australia June 2013
[12] H Bai S Cai N Ye D Hsu and W S Lee ldquoIntention-awareonline POMDP planning for autonomous driving in a crowdrdquoin Proceedings of the IEEE International Conference on Roboticsand Automation (ICRA rsquo15) pp 454ndash460 IEEE Seattle WashUSA May 2015
[13] A Somani N Ye D Hsu and W S Lee ldquoDespot onlinepomdp planning with regularizationrdquo in Advances in NeuralInformation Processing Systems pp 1772ndash1780 2013
[14] S Brechtel T Gindele and R Dillmann ldquoProbabilistic deci-sion-making under uncertainty for autonomous driving usingcontinuous POMDPsrdquo in Proceedings of the 17th IEEE Interna-tional Conference on Intelligent Transportation Systems (ITSCrsquo14) pp 392ndash399 Qingdao China October 2014
[15] C H Papadimitriou and J N Tsitsiklis ldquoThe complexity ofMarkov decision processesrdquo Mathematics of Operations Re-search vol 12 no 3 pp 441ndash450 1987
[16] O Madani S Hanks and A Condon ldquoOn the undecidabilityof probabilistic planning and related stochastic optimizationproblemsrdquoArtificial Intelligence vol 147 no 1-2 pp 5ndash34 2003
[17] S Ulbrich and M Maurer ldquoProbabilistic online POMDP deci-sion making for lane changes in fully automated drivingrdquo inProceedings of the 16th International IEEE Conference on Intelli-gent Transportation Systems (ITSC rsquo13) pp 2063ndash2067 TheHague The Netherlands October 2013
[18] A G Cunningham E Galceran R M Eustice and E OlsonldquoMPDM multipolicy decision-making in dynamic uncertainenvironments for autonomous drivingrdquo in Proceedings of theIEEE International Conference on Robotics and Automation(ICRA rsquo15) pp 1670ndash1677 Seattle Wash USA May 2015
[19] D A Reynolds and R C Rose ldquoRobust text-independentspeaker identification using Gaussian mixture speaker modelsrdquoIEEE Transactions on Speech and Audio Processing vol 3 no 1pp 72ndash83 1995
[20] L E Baum and T Petrie ldquoStatistical inference for probabilisticfunctions of finite state Markov chainsrdquo The Annals of Mathe-matical Statistics vol 37 pp 1554ndash1563 1966
[21] M Tideman and M Van Noort ldquoA simulation tool suite fordeveloping connected vehicle systemsrdquo in Proceedings of theIEEE Intelligent Vehicles Symposium (IEEE IV rsquo13) pp 713ndash718Queensland Australia June 2013
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Mathematical Problems in Engineering 5
Distance tointersection
Velocity
Acceleration
Yaw rate
Observe sequence
HMM_TR Forward algorithm
HMM_TL Forward algorithm
HMM_S Forward algorithm
HMM_GS Forward algorithm
DriverintentionO = B1 B2 BM
1205821 = (1205871 A1 C1 1205831 Σ1)
1205822 = (1205872 A2 C2 1205832 Σ2)
1205823 = (1205873 A3 C3 1205833 Σ4)
1205824 = (1205874 A4 C4 1205834 Σ4)
P(O | 1205822)
P(O | 1205821)
P(O | 1205824)
P(O | 1205824)
arg max P(O | 120582i)i
Figure 2 Prediction process for HMMThe observed sequence will be evaluated by four HMMs Forward algorithm is used to calculate theconditional probabilities and the intention corresponding to the largest value will be considered as the vehiclersquos intention
Time (s)1 2 3 4 5 6
024
0
Predict results
(a)
01 2 3 4 5 60
1020
Velocity
v (m
s)
Time (s)
(b)
0
1 2 3 4 5 60
10
minus10
Acceleration
a (m
s2)
Time (s)
(c)
Time (s)0 1 2 3 4 5 6
minus30minus20minus10
010
Yaw rate
Yaw
rate
(deg
s)
(d)
0 1 2 3 4 5 60
1020
Time (s)
Distance to intersection (DTI)
DTI
(m)
(e)
Figure 3 Lateral intention prediction example The true intention of human-driven vehicle is to turn left in this scenario In the first figurethe value 1 of the 119910 label means turn left 2 means turn right 3 represents go straight and 4 corresponds to stop
42 State Space Because of the Markov property sufficientinformation should be contained in the state space S fordecision-making process [14] The state space includes thevehicle pose [119909 119910 120579] velocity V the average yaw rate yawaveand acceleration 119886ave in the last planning period for allthe vehicles For the human-driven vehicles the lateral andlongitudinal intentions [119868lat 119868lon] also need to be containedfor state transition modeling However the road contextknowledge is static reference information so that it will be notadded to the state space
The joint state 119904 isin S could be denoted as 119904 = [119904host 1199041
1199042 119904
119873]119879 where 119904host is the state of host vehicle (auto-
nomous vehicle) 119904119894 119894 isin 1 2 3 119873 is the state of
human-driven vehicles and 119873 is the number of human-driven vehicles involved Let us define metric state 119909 =
[119909 119910 120579 V 119886ave yawave]119879 including the vehicle position head-
ing velocity acceleration and yaw rateThus the state of hostvehicle can be defined as 119904host = 119909host while the human-driven vehicle state 119904
119894is 119904119894= [119909
119894 119868lat119894 119868lon119894]
119879 With theadvanced perception system and V2V communication tech-nology we assume that the metric state 119909 could be observedBecause the sensor noise is small and hardly affects decision-making process we do not model observation noise for themetric state However the intention state cannot be directlyobserved so it is the partially observable variables in ourpaper The intention state should be inferred from obser-vation data and predictive model over time
43 Action Space In our autonomous vehicle system thedecision-making system is used to select the suitable tactical
6 Mathematical Problems in Engineering
Time (s)0 1 2 3 4 5 6 7 8
012
Predict results
(a)
0 1 2 3 4 5 6 7 80
1020
Relative velocity (RV)
RV (m
s)
Time (s)
(b)
0 1 2 3 4 5 6 7 8minus2
02
Relative acceleration (RA)
RA (m
s2)
Time (s)
(c)
0 1 2 3 4 5 6 7
02040
Time (s)
minus20
Relative DTC (RDTC)
RDTC
(m)
(d)
Figure 4 One example of predicting longitudinal intentions This example is based on the scenario of Figure 1 and two vehicles both gostraightThe value 1 of 119910-axis in the first figure denotes the intention of yielding while 2 represents not yielding In the first 28 s the intentionis yielding After that due to the acceleration action and less relative DTC autonomous vehicle could understand human-driven vehiclersquosnot-yielding intention
maneuvers Specifically in the intersection area autonomousvehicles should follow a global reference path generated bypath planning module The decision-making module onlyneeds to generate accelerationdeceleration commands to thecontrol layer As the reference path may not be straight thesteering control module can adjust the front wheel angleto follow the reference path Therefore the action space Acould be defined as a discrete set A = [acc dec con] whichcontains commands including acceleration deceleration andmaintaining current velocity
44 Observation Space Similar to the joint state space theobservation 119911 is denoted as 119911 = [119911host 1199111 1199112 119911119873]
119879 where119911host and 119911119894 are the host vehicle and human-driven vehiclersquosobservations respectively The acceleration and yaw rate canbe approximately calculated by speed and heading in theconsecutive states
45 State Transition Model In state transition process weneed to model transition probability Pr(1199041015840 | 119904 119886) Thisprobability is determined by each targeted element in thescenario So the transition model can be calculated by thefollowing probabilistic equation
Pr (1199041015840 | 119904 119886) = Pr (1199041015840host | 119904host 119886host)119873
prod
119894=1
Pr (1199041015840119894| 119904119894) (10)
In the decision-making layer we do not need to considercomplex vehicle dynamic model Thus the host vehiclersquosmotion Pr(1199041015840host | 119904host 119886host) can be simply represented by thefollowing equations given action 119886
1199091015840
= 119909 + (V +119886Δ119905
2)Δ119905 cos (120579 + Δ120579)
1199101015840
= 119910 + (V +119886Δ119905
2)Δ119905 sin (120579 + Δ120579)
1205791015840
= 120579 + Δ120579
V1015840 = V + 119886Δ119905
yaw1015840ave =Δ120579
Δ119905
1198861015840
ave = 119886
(11)
Thus the key problem is converted to compute Pr(1199041015840119894|
119904119894) the state transition probability of human-driven vehicles
Based on the total probability formula this probability can befactorized as a sum in whole action space
Pr (1199041015840119894| 119904119894) = sum
119886119894
Pr (1199041015840119894| 119904119894 119886119894)Pr (119886
119894| 119904119894) (12)
With this equation we only need to calculate the statetransition probability Pr(1199041015840
119894| 119904119894 119886119894) given a specific action 119886
119894
and the probability of selecting this action Pr(119886119894| 119904119894) under
current state 119904119894
Because the human-driven vehiclesrsquo state 119904119894= [119909119894 119868119894] the
probability Pr(1199041015840119894| 119904119894 119886119894) can be calculated as
Pr (1199041015840119894| 119904119894 119886119894) = Pr (1199091015840
119894 1198681015840
119894| 119909119894 119868119894 119886119894)
= Pr (1199091015840119894| 119909119894 119868119894 119886119894)Pr (1198681015840
119894| 1199091015840
119894 119909119894 119868119894 119886119894)
(13)
With a certain action 119886119894 Pr(1199091015840119894| 119909119894 119868119894 119886119894) is equal to Pr(1199091015840
119894|
119909119894 119868lat119894 119886119894)The lateral behavior 119868lat119894 is considered to be a goal-
directed driving intention which will not be changed in thedriving process So Pr(1199091015840
119894| 119909119894 119868lat119894 119886119894) is equal to Pr(119909
1015840
119894| 119909119894 119886119894)
given a reference path corresponding to the intention of 119868lat119894Using (11) Pr(1199091015840
119894| 119909119894 119886119894) can be well solved
The remaining problem for calculating Pr(1199041015840119894| 119904119894 119886119894) is
to deal with Pr(1198681015840119894| 1199091015840
119894 119909119894 119868119894 119886119894) The lateral intention 1198681015840lat119894
is assumed stable through the above explanation And the
Mathematical Problems in Engineering 7
longitudinal intention 1198681015840
lon119894 is assumed to be not updatedin this process But it will be updated with new inputs inobservation space
Now Pr(1199041015840119894| 119904119894 119886119894) is well modeled and the remaining
problem is to compute the probabilities Pr(119886119894| 119904119894) of human-
driven vehiclesrsquo future actions
Pr (119886119894| 119904119894) = Pr (119886
119894| 119909119894 119868119894)
= sum
1199091015840
host
Pr (119886119894| 1199091015840
host 119909119894 119868119894)Pr (1199091015840
host | 119909119894 119868119894) (14)
Because 1199091015840
host is determined by the designed policyPr(1199091015840host | 119909119894 119868119894) could be calculated by (11) given an action119886host The probability Pr(119886
119894| 1199091015840
host 119909119894 119868119894) means the distribu-tion of human-driven vehiclesrsquo actions given the new state1199091015840
host of host vehicle the current state of itself and itsintentions Instead of building a complex probability modelwe designed a deterministic mechanism to calculate themostpossible action 119886
119894given 1199091015840host 119909119894 and 119868119894
In this prediction process the host vehicle is assumed tobe maintaining the current actions in the next time step andthe action 119886
119894will be leading human-driven vehicle passing
through the potential collision area either in advance of hostvehicle under the intention 119868
119873Yield or behind the host vehicleunder the intention 119868Yield to keep a safe distance 119889safe In thecase with the intention of 119868
119873Yield we can calculate the lowboundary 119886
119894low of 119886119894 through the above process anddeterminethe upper one using the largest comfort value 119886
119894comfort If119886119894comfort lt 119886
119894low 119886119894low will be used as the human-drivenvehiclersquos action If not we consider the targeted 119886
119894following
a normal distribution with mean value 120583119886119894
between 119886119894low and
119886119894comfort To simplify our model we use the mean value ofthese two boundaries to represent human-driven vehiclersquosaction 119886
119894 Similarly the case with the intention of 119868Yield can
be analyzed in the same processAfter these steps the transition probability Pr(1199041015840 | 119904 119886)
is well formulized and the autonomous vehicle could havethe ability to understand the future motion of the scenariothrough this model
46 Observation Model The observation model is built tosimulate the measurement process The motion intentionis updated in this process The measurements of human-driven vehicles are modeled with conditional independentassumption Thus the observation model can be calculatedas
Pr (119911 | 119886 1199041015840) = Pr (119911host | 1199041015840
host)119873
prod
119894=1
Pr (119911119894| 1199041015840
119894) (15)
The host vehiclersquos observation function is denoted as
Pr (119911host | 1199041015840
host) simN (119911host | 1199091015840
host Σ119911host) (16)
But in this paper due to the use of V2V communicationsensor the observation error almost does not affect the deci-sion-making result The variance matrix is set as zero
The human-driven vehiclersquos observation will follow thevehiclersquos motion intentions Because we do not consider
the observation error the value in metric state will be thesame as the state transition results But the longitudinalintention of human-driven vehicles in the state space will beupdated using the new observations andHMMmentioned inSection 3 The new observation space will be confirmed withthe above step
47 Reward Function The candidate policies have to satisfyseveral evaluation criterions Autonomous vehicles should bedriven safely and comfortably At the same time they shouldfollow the traffic rules and reach the destination as soon aspossible As a result we design objective function (17) consid-ering three aspects including safety time efficiency and trafficlaws where 120583
1 1205832 and 120583
3are the weight coefficient
119877 (119904 119886) = 1205831119877safety (119904 119886) + 1205832119877time (119904 119886)
+ 1205833119877law (119904 119886)
(17)
The detailed information will be discussed in the follow-ing subsections In addition the factor of comfort will be con-sidered and discussed in policy generation part (Section 51)
471 Safety Reward The safety reward function 119877safety(119904 119886)is based on the potential conflict status In our strategysafety reward is defined as a penalty If there are no potentialconflicts the safety rewardwill be set as 0 A large penalty willbe assigned due to the risk of collision status
In an uncontrolled intersection the four approachingdirections are defined as 119860
119894isin 1 2 3 4 (Figure 5) The
driverrsquos lateral intentions are defined as 119868lat isin 119868TR 119868TL 119868GS119868S So the driving trajectory for each vehicle in the inter-section can be generally represented by 119860
119894and 119868lat119895 and we
marked it as 119879119860119894119868lat119895
1 le 119894 le 4 1 le 119895 le 4 The function 119865 isused to judge the potential collision status which is denotedas
119865 (119879119909 119879119910) =
1 if potential conflict
0 otherwise(18)
where 119879119909and 119879
119910are vehiclesrsquo maneuver 119879
119860119894119868lat119895
119865(119909 119910) can be calculated through relative direction
between two cars which is shown in Table 1The safety reward is based on the following items(i) If 119865(119909 119910) is equal to 0 then the safety reward is equal
to 0 due to the noncollision status(ii) If potential collision occurs there will be a large
penalty(iii) If |TTC
119894minus TTChost| lt 119905threshod there is a penalty
depending on |TTC119894minus TTChost| and TTChost
472 Traffic Law Reward Autonomous vehicles should fol-low traffic laws to interactwith human-driven vehicles Trafficlaw is modeled as a function Law(119909 119910) for each two vehicles119909 and 119910
Law (119879119909 119879119910) =
1 if 119909 is prior
0 otherwise(19)
8 Mathematical Problems in Engineering
Human-driven vehicle
Autonomous vehicle
1
4
3
2
Figure 5 One typical scenario for calculating safety reward
Table 1 Safe condition judgments in the intersection
Human-driven vehicle Left side Right side Opposite side Same sideDriving direction TL LK TR S TL LK TR S TL LK TR S TL LK TR S
Turn left permil permil I I permil permil I I permil permil permil I I I I ILane keeping permil permil I I permil permil permil I permil I I I I I I ITurn right I permil I I I I I I permil I I I I I I IStop I I I I I I I I I I I I I I I I
permil indicates potential collision I indicates no potential collision
where 119879119909and 119879
119910are vehiclesrsquo maneuver 119879
119860119894119878119895
This functionLaw(119879
119909 119879119910) is formulized as shown in Algorithm 1
If the behavior will break the law a large penalty is appliedand the behavior of obeying traffic laws will get a zero reward
473 Time Reward The time cost is based on the time to thedestination for the targeted vehicles in the intersection area
Costtime =DTGVhost
(20)
DTG is the distance to the driving goal In addition wealso need to consider the speed limit which is discussed inpolicy generation part in Section 5
5 Approximations on SolvingPOMDP Problem
Solving POMDP is quite difficult The complexity of search-ing total brief space is O(|A|119867|Z|
119867
) [12] where 119867 is theprediction horizon In this paper we model the intentionrecognition process as a deterministic model and use com-munication sensors to ignore the perception error and thusthe size of |Z| is reduced to 1 in the simplified problem Tosolve this problem we first generate the suitable potentialpolicies according to the property of driving tasks and then
select the reasonable total predicting interval time and totalhorizon After that the approximate optimal policy canbe calculated through searching all possible policies withmaximum total rewardThe policy selection process is shownin Algorithm 2 and some detailed explanations are discussedin the subsections
51 Policy Generation For autonomous driving near inter-section the desired velocity curves need to satisfy severalconstraints Firstly except for emergency braking the accel-eration constraints are applied to ensure comfort Secondlythe speed limit constraints should be used in this process Weaim to avoid the acceleration commands when autonomousvehicle is reaching maximum speed limit Thirdly for thecomfort purpose the acceleration command should not bealways changed In otherwords we need tominimize the jerk
Similar to [11] the candidate policies are divided intothree time segments The first two segments are like ldquokeepconstant accelerationdeceleration actionsrdquo while keepingconstant velocity in the third segment We use 119905
1 1199052 and 119905
3
to represent the time periods of these three segments Toguarantee comfort the acceleration is limited to the rangefrom minus4ms2 to 2ms2 and we discrete acceleration actioninto a multiple of [minus05 05 0] Then the action space can berepresented by a discretizing acceleration setThenwe can set
Mathematical Problems in Engineering 9
Law(119879119909 119879119910) larr 0
119905119909larr 1198891199092119868V119909
119905119910larr 1198891199102119868V119910
if 119905119909lt 119905119910minus Δ119905 then
Law(119879119909 119879119910) larr 1
else if |119905119909minus 119905119910| lt Δ119905 then
statuslarr 119865(119879119909 119879119910)
if status = 1 thenif 119868lat119909 = lanekeeping and 119868lat119910 ltgt lanekeeping thenLaw(119879
119909 119879119910) larr 1
else if 119868lat119909 119868lat119910 = lanekeeping and 119860119909minus 119860119910= 1 or minus3 then
Law(119879119909 119879119910) larr 1
else if 119868lat119909 = turnleft and 119868lat119910 = turnright thenLaw(119879
119909 119879119910) larr 1
end ifend if
end ifreturn Law(119879
119909 119879119910)
Algorithm 1 Traffic law formulization
InputPredict horizon119867 time step Δ119905Current states 119904host = 119909host 119904human = [119909human 119868human]
(1) 119875 larr genenratepolicyset()(2) for each 120587
119896isin 119875 do
(3) for 119894 = 1 to119867Δ119905 do(4) 119886host larr 120587
119896(119894)
(5) 1199041015840
host larr updatestate(119904host 119886host)(6) 119886human larr predictactions(1199041015840host 119904human 119868human)
(7) 1199091015840
human larr updatestate(119909human 119886human)
(8) 1198681015840
human larr updateintention(1199041015840host 1199091015840
human)
(9) 1199041015840
human larr [1199091015840
human 1198681015840
human]
(10) 119877(119894) = calculatereward(1199041015840host 1199041015840
human)
(11) 119904host larr 1199041015840
host(12) 119904human larr 119904
1015840
human(13) end(14) 119877
total119896
larr sum119894sum(119877(119894))
(15) end(16) 119896lowast larr argmax
119896(119877
total119896
)
(17) 120587lowast larr 120587119896lowast
(18) return 120587lowast
Algorithm 2 Policy selection process
the value of 1199051 1199052 and 119905
3and the prediction period of single
step An example of policy generation is shown in Figure 6
52 Planning Horizon Selection After building policy gener-ation model the next problem is to select a suitable planninghorizon Longer horizon can lead to a better solution but con-suming more computing resources However as our purpose
is to deal with the interaction problem in the uncontrolledintersection we only need to consider the situation beforeautonomous vehicle gets through In our algorithmwe set theprediction horizon as 8 seconds In addition in the process ofupdating the future state of each vehicle using each policy thecar following mode is used after autonomous vehicle passesthrough the intersection area
10 Mathematical Problems in Engineering
0
1
2Candidate policies
minus4
minus3
minus2
minus1
Acce
lera
tion
(ms2)
1 2 3 4 5 6 7 80Time (s)
(a)
Time (s)0 1 2 3 4 5 6 7 8
0
2
4
6
8
10
12
14
16
18
20
Velo
city
(ms
)
Velocity strategies
(b)
Figure 6 An example of policy generation process (a) is the generated policies and (b) is the corresponding speed profiles The interval ofeach prediction step is 05 s current speed is 12ms2 and the speed limit is 20ms2 The bold black line is one policy In the first 3 secondsautonomous vehicles decelerate in minus35ms2 then accelerate at 2ms2 for 4 seconds and finally stop in the last one second In this case 109policies were generated which is suitable for replanning fast
AB
Figure 7 Testing scenario Autonomous vehicle B and human-driven vehicle A are both approaching the uncontrolled intersection To goacross the intersection successfully autonomous vehicle should interact with human-driven vehicle
53 Time Step Selection Another problem is the predic-tion time step The intention prediction algorithm and thePOMDP are computed in each step If the time step is 119905stepthe total computation timeswill be119867119905stepThus smaller timestep leads to more computation time To solve this problemwe use a simple adaptive time step calculation mechanism togive a final value The time step is selected based on the TTCof autonomous vehicle If the host vehicle is far away from theintersection we can use a very large time step But if the TTCis quite small the low 119905step is applied to ensure safety
6 Experiment and Results
61 Settings In this paper we evaluate our approach throughPreScan 710 [21] a simulation tool for autonomous drivingand connected vehicles Using this software we can build thetesting scenarios (Figure 7) and add vehicles with dynamicmodel In order to get a similar scenario considering social
interaction the driver simulator is added in our experiment(Figure 8) The human-driven vehicle is driven by severalpeople during the experiment and the autonomous vehiclemakes decisions based on the human-driven vehiclersquos drivingbehavior The reference trajectory for autonomous vehicleis generated from path planning module and the human-driven vehiclersquos data (eg position velocity and heading)are transferred through V2V communication sensor Thedecision-making module sends desired velocity command tothe PID controlled to follow the reference path All policies inthe experiment part use a planning horizon119867 = 8 s which isdiscretized into the time step of 05 s
62 Results It is difficult to compare different approachesin the same scenario because the environment is dynamicand not exactly the same However we select two typicalsituations and special settings to make it possible The sameinitial conditions including position orientation and velocity
Mathematical Problems in Engineering 11
Figure 8 Logitech G27 driving simulator
T = 0 1 2 6 s T = 5 s
T = 4 s
T = 3 s
T = 2 s
T = 1 s
T = 0 s
T = 6 s
(a)
T = 0 1 2 9 s
T = 4 s
T = 5 s
T = 6 sT = 7 sT = 8 s
T = 9 s
T = 3 s
T = 2 s
T = 1 s
T = 0 s
(b)
Figure 9 The visualized passing sequence (a) is the result of our approach and (b) represents the result of reactive approach withoutconsidering intention The black vehicle is an autonomous vehicle while the red car is the human-driven vehicle Each vehicle representsthe position in a specific time 119879 with an interval of 1 second
for each vehicle are used in different tests Besides twotypical situations including human-driven vehicle gettingthrough before or after autonomous vehicle are comparedin this section With the same initial state different reac-tions will occur based on various methods We compareour approach and reactive-based method [6] in this sec-tion The key difference for these two methods is that ourapproach considers human-driven vehiclersquos driving inten-tion
The first experiment is that human-driven vehicle triesto yield autonomous vehicle in the interaction process Theresults are shown in Figures 9 and 10 Firstly Figure 9 givesus a visual comparison of the different approaches Fromalmost the same initial state (eg position and velocity) ourapproach could lead to autonomous vehicle passing throughthe intersection more quickly and reasonable
Then let us look at Figure 10 for detailed explanationIn the first 12 s in Figures 10(a) and 10(c) autonomousvehicle maintains speed and understands that human-drivenvehicle will not perform yielding actions Then autonomousvehicle gets yielding intention of human-driven vehicle andunderstands that human-driven vehiclersquos lateral intentionis to go straight Based on candidate policies autonomousvehicle selects acceleration strategy with maximum rewardand finally crosses the intersection In this process we canobviously find that autonomous vehicle understands human-driven vehiclersquos yielding intention Figure 10(c) is an exampleof understand human-driven vehiclersquos behavior based on egovehiclersquos future actions in a specific time Our strategy pre-dicts the future actions of human-driven vehicle Althoughthe velocity curves after 1 s do not correspond it does notaffect the performance of our methods The reason is that
12 Mathematical Problems in Engineering
0 1 2 3 4 5 6 7 80
1020
Time (s)
Velo
city
(ms
) Velocity profile
0 1 2 3 4 5 6 7 8024
Time (s)
Inte
ntio
n Longitudinal intention
0 1 2 3 4 5 6 7 8024
Time (s)
Inte
ntio
n Lateral intention
(a)
0 1 2 3 4 5 6 7 8 90
1020
Time (s)
Velo
city
(ms
) Velocity profile
0 1 2 3 4 5 6 7 8 9024
Time (s)
Inte
ntio
n
Longitudinal intention
0 1 2 3 4 5 6 7 8 9024
Time (s)
Inte
ntio
n
Lateral intention
(b)
0 1 2 3 4 5 6 7 8minus60
minus40
minus20
0
20
40
60
80
Time (s)
Distance to collision region
Dist
ance
(m)
(c)
0 1 2 3 4 5 6 7 8 9minus10
0
10
20
30
40
50
60
70
80
Time (s)
Dist
ance
(m)
Distance to collision region
(d)
0 1 2 3 4 5 6 7 87
8
9
10
11
12
13
14
15
Time (s)
Velo
city
(ms
)
(e)
Time (s)0 1 2 3 4 5 6 7 8
0
2
4
6
8
10
12
14
Velo
city
(ms
)
(f)
Figure 10 Case test 1 In this case human-driven vehicle passes through intersection after autonomous vehicle (a) (c) and (e) are theperformance of our method while (b) (d) and (f) are from the strategy without considering the driving intention (a) and (b) are the velocityprofiles and the corresponding driving intention For longitudinal intention label 1 means yielding and label 2 means not yielding In lateralintention 1 means turning left 2 means turning right 3 means going straight and 4 means stop The intentions in (b) are not used in thatmethod but for detailed analysis (c) and (d) are the distance to collision area for autonomous vehicle and human-driven vehicle respectively(e) and (f) are the prediction and true motions of human-driven vehicles in time 15 s with a prediction length of 8 s The red curves in thesesubfigures are from autonomous vehicle while blue lines are from human-driven vehicle The green lines in (e) and (f) are the predictionvelocity curves of human-driven vehicle
Mathematical Problems in Engineering 13
T = 4 s
T = 1 s
T = 2 s
T = 3 s
T = 5 s
T = 0 s
T = 0 1 2 5 s
(a)
T = 4 s
T = 1 s
T = 2 s
T = 3 s
T = 5 s
T = 0 s
T = 0 1 2 5 s
(b)
Figure 11 The visualized passing sequence for the case of human-driven vehicle first getting through (a) is the result of our approach and(b) is the reactive-based approach
we use a deterministic model in the prediction process andthe prediction value is inside two boundaries to ensure safetyBesides the whole actions of autonomous vehicle in thisprocess could also help human-driven vehicle to understandnot-yielding intention of autonomous vehicles In this casecooperative driving behaviors are performed by both vehi-cles
However if the intention is not considered in this processwe can find the results in Figures 10(b) 10(d) and 10(f)After 2 s in Figure 10(b) while the human-driven vehiclegives a yielding intention autonomous vehicle could notunderstand and they find a potential collision based on theconstant velocity assumptions Then it decreases the speedbut the human-driven vehicle also slows down The puzzledbehavior leads both vehicles to slow down near intersectionFinally human-driven vehicle stops at the stop line and thenautonomous vehicle could pass the intersection In this strat-egy the human-driven vehiclersquos future motion is assumed tobe constant (Figure 10(f))Without understanding of human-driven vehiclersquos intentions this strategy can increase conges-tion problem
Another experiment is that human-driven vehicle tries toget through the intersection first The results are shown inFigures 11 and 12This case is quite typical because many traf-fic accidents in real world are happening in this situation Indetail if one vehicle tries to cross an intersection while violat-ing the law another vehicle will be in great danger if it doesnot understand its behavior From the visualized perform-ance in Figure 11 our method is a little more safe than otherapproaches as there is nearly collision situation in Figure11(b) In detail we can see from Figure 12(a) that our strategycould perform deceleration actions after we understand thenot-yielding intention in 08 sHowever without understand-ing human-driven vehiclersquos motion intention the responsetime has a 1-second delay which may be quite dangerous
In addition it is shown that good performance is in thepredictions of human-driven vehiclersquos future motion in ourmethods (Figure 12(e))
The results of these two cases demonstrate that our algo-rithm could deal with typical scenarios and have better per-formance than traditional reactive controller Autonomousvehicle could be driven more safely fast and comfortablythrough our strategy
7 Conclusion and Future Work
In this paper we proposed an autonomous driving decision-making algorithm considering human-driven vehiclersquos uncer-tain intentions in an uncontrolled intersection The lat-eral and longitudinal intentions are recognized by a con-tinuous HMM Based on HMM and POMDP we modelgeneral decision-making process and then use an approxi-mate approach to solve this complex problem Finally weuse PreScan software and a driving simulator to emulatesocial interaction process The experiment results show thatautonomous vehicles with our approach can pass throughuncontrolled intersections more safely and efficiently thanusing the strategy without considering human-driven vehi-clesrsquo driving intentions
In the near future we aim to implement our approach intoa real autonomous vehicle and perform real world experi-ments In additionmore precious intention recognition algo-rithm aims to be figured out Somemethods like probabilisticgraphic model can be used to get a distribution of each inten-tion Finally designing online POMDP planning algorithmsis also valuable
Competing Interests
The authors declare that there are no competing interestsregarding the publication of this paper
14 Mathematical Problems in Engineering
0 05 1 15 2 25 3 35 4 45 510
20
30
Time (s)
Velo
city
(ms
) Velocity profile
0 05 1 15 2 25 3 35 4 45 50
2
4
Time (s)
Inte
ntio
n
Longitudinal intention
0 05 1 15 2 25 3 35 4 45 5024
Inte
ntio
n Lateral intention
Time (s)
(a)
0 05 1 15 2 25 3 35 4 45 5102030
Time (s)
Velo
city
(ms
) Velocity profile
0 05 1 15 2 25 3 35 4 45 5024
Time (s)
Inte
ntio
n
Longitudinal intention
0 05 1 15 2 25 3 35 4 45 5024
Time (s)
Inte
ntio
n
Lateral intention
(b)
0 05 1 15 2 25 3 35 4 45 5
0
20
40
60
80
Dist
ance
(m)
Distance to collision region
Time (s)
minus40
minus20
(c)
0 05 1 15 2 25 3 35 4 45 5
0
20
40
60
80D
istan
ce (m
)
Distance to collision region
Time (s)
minus40
minus20
(d)
0 1 2 3 4 5 6 7 816
17
18
19
20
21
22
23
24
25
Velo
city
(ms
)
Time (s)
(e)
0 1 2 3 4 5 6 7 817
18
19
20
21
22
23
24
25
Time (s)
Velo
city
(ms
)
(f)
Figure 12 Case test 2 In this case human-driven vehicle passes through intersection before autonomous vehicle through different strategiesThe definition of each subfigure is the same as in Figure 10
Mathematical Problems in Engineering 15
Acknowledgments
This study is supported by the National Natural ScienceFoundation of China (no 91420203)
References
[1] C Urmson J Anhalt D Bagnell et al ldquoAutonomous driving inurban environments boss and the urban challengerdquo Journal ofField Robotics vol 25 no 8 pp 425ndash466 2008
[2] J Markoff ldquoGoogle cars drive themselves in trafficrdquo New YorkTimes vol 9 2010
[3] L R Rabiner and B-H Juang ldquoAn introduction to hiddenMarkov modelsrdquo IEEE ASSP Magazine vol 3 no 1 pp 4ndash161986
[4] M Buehler K Iagnemma and S Singh The DARPA UrbanChallenge Autonomous Vehicles in City Traffic vol 56 Springer2009
[5] S Kammel J Ziegler B Pitzer et al ldquoTeam AnnieWAYrsquos auto-nomous system for the 2007 DARPAUrban Challengerdquo Journalof Field Robotics vol 25 no 9 pp 615ndash639 2008
[6] C R Baker and J M Dolan ldquoTraffic interaction in the urbanchallenge putting boss on its best behaviorrdquo in Proceedingsof the IEEERSJ International Conference on Intelligent Robotsand Systems (IROS rsquo08) pp 1752ndash1758 IEEE Nice FranceSeptember 2008
[7] M Montemerlo J Becker S Shat et al ldquoJunior the Stanfordentry in the urban challengerdquo Journal of Field Robotics vol 25no 9 pp 569ndash597 2008
[8] L Fletcher S Teller E Olson et al ldquoTheMIT-Cornell collisionand why it happenedrdquo Journal of Field Robotics vol 25 no 10pp 775ndash807 2008
[9] M Bahram A Wolf M Aeberhard and D Wollherr ldquoAprediction-based reactive driving strategy for highly automateddriving function on freewaysrdquo in Proceedings of the 25th IEEEIntelligent Vehicles Symposium pp 400ndash406 IEEE DearbornMich USA June 2014
[10] J Wei J M Dolan and B Litkouhi ldquoA prediction- and costfunction-based algorithm for robust autonomous freeway driv-ingrdquo in Proceedings of the IEEE Intelligent Vehicles Symposium(IV rsquo10) pp 512ndash517 San Diego Calif USA June 2010
[11] J Wei J M Dolan and B Litkouhi ldquoAutonomous vehiclesocial behavior for highway entrance ramp managementrdquo inProceedings of the IEEE Intelligent Vehicles Symposium (IV rsquo13)pp 201ndash207 IEEE Gold Coast Australia June 2013
[12] H Bai S Cai N Ye D Hsu and W S Lee ldquoIntention-awareonline POMDP planning for autonomous driving in a crowdrdquoin Proceedings of the IEEE International Conference on Roboticsand Automation (ICRA rsquo15) pp 454ndash460 IEEE Seattle WashUSA May 2015
[13] A Somani N Ye D Hsu and W S Lee ldquoDespot onlinepomdp planning with regularizationrdquo in Advances in NeuralInformation Processing Systems pp 1772ndash1780 2013
[14] S Brechtel T Gindele and R Dillmann ldquoProbabilistic deci-sion-making under uncertainty for autonomous driving usingcontinuous POMDPsrdquo in Proceedings of the 17th IEEE Interna-tional Conference on Intelligent Transportation Systems (ITSCrsquo14) pp 392ndash399 Qingdao China October 2014
[15] C H Papadimitriou and J N Tsitsiklis ldquoThe complexity ofMarkov decision processesrdquo Mathematics of Operations Re-search vol 12 no 3 pp 441ndash450 1987
[16] O Madani S Hanks and A Condon ldquoOn the undecidabilityof probabilistic planning and related stochastic optimizationproblemsrdquoArtificial Intelligence vol 147 no 1-2 pp 5ndash34 2003
[17] S Ulbrich and M Maurer ldquoProbabilistic online POMDP deci-sion making for lane changes in fully automated drivingrdquo inProceedings of the 16th International IEEE Conference on Intelli-gent Transportation Systems (ITSC rsquo13) pp 2063ndash2067 TheHague The Netherlands October 2013
[18] A G Cunningham E Galceran R M Eustice and E OlsonldquoMPDM multipolicy decision-making in dynamic uncertainenvironments for autonomous drivingrdquo in Proceedings of theIEEE International Conference on Robotics and Automation(ICRA rsquo15) pp 1670ndash1677 Seattle Wash USA May 2015
[19] D A Reynolds and R C Rose ldquoRobust text-independentspeaker identification using Gaussian mixture speaker modelsrdquoIEEE Transactions on Speech and Audio Processing vol 3 no 1pp 72ndash83 1995
[20] L E Baum and T Petrie ldquoStatistical inference for probabilisticfunctions of finite state Markov chainsrdquo The Annals of Mathe-matical Statistics vol 37 pp 1554ndash1563 1966
[21] M Tideman and M Van Noort ldquoA simulation tool suite fordeveloping connected vehicle systemsrdquo in Proceedings of theIEEE Intelligent Vehicles Symposium (IEEE IV rsquo13) pp 713ndash718Queensland Australia June 2013
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
6 Mathematical Problems in Engineering
Time (s)0 1 2 3 4 5 6 7 8
012
Predict results
(a)
0 1 2 3 4 5 6 7 80
1020
Relative velocity (RV)
RV (m
s)
Time (s)
(b)
0 1 2 3 4 5 6 7 8minus2
02
Relative acceleration (RA)
RA (m
s2)
Time (s)
(c)
0 1 2 3 4 5 6 7
02040
Time (s)
minus20
Relative DTC (RDTC)
RDTC
(m)
(d)
Figure 4 One example of predicting longitudinal intentions This example is based on the scenario of Figure 1 and two vehicles both gostraightThe value 1 of 119910-axis in the first figure denotes the intention of yielding while 2 represents not yielding In the first 28 s the intentionis yielding After that due to the acceleration action and less relative DTC autonomous vehicle could understand human-driven vehiclersquosnot-yielding intention
maneuvers Specifically in the intersection area autonomousvehicles should follow a global reference path generated bypath planning module The decision-making module onlyneeds to generate accelerationdeceleration commands to thecontrol layer As the reference path may not be straight thesteering control module can adjust the front wheel angleto follow the reference path Therefore the action space Acould be defined as a discrete set A = [acc dec con] whichcontains commands including acceleration deceleration andmaintaining current velocity
44 Observation Space Similar to the joint state space theobservation 119911 is denoted as 119911 = [119911host 1199111 1199112 119911119873]
119879 where119911host and 119911119894 are the host vehicle and human-driven vehiclersquosobservations respectively The acceleration and yaw rate canbe approximately calculated by speed and heading in theconsecutive states
45 State Transition Model In state transition process weneed to model transition probability Pr(1199041015840 | 119904 119886) Thisprobability is determined by each targeted element in thescenario So the transition model can be calculated by thefollowing probabilistic equation
Pr (1199041015840 | 119904 119886) = Pr (1199041015840host | 119904host 119886host)119873
prod
119894=1
Pr (1199041015840119894| 119904119894) (10)
In the decision-making layer we do not need to considercomplex vehicle dynamic model Thus the host vehiclersquosmotion Pr(1199041015840host | 119904host 119886host) can be simply represented by thefollowing equations given action 119886
1199091015840
= 119909 + (V +119886Δ119905
2)Δ119905 cos (120579 + Δ120579)
1199101015840
= 119910 + (V +119886Δ119905
2)Δ119905 sin (120579 + Δ120579)
1205791015840
= 120579 + Δ120579
V1015840 = V + 119886Δ119905
yaw1015840ave =Δ120579
Δ119905
1198861015840
ave = 119886
(11)
Thus the key problem is converted to compute Pr(1199041015840119894|
119904119894) the state transition probability of human-driven vehicles
Based on the total probability formula this probability can befactorized as a sum in whole action space
Pr (1199041015840119894| 119904119894) = sum
119886119894
Pr (1199041015840119894| 119904119894 119886119894)Pr (119886
119894| 119904119894) (12)
With this equation we only need to calculate the statetransition probability Pr(1199041015840
119894| 119904119894 119886119894) given a specific action 119886
119894
and the probability of selecting this action Pr(119886119894| 119904119894) under
current state 119904119894
Because the human-driven vehiclesrsquo state 119904119894= [119909119894 119868119894] the
probability Pr(1199041015840119894| 119904119894 119886119894) can be calculated as
Pr (1199041015840119894| 119904119894 119886119894) = Pr (1199091015840
119894 1198681015840
119894| 119909119894 119868119894 119886119894)
= Pr (1199091015840119894| 119909119894 119868119894 119886119894)Pr (1198681015840
119894| 1199091015840
119894 119909119894 119868119894 119886119894)
(13)
With a certain action 119886119894 Pr(1199091015840119894| 119909119894 119868119894 119886119894) is equal to Pr(1199091015840
119894|
119909119894 119868lat119894 119886119894)The lateral behavior 119868lat119894 is considered to be a goal-
directed driving intention which will not be changed in thedriving process So Pr(1199091015840
119894| 119909119894 119868lat119894 119886119894) is equal to Pr(119909
1015840
119894| 119909119894 119886119894)
given a reference path corresponding to the intention of 119868lat119894Using (11) Pr(1199091015840
119894| 119909119894 119886119894) can be well solved
The remaining problem for calculating Pr(1199041015840119894| 119904119894 119886119894) is
to deal with Pr(1198681015840119894| 1199091015840
119894 119909119894 119868119894 119886119894) The lateral intention 1198681015840lat119894
is assumed stable through the above explanation And the
Mathematical Problems in Engineering 7
longitudinal intention 1198681015840
lon119894 is assumed to be not updatedin this process But it will be updated with new inputs inobservation space
Now Pr(1199041015840119894| 119904119894 119886119894) is well modeled and the remaining
problem is to compute the probabilities Pr(119886119894| 119904119894) of human-
driven vehiclesrsquo future actions
Pr (119886119894| 119904119894) = Pr (119886
119894| 119909119894 119868119894)
= sum
1199091015840
host
Pr (119886119894| 1199091015840
host 119909119894 119868119894)Pr (1199091015840
host | 119909119894 119868119894) (14)
Because 1199091015840
host is determined by the designed policyPr(1199091015840host | 119909119894 119868119894) could be calculated by (11) given an action119886host The probability Pr(119886
119894| 1199091015840
host 119909119894 119868119894) means the distribu-tion of human-driven vehiclesrsquo actions given the new state1199091015840
host of host vehicle the current state of itself and itsintentions Instead of building a complex probability modelwe designed a deterministic mechanism to calculate themostpossible action 119886
119894given 1199091015840host 119909119894 and 119868119894
In this prediction process the host vehicle is assumed tobe maintaining the current actions in the next time step andthe action 119886
119894will be leading human-driven vehicle passing
through the potential collision area either in advance of hostvehicle under the intention 119868
119873Yield or behind the host vehicleunder the intention 119868Yield to keep a safe distance 119889safe In thecase with the intention of 119868
119873Yield we can calculate the lowboundary 119886
119894low of 119886119894 through the above process anddeterminethe upper one using the largest comfort value 119886
119894comfort If119886119894comfort lt 119886
119894low 119886119894low will be used as the human-drivenvehiclersquos action If not we consider the targeted 119886
119894following
a normal distribution with mean value 120583119886119894
between 119886119894low and
119886119894comfort To simplify our model we use the mean value ofthese two boundaries to represent human-driven vehiclersquosaction 119886
119894 Similarly the case with the intention of 119868Yield can
be analyzed in the same processAfter these steps the transition probability Pr(1199041015840 | 119904 119886)
is well formulized and the autonomous vehicle could havethe ability to understand the future motion of the scenariothrough this model
46 Observation Model The observation model is built tosimulate the measurement process The motion intentionis updated in this process The measurements of human-driven vehicles are modeled with conditional independentassumption Thus the observation model can be calculatedas
Pr (119911 | 119886 1199041015840) = Pr (119911host | 1199041015840
host)119873
prod
119894=1
Pr (119911119894| 1199041015840
119894) (15)
The host vehiclersquos observation function is denoted as
Pr (119911host | 1199041015840
host) simN (119911host | 1199091015840
host Σ119911host) (16)
But in this paper due to the use of V2V communicationsensor the observation error almost does not affect the deci-sion-making result The variance matrix is set as zero
The human-driven vehiclersquos observation will follow thevehiclersquos motion intentions Because we do not consider
the observation error the value in metric state will be thesame as the state transition results But the longitudinalintention of human-driven vehicles in the state space will beupdated using the new observations andHMMmentioned inSection 3 The new observation space will be confirmed withthe above step
47 Reward Function The candidate policies have to satisfyseveral evaluation criterions Autonomous vehicles should bedriven safely and comfortably At the same time they shouldfollow the traffic rules and reach the destination as soon aspossible As a result we design objective function (17) consid-ering three aspects including safety time efficiency and trafficlaws where 120583
1 1205832 and 120583
3are the weight coefficient
119877 (119904 119886) = 1205831119877safety (119904 119886) + 1205832119877time (119904 119886)
+ 1205833119877law (119904 119886)
(17)
The detailed information will be discussed in the follow-ing subsections In addition the factor of comfort will be con-sidered and discussed in policy generation part (Section 51)
471 Safety Reward The safety reward function 119877safety(119904 119886)is based on the potential conflict status In our strategysafety reward is defined as a penalty If there are no potentialconflicts the safety rewardwill be set as 0 A large penalty willbe assigned due to the risk of collision status
In an uncontrolled intersection the four approachingdirections are defined as 119860
119894isin 1 2 3 4 (Figure 5) The
driverrsquos lateral intentions are defined as 119868lat isin 119868TR 119868TL 119868GS119868S So the driving trajectory for each vehicle in the inter-section can be generally represented by 119860
119894and 119868lat119895 and we
marked it as 119879119860119894119868lat119895
1 le 119894 le 4 1 le 119895 le 4 The function 119865 isused to judge the potential collision status which is denotedas
119865 (119879119909 119879119910) =
1 if potential conflict
0 otherwise(18)
where 119879119909and 119879
119910are vehiclesrsquo maneuver 119879
119860119894119868lat119895
119865(119909 119910) can be calculated through relative direction
between two cars which is shown in Table 1The safety reward is based on the following items(i) If 119865(119909 119910) is equal to 0 then the safety reward is equal
to 0 due to the noncollision status(ii) If potential collision occurs there will be a large
penalty(iii) If |TTC
119894minus TTChost| lt 119905threshod there is a penalty
depending on |TTC119894minus TTChost| and TTChost
472 Traffic Law Reward Autonomous vehicles should fol-low traffic laws to interactwith human-driven vehicles Trafficlaw is modeled as a function Law(119909 119910) for each two vehicles119909 and 119910
Law (119879119909 119879119910) =
1 if 119909 is prior
0 otherwise(19)
8 Mathematical Problems in Engineering
Human-driven vehicle
Autonomous vehicle
1
4
3
2
Figure 5 One typical scenario for calculating safety reward
Table 1 Safe condition judgments in the intersection
Human-driven vehicle Left side Right side Opposite side Same sideDriving direction TL LK TR S TL LK TR S TL LK TR S TL LK TR S
Turn left permil permil I I permil permil I I permil permil permil I I I I ILane keeping permil permil I I permil permil permil I permil I I I I I I ITurn right I permil I I I I I I permil I I I I I I IStop I I I I I I I I I I I I I I I I
permil indicates potential collision I indicates no potential collision
where 119879119909and 119879
119910are vehiclesrsquo maneuver 119879
119860119894119878119895
This functionLaw(119879
119909 119879119910) is formulized as shown in Algorithm 1
If the behavior will break the law a large penalty is appliedand the behavior of obeying traffic laws will get a zero reward
473 Time Reward The time cost is based on the time to thedestination for the targeted vehicles in the intersection area
Costtime =DTGVhost
(20)
DTG is the distance to the driving goal In addition wealso need to consider the speed limit which is discussed inpolicy generation part in Section 5
5 Approximations on SolvingPOMDP Problem
Solving POMDP is quite difficult The complexity of search-ing total brief space is O(|A|119867|Z|
119867
) [12] where 119867 is theprediction horizon In this paper we model the intentionrecognition process as a deterministic model and use com-munication sensors to ignore the perception error and thusthe size of |Z| is reduced to 1 in the simplified problem Tosolve this problem we first generate the suitable potentialpolicies according to the property of driving tasks and then
select the reasonable total predicting interval time and totalhorizon After that the approximate optimal policy canbe calculated through searching all possible policies withmaximum total rewardThe policy selection process is shownin Algorithm 2 and some detailed explanations are discussedin the subsections
51 Policy Generation For autonomous driving near inter-section the desired velocity curves need to satisfy severalconstraints Firstly except for emergency braking the accel-eration constraints are applied to ensure comfort Secondlythe speed limit constraints should be used in this process Weaim to avoid the acceleration commands when autonomousvehicle is reaching maximum speed limit Thirdly for thecomfort purpose the acceleration command should not bealways changed In otherwords we need tominimize the jerk
Similar to [11] the candidate policies are divided intothree time segments The first two segments are like ldquokeepconstant accelerationdeceleration actionsrdquo while keepingconstant velocity in the third segment We use 119905
1 1199052 and 119905
3
to represent the time periods of these three segments Toguarantee comfort the acceleration is limited to the rangefrom minus4ms2 to 2ms2 and we discrete acceleration actioninto a multiple of [minus05 05 0] Then the action space can berepresented by a discretizing acceleration setThenwe can set
Mathematical Problems in Engineering 9
Law(119879119909 119879119910) larr 0
119905119909larr 1198891199092119868V119909
119905119910larr 1198891199102119868V119910
if 119905119909lt 119905119910minus Δ119905 then
Law(119879119909 119879119910) larr 1
else if |119905119909minus 119905119910| lt Δ119905 then
statuslarr 119865(119879119909 119879119910)
if status = 1 thenif 119868lat119909 = lanekeeping and 119868lat119910 ltgt lanekeeping thenLaw(119879
119909 119879119910) larr 1
else if 119868lat119909 119868lat119910 = lanekeeping and 119860119909minus 119860119910= 1 or minus3 then
Law(119879119909 119879119910) larr 1
else if 119868lat119909 = turnleft and 119868lat119910 = turnright thenLaw(119879
119909 119879119910) larr 1
end ifend if
end ifreturn Law(119879
119909 119879119910)
Algorithm 1 Traffic law formulization
InputPredict horizon119867 time step Δ119905Current states 119904host = 119909host 119904human = [119909human 119868human]
(1) 119875 larr genenratepolicyset()(2) for each 120587
119896isin 119875 do
(3) for 119894 = 1 to119867Δ119905 do(4) 119886host larr 120587
119896(119894)
(5) 1199041015840
host larr updatestate(119904host 119886host)(6) 119886human larr predictactions(1199041015840host 119904human 119868human)
(7) 1199091015840
human larr updatestate(119909human 119886human)
(8) 1198681015840
human larr updateintention(1199041015840host 1199091015840
human)
(9) 1199041015840
human larr [1199091015840
human 1198681015840
human]
(10) 119877(119894) = calculatereward(1199041015840host 1199041015840
human)
(11) 119904host larr 1199041015840
host(12) 119904human larr 119904
1015840
human(13) end(14) 119877
total119896
larr sum119894sum(119877(119894))
(15) end(16) 119896lowast larr argmax
119896(119877
total119896
)
(17) 120587lowast larr 120587119896lowast
(18) return 120587lowast
Algorithm 2 Policy selection process
the value of 1199051 1199052 and 119905
3and the prediction period of single
step An example of policy generation is shown in Figure 6
52 Planning Horizon Selection After building policy gener-ation model the next problem is to select a suitable planninghorizon Longer horizon can lead to a better solution but con-suming more computing resources However as our purpose
is to deal with the interaction problem in the uncontrolledintersection we only need to consider the situation beforeautonomous vehicle gets through In our algorithmwe set theprediction horizon as 8 seconds In addition in the process ofupdating the future state of each vehicle using each policy thecar following mode is used after autonomous vehicle passesthrough the intersection area
10 Mathematical Problems in Engineering
0
1
2Candidate policies
minus4
minus3
minus2
minus1
Acce
lera
tion
(ms2)
1 2 3 4 5 6 7 80Time (s)
(a)
Time (s)0 1 2 3 4 5 6 7 8
0
2
4
6
8
10
12
14
16
18
20
Velo
city
(ms
)
Velocity strategies
(b)
Figure 6 An example of policy generation process (a) is the generated policies and (b) is the corresponding speed profiles The interval ofeach prediction step is 05 s current speed is 12ms2 and the speed limit is 20ms2 The bold black line is one policy In the first 3 secondsautonomous vehicles decelerate in minus35ms2 then accelerate at 2ms2 for 4 seconds and finally stop in the last one second In this case 109policies were generated which is suitable for replanning fast
AB
Figure 7 Testing scenario Autonomous vehicle B and human-driven vehicle A are both approaching the uncontrolled intersection To goacross the intersection successfully autonomous vehicle should interact with human-driven vehicle
53 Time Step Selection Another problem is the predic-tion time step The intention prediction algorithm and thePOMDP are computed in each step If the time step is 119905stepthe total computation timeswill be119867119905stepThus smaller timestep leads to more computation time To solve this problemwe use a simple adaptive time step calculation mechanism togive a final value The time step is selected based on the TTCof autonomous vehicle If the host vehicle is far away from theintersection we can use a very large time step But if the TTCis quite small the low 119905step is applied to ensure safety
6 Experiment and Results
61 Settings In this paper we evaluate our approach throughPreScan 710 [21] a simulation tool for autonomous drivingand connected vehicles Using this software we can build thetesting scenarios (Figure 7) and add vehicles with dynamicmodel In order to get a similar scenario considering social
interaction the driver simulator is added in our experiment(Figure 8) The human-driven vehicle is driven by severalpeople during the experiment and the autonomous vehiclemakes decisions based on the human-driven vehiclersquos drivingbehavior The reference trajectory for autonomous vehicleis generated from path planning module and the human-driven vehiclersquos data (eg position velocity and heading)are transferred through V2V communication sensor Thedecision-making module sends desired velocity command tothe PID controlled to follow the reference path All policies inthe experiment part use a planning horizon119867 = 8 s which isdiscretized into the time step of 05 s
62 Results It is difficult to compare different approachesin the same scenario because the environment is dynamicand not exactly the same However we select two typicalsituations and special settings to make it possible The sameinitial conditions including position orientation and velocity
Mathematical Problems in Engineering 11
Figure 8 Logitech G27 driving simulator
T = 0 1 2 6 s T = 5 s
T = 4 s
T = 3 s
T = 2 s
T = 1 s
T = 0 s
T = 6 s
(a)
T = 0 1 2 9 s
T = 4 s
T = 5 s
T = 6 sT = 7 sT = 8 s
T = 9 s
T = 3 s
T = 2 s
T = 1 s
T = 0 s
(b)
Figure 9 The visualized passing sequence (a) is the result of our approach and (b) represents the result of reactive approach withoutconsidering intention The black vehicle is an autonomous vehicle while the red car is the human-driven vehicle Each vehicle representsthe position in a specific time 119879 with an interval of 1 second
for each vehicle are used in different tests Besides twotypical situations including human-driven vehicle gettingthrough before or after autonomous vehicle are comparedin this section With the same initial state different reac-tions will occur based on various methods We compareour approach and reactive-based method [6] in this sec-tion The key difference for these two methods is that ourapproach considers human-driven vehiclersquos driving inten-tion
The first experiment is that human-driven vehicle triesto yield autonomous vehicle in the interaction process Theresults are shown in Figures 9 and 10 Firstly Figure 9 givesus a visual comparison of the different approaches Fromalmost the same initial state (eg position and velocity) ourapproach could lead to autonomous vehicle passing throughthe intersection more quickly and reasonable
Then let us look at Figure 10 for detailed explanationIn the first 12 s in Figures 10(a) and 10(c) autonomousvehicle maintains speed and understands that human-drivenvehicle will not perform yielding actions Then autonomousvehicle gets yielding intention of human-driven vehicle andunderstands that human-driven vehiclersquos lateral intentionis to go straight Based on candidate policies autonomousvehicle selects acceleration strategy with maximum rewardand finally crosses the intersection In this process we canobviously find that autonomous vehicle understands human-driven vehiclersquos yielding intention Figure 10(c) is an exampleof understand human-driven vehiclersquos behavior based on egovehiclersquos future actions in a specific time Our strategy pre-dicts the future actions of human-driven vehicle Althoughthe velocity curves after 1 s do not correspond it does notaffect the performance of our methods The reason is that
12 Mathematical Problems in Engineering
0 1 2 3 4 5 6 7 80
1020
Time (s)
Velo
city
(ms
) Velocity profile
0 1 2 3 4 5 6 7 8024
Time (s)
Inte
ntio
n Longitudinal intention
0 1 2 3 4 5 6 7 8024
Time (s)
Inte
ntio
n Lateral intention
(a)
0 1 2 3 4 5 6 7 8 90
1020
Time (s)
Velo
city
(ms
) Velocity profile
0 1 2 3 4 5 6 7 8 9024
Time (s)
Inte
ntio
n
Longitudinal intention
0 1 2 3 4 5 6 7 8 9024
Time (s)
Inte
ntio
n
Lateral intention
(b)
0 1 2 3 4 5 6 7 8minus60
minus40
minus20
0
20
40
60
80
Time (s)
Distance to collision region
Dist
ance
(m)
(c)
0 1 2 3 4 5 6 7 8 9minus10
0
10
20
30
40
50
60
70
80
Time (s)
Dist
ance
(m)
Distance to collision region
(d)
0 1 2 3 4 5 6 7 87
8
9
10
11
12
13
14
15
Time (s)
Velo
city
(ms
)
(e)
Time (s)0 1 2 3 4 5 6 7 8
0
2
4
6
8
10
12
14
Velo
city
(ms
)
(f)
Figure 10 Case test 1 In this case human-driven vehicle passes through intersection after autonomous vehicle (a) (c) and (e) are theperformance of our method while (b) (d) and (f) are from the strategy without considering the driving intention (a) and (b) are the velocityprofiles and the corresponding driving intention For longitudinal intention label 1 means yielding and label 2 means not yielding In lateralintention 1 means turning left 2 means turning right 3 means going straight and 4 means stop The intentions in (b) are not used in thatmethod but for detailed analysis (c) and (d) are the distance to collision area for autonomous vehicle and human-driven vehicle respectively(e) and (f) are the prediction and true motions of human-driven vehicles in time 15 s with a prediction length of 8 s The red curves in thesesubfigures are from autonomous vehicle while blue lines are from human-driven vehicle The green lines in (e) and (f) are the predictionvelocity curves of human-driven vehicle
Mathematical Problems in Engineering 13
T = 4 s
T = 1 s
T = 2 s
T = 3 s
T = 5 s
T = 0 s
T = 0 1 2 5 s
(a)
T = 4 s
T = 1 s
T = 2 s
T = 3 s
T = 5 s
T = 0 s
T = 0 1 2 5 s
(b)
Figure 11 The visualized passing sequence for the case of human-driven vehicle first getting through (a) is the result of our approach and(b) is the reactive-based approach
we use a deterministic model in the prediction process andthe prediction value is inside two boundaries to ensure safetyBesides the whole actions of autonomous vehicle in thisprocess could also help human-driven vehicle to understandnot-yielding intention of autonomous vehicles In this casecooperative driving behaviors are performed by both vehi-cles
However if the intention is not considered in this processwe can find the results in Figures 10(b) 10(d) and 10(f)After 2 s in Figure 10(b) while the human-driven vehiclegives a yielding intention autonomous vehicle could notunderstand and they find a potential collision based on theconstant velocity assumptions Then it decreases the speedbut the human-driven vehicle also slows down The puzzledbehavior leads both vehicles to slow down near intersectionFinally human-driven vehicle stops at the stop line and thenautonomous vehicle could pass the intersection In this strat-egy the human-driven vehiclersquos future motion is assumed tobe constant (Figure 10(f))Without understanding of human-driven vehiclersquos intentions this strategy can increase conges-tion problem
Another experiment is that human-driven vehicle tries toget through the intersection first The results are shown inFigures 11 and 12This case is quite typical because many traf-fic accidents in real world are happening in this situation Indetail if one vehicle tries to cross an intersection while violat-ing the law another vehicle will be in great danger if it doesnot understand its behavior From the visualized perform-ance in Figure 11 our method is a little more safe than otherapproaches as there is nearly collision situation in Figure11(b) In detail we can see from Figure 12(a) that our strategycould perform deceleration actions after we understand thenot-yielding intention in 08 sHowever without understand-ing human-driven vehiclersquos motion intention the responsetime has a 1-second delay which may be quite dangerous
In addition it is shown that good performance is in thepredictions of human-driven vehiclersquos future motion in ourmethods (Figure 12(e))
The results of these two cases demonstrate that our algo-rithm could deal with typical scenarios and have better per-formance than traditional reactive controller Autonomousvehicle could be driven more safely fast and comfortablythrough our strategy
7 Conclusion and Future Work
In this paper we proposed an autonomous driving decision-making algorithm considering human-driven vehiclersquos uncer-tain intentions in an uncontrolled intersection The lat-eral and longitudinal intentions are recognized by a con-tinuous HMM Based on HMM and POMDP we modelgeneral decision-making process and then use an approxi-mate approach to solve this complex problem Finally weuse PreScan software and a driving simulator to emulatesocial interaction process The experiment results show thatautonomous vehicles with our approach can pass throughuncontrolled intersections more safely and efficiently thanusing the strategy without considering human-driven vehi-clesrsquo driving intentions
In the near future we aim to implement our approach intoa real autonomous vehicle and perform real world experi-ments In additionmore precious intention recognition algo-rithm aims to be figured out Somemethods like probabilisticgraphic model can be used to get a distribution of each inten-tion Finally designing online POMDP planning algorithmsis also valuable
Competing Interests
The authors declare that there are no competing interestsregarding the publication of this paper
14 Mathematical Problems in Engineering
0 05 1 15 2 25 3 35 4 45 510
20
30
Time (s)
Velo
city
(ms
) Velocity profile
0 05 1 15 2 25 3 35 4 45 50
2
4
Time (s)
Inte
ntio
n
Longitudinal intention
0 05 1 15 2 25 3 35 4 45 5024
Inte
ntio
n Lateral intention
Time (s)
(a)
0 05 1 15 2 25 3 35 4 45 5102030
Time (s)
Velo
city
(ms
) Velocity profile
0 05 1 15 2 25 3 35 4 45 5024
Time (s)
Inte
ntio
n
Longitudinal intention
0 05 1 15 2 25 3 35 4 45 5024
Time (s)
Inte
ntio
n
Lateral intention
(b)
0 05 1 15 2 25 3 35 4 45 5
0
20
40
60
80
Dist
ance
(m)
Distance to collision region
Time (s)
minus40
minus20
(c)
0 05 1 15 2 25 3 35 4 45 5
0
20
40
60
80D
istan
ce (m
)
Distance to collision region
Time (s)
minus40
minus20
(d)
0 1 2 3 4 5 6 7 816
17
18
19
20
21
22
23
24
25
Velo
city
(ms
)
Time (s)
(e)
0 1 2 3 4 5 6 7 817
18
19
20
21
22
23
24
25
Time (s)
Velo
city
(ms
)
(f)
Figure 12 Case test 2 In this case human-driven vehicle passes through intersection before autonomous vehicle through different strategiesThe definition of each subfigure is the same as in Figure 10
Mathematical Problems in Engineering 15
Acknowledgments
This study is supported by the National Natural ScienceFoundation of China (no 91420203)
References
[1] C Urmson J Anhalt D Bagnell et al ldquoAutonomous driving inurban environments boss and the urban challengerdquo Journal ofField Robotics vol 25 no 8 pp 425ndash466 2008
[2] J Markoff ldquoGoogle cars drive themselves in trafficrdquo New YorkTimes vol 9 2010
[3] L R Rabiner and B-H Juang ldquoAn introduction to hiddenMarkov modelsrdquo IEEE ASSP Magazine vol 3 no 1 pp 4ndash161986
[4] M Buehler K Iagnemma and S Singh The DARPA UrbanChallenge Autonomous Vehicles in City Traffic vol 56 Springer2009
[5] S Kammel J Ziegler B Pitzer et al ldquoTeam AnnieWAYrsquos auto-nomous system for the 2007 DARPAUrban Challengerdquo Journalof Field Robotics vol 25 no 9 pp 615ndash639 2008
[6] C R Baker and J M Dolan ldquoTraffic interaction in the urbanchallenge putting boss on its best behaviorrdquo in Proceedingsof the IEEERSJ International Conference on Intelligent Robotsand Systems (IROS rsquo08) pp 1752ndash1758 IEEE Nice FranceSeptember 2008
[7] M Montemerlo J Becker S Shat et al ldquoJunior the Stanfordentry in the urban challengerdquo Journal of Field Robotics vol 25no 9 pp 569ndash597 2008
[8] L Fletcher S Teller E Olson et al ldquoTheMIT-Cornell collisionand why it happenedrdquo Journal of Field Robotics vol 25 no 10pp 775ndash807 2008
[9] M Bahram A Wolf M Aeberhard and D Wollherr ldquoAprediction-based reactive driving strategy for highly automateddriving function on freewaysrdquo in Proceedings of the 25th IEEEIntelligent Vehicles Symposium pp 400ndash406 IEEE DearbornMich USA June 2014
[10] J Wei J M Dolan and B Litkouhi ldquoA prediction- and costfunction-based algorithm for robust autonomous freeway driv-ingrdquo in Proceedings of the IEEE Intelligent Vehicles Symposium(IV rsquo10) pp 512ndash517 San Diego Calif USA June 2010
[11] J Wei J M Dolan and B Litkouhi ldquoAutonomous vehiclesocial behavior for highway entrance ramp managementrdquo inProceedings of the IEEE Intelligent Vehicles Symposium (IV rsquo13)pp 201ndash207 IEEE Gold Coast Australia June 2013
[12] H Bai S Cai N Ye D Hsu and W S Lee ldquoIntention-awareonline POMDP planning for autonomous driving in a crowdrdquoin Proceedings of the IEEE International Conference on Roboticsand Automation (ICRA rsquo15) pp 454ndash460 IEEE Seattle WashUSA May 2015
[13] A Somani N Ye D Hsu and W S Lee ldquoDespot onlinepomdp planning with regularizationrdquo in Advances in NeuralInformation Processing Systems pp 1772ndash1780 2013
[14] S Brechtel T Gindele and R Dillmann ldquoProbabilistic deci-sion-making under uncertainty for autonomous driving usingcontinuous POMDPsrdquo in Proceedings of the 17th IEEE Interna-tional Conference on Intelligent Transportation Systems (ITSCrsquo14) pp 392ndash399 Qingdao China October 2014
[15] C H Papadimitriou and J N Tsitsiklis ldquoThe complexity ofMarkov decision processesrdquo Mathematics of Operations Re-search vol 12 no 3 pp 441ndash450 1987
[16] O Madani S Hanks and A Condon ldquoOn the undecidabilityof probabilistic planning and related stochastic optimizationproblemsrdquoArtificial Intelligence vol 147 no 1-2 pp 5ndash34 2003
[17] S Ulbrich and M Maurer ldquoProbabilistic online POMDP deci-sion making for lane changes in fully automated drivingrdquo inProceedings of the 16th International IEEE Conference on Intelli-gent Transportation Systems (ITSC rsquo13) pp 2063ndash2067 TheHague The Netherlands October 2013
[18] A G Cunningham E Galceran R M Eustice and E OlsonldquoMPDM multipolicy decision-making in dynamic uncertainenvironments for autonomous drivingrdquo in Proceedings of theIEEE International Conference on Robotics and Automation(ICRA rsquo15) pp 1670ndash1677 Seattle Wash USA May 2015
[19] D A Reynolds and R C Rose ldquoRobust text-independentspeaker identification using Gaussian mixture speaker modelsrdquoIEEE Transactions on Speech and Audio Processing vol 3 no 1pp 72ndash83 1995
[20] L E Baum and T Petrie ldquoStatistical inference for probabilisticfunctions of finite state Markov chainsrdquo The Annals of Mathe-matical Statistics vol 37 pp 1554ndash1563 1966
[21] M Tideman and M Van Noort ldquoA simulation tool suite fordeveloping connected vehicle systemsrdquo in Proceedings of theIEEE Intelligent Vehicles Symposium (IEEE IV rsquo13) pp 713ndash718Queensland Australia June 2013
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Mathematical Problems in Engineering 7
longitudinal intention 1198681015840
lon119894 is assumed to be not updatedin this process But it will be updated with new inputs inobservation space
Now Pr(1199041015840119894| 119904119894 119886119894) is well modeled and the remaining
problem is to compute the probabilities Pr(119886119894| 119904119894) of human-
driven vehiclesrsquo future actions
Pr (119886119894| 119904119894) = Pr (119886
119894| 119909119894 119868119894)
= sum
1199091015840
host
Pr (119886119894| 1199091015840
host 119909119894 119868119894)Pr (1199091015840
host | 119909119894 119868119894) (14)
Because 1199091015840
host is determined by the designed policyPr(1199091015840host | 119909119894 119868119894) could be calculated by (11) given an action119886host The probability Pr(119886
119894| 1199091015840
host 119909119894 119868119894) means the distribu-tion of human-driven vehiclesrsquo actions given the new state1199091015840
host of host vehicle the current state of itself and itsintentions Instead of building a complex probability modelwe designed a deterministic mechanism to calculate themostpossible action 119886
119894given 1199091015840host 119909119894 and 119868119894
In this prediction process the host vehicle is assumed tobe maintaining the current actions in the next time step andthe action 119886
119894will be leading human-driven vehicle passing
through the potential collision area either in advance of hostvehicle under the intention 119868
119873Yield or behind the host vehicleunder the intention 119868Yield to keep a safe distance 119889safe In thecase with the intention of 119868
119873Yield we can calculate the lowboundary 119886
119894low of 119886119894 through the above process anddeterminethe upper one using the largest comfort value 119886
119894comfort If119886119894comfort lt 119886
119894low 119886119894low will be used as the human-drivenvehiclersquos action If not we consider the targeted 119886
119894following
a normal distribution with mean value 120583119886119894
between 119886119894low and
119886119894comfort To simplify our model we use the mean value ofthese two boundaries to represent human-driven vehiclersquosaction 119886
119894 Similarly the case with the intention of 119868Yield can
be analyzed in the same processAfter these steps the transition probability Pr(1199041015840 | 119904 119886)
is well formulized and the autonomous vehicle could havethe ability to understand the future motion of the scenariothrough this model
46 Observation Model The observation model is built tosimulate the measurement process The motion intentionis updated in this process The measurements of human-driven vehicles are modeled with conditional independentassumption Thus the observation model can be calculatedas
Pr (119911 | 119886 1199041015840) = Pr (119911host | 1199041015840
host)119873
prod
119894=1
Pr (119911119894| 1199041015840
119894) (15)
The host vehiclersquos observation function is denoted as
Pr (119911host | 1199041015840
host) simN (119911host | 1199091015840
host Σ119911host) (16)
But in this paper due to the use of V2V communicationsensor the observation error almost does not affect the deci-sion-making result The variance matrix is set as zero
The human-driven vehiclersquos observation will follow thevehiclersquos motion intentions Because we do not consider
the observation error the value in metric state will be thesame as the state transition results But the longitudinalintention of human-driven vehicles in the state space will beupdated using the new observations andHMMmentioned inSection 3 The new observation space will be confirmed withthe above step
47 Reward Function The candidate policies have to satisfyseveral evaluation criterions Autonomous vehicles should bedriven safely and comfortably At the same time they shouldfollow the traffic rules and reach the destination as soon aspossible As a result we design objective function (17) consid-ering three aspects including safety time efficiency and trafficlaws where 120583
1 1205832 and 120583
3are the weight coefficient
119877 (119904 119886) = 1205831119877safety (119904 119886) + 1205832119877time (119904 119886)
+ 1205833119877law (119904 119886)
(17)
The detailed information will be discussed in the follow-ing subsections In addition the factor of comfort will be con-sidered and discussed in policy generation part (Section 51)
471 Safety Reward The safety reward function 119877safety(119904 119886)is based on the potential conflict status In our strategysafety reward is defined as a penalty If there are no potentialconflicts the safety rewardwill be set as 0 A large penalty willbe assigned due to the risk of collision status
In an uncontrolled intersection the four approachingdirections are defined as 119860
119894isin 1 2 3 4 (Figure 5) The
driverrsquos lateral intentions are defined as 119868lat isin 119868TR 119868TL 119868GS119868S So the driving trajectory for each vehicle in the inter-section can be generally represented by 119860
119894and 119868lat119895 and we
marked it as 119879119860119894119868lat119895
1 le 119894 le 4 1 le 119895 le 4 The function 119865 isused to judge the potential collision status which is denotedas
119865 (119879119909 119879119910) =
1 if potential conflict
0 otherwise(18)
where 119879119909and 119879
119910are vehiclesrsquo maneuver 119879
119860119894119868lat119895
119865(119909 119910) can be calculated through relative direction
between two cars which is shown in Table 1The safety reward is based on the following items(i) If 119865(119909 119910) is equal to 0 then the safety reward is equal
to 0 due to the noncollision status(ii) If potential collision occurs there will be a large
penalty(iii) If |TTC
119894minus TTChost| lt 119905threshod there is a penalty
depending on |TTC119894minus TTChost| and TTChost
472 Traffic Law Reward Autonomous vehicles should fol-low traffic laws to interactwith human-driven vehicles Trafficlaw is modeled as a function Law(119909 119910) for each two vehicles119909 and 119910
Law (119879119909 119879119910) =
1 if 119909 is prior
0 otherwise(19)
8 Mathematical Problems in Engineering
Human-driven vehicle
Autonomous vehicle
1
4
3
2
Figure 5 One typical scenario for calculating safety reward
Table 1 Safe condition judgments in the intersection
Human-driven vehicle Left side Right side Opposite side Same sideDriving direction TL LK TR S TL LK TR S TL LK TR S TL LK TR S
Turn left permil permil I I permil permil I I permil permil permil I I I I ILane keeping permil permil I I permil permil permil I permil I I I I I I ITurn right I permil I I I I I I permil I I I I I I IStop I I I I I I I I I I I I I I I I
permil indicates potential collision I indicates no potential collision
where 119879119909and 119879
119910are vehiclesrsquo maneuver 119879
119860119894119878119895
This functionLaw(119879
119909 119879119910) is formulized as shown in Algorithm 1
If the behavior will break the law a large penalty is appliedand the behavior of obeying traffic laws will get a zero reward
473 Time Reward The time cost is based on the time to thedestination for the targeted vehicles in the intersection area
Costtime =DTGVhost
(20)
DTG is the distance to the driving goal In addition wealso need to consider the speed limit which is discussed inpolicy generation part in Section 5
5 Approximations on SolvingPOMDP Problem
Solving POMDP is quite difficult The complexity of search-ing total brief space is O(|A|119867|Z|
119867
) [12] where 119867 is theprediction horizon In this paper we model the intentionrecognition process as a deterministic model and use com-munication sensors to ignore the perception error and thusthe size of |Z| is reduced to 1 in the simplified problem Tosolve this problem we first generate the suitable potentialpolicies according to the property of driving tasks and then
select the reasonable total predicting interval time and totalhorizon After that the approximate optimal policy canbe calculated through searching all possible policies withmaximum total rewardThe policy selection process is shownin Algorithm 2 and some detailed explanations are discussedin the subsections
51 Policy Generation For autonomous driving near inter-section the desired velocity curves need to satisfy severalconstraints Firstly except for emergency braking the accel-eration constraints are applied to ensure comfort Secondlythe speed limit constraints should be used in this process Weaim to avoid the acceleration commands when autonomousvehicle is reaching maximum speed limit Thirdly for thecomfort purpose the acceleration command should not bealways changed In otherwords we need tominimize the jerk
Similar to [11] the candidate policies are divided intothree time segments The first two segments are like ldquokeepconstant accelerationdeceleration actionsrdquo while keepingconstant velocity in the third segment We use 119905
1 1199052 and 119905
3
to represent the time periods of these three segments Toguarantee comfort the acceleration is limited to the rangefrom minus4ms2 to 2ms2 and we discrete acceleration actioninto a multiple of [minus05 05 0] Then the action space can berepresented by a discretizing acceleration setThenwe can set
Mathematical Problems in Engineering 9
Law(119879119909 119879119910) larr 0
119905119909larr 1198891199092119868V119909
119905119910larr 1198891199102119868V119910
if 119905119909lt 119905119910minus Δ119905 then
Law(119879119909 119879119910) larr 1
else if |119905119909minus 119905119910| lt Δ119905 then
statuslarr 119865(119879119909 119879119910)
if status = 1 thenif 119868lat119909 = lanekeeping and 119868lat119910 ltgt lanekeeping thenLaw(119879
119909 119879119910) larr 1
else if 119868lat119909 119868lat119910 = lanekeeping and 119860119909minus 119860119910= 1 or minus3 then
Law(119879119909 119879119910) larr 1
else if 119868lat119909 = turnleft and 119868lat119910 = turnright thenLaw(119879
119909 119879119910) larr 1
end ifend if
end ifreturn Law(119879
119909 119879119910)
Algorithm 1 Traffic law formulization
InputPredict horizon119867 time step Δ119905Current states 119904host = 119909host 119904human = [119909human 119868human]
(1) 119875 larr genenratepolicyset()(2) for each 120587
119896isin 119875 do
(3) for 119894 = 1 to119867Δ119905 do(4) 119886host larr 120587
119896(119894)
(5) 1199041015840
host larr updatestate(119904host 119886host)(6) 119886human larr predictactions(1199041015840host 119904human 119868human)
(7) 1199091015840
human larr updatestate(119909human 119886human)
(8) 1198681015840
human larr updateintention(1199041015840host 1199091015840
human)
(9) 1199041015840
human larr [1199091015840
human 1198681015840
human]
(10) 119877(119894) = calculatereward(1199041015840host 1199041015840
human)
(11) 119904host larr 1199041015840
host(12) 119904human larr 119904
1015840
human(13) end(14) 119877
total119896
larr sum119894sum(119877(119894))
(15) end(16) 119896lowast larr argmax
119896(119877
total119896
)
(17) 120587lowast larr 120587119896lowast
(18) return 120587lowast
Algorithm 2 Policy selection process
the value of 1199051 1199052 and 119905
3and the prediction period of single
step An example of policy generation is shown in Figure 6
52 Planning Horizon Selection After building policy gener-ation model the next problem is to select a suitable planninghorizon Longer horizon can lead to a better solution but con-suming more computing resources However as our purpose
is to deal with the interaction problem in the uncontrolledintersection we only need to consider the situation beforeautonomous vehicle gets through In our algorithmwe set theprediction horizon as 8 seconds In addition in the process ofupdating the future state of each vehicle using each policy thecar following mode is used after autonomous vehicle passesthrough the intersection area
10 Mathematical Problems in Engineering
0
1
2Candidate policies
minus4
minus3
minus2
minus1
Acce
lera
tion
(ms2)
1 2 3 4 5 6 7 80Time (s)
(a)
Time (s)0 1 2 3 4 5 6 7 8
0
2
4
6
8
10
12
14
16
18
20
Velo
city
(ms
)
Velocity strategies
(b)
Figure 6 An example of policy generation process (a) is the generated policies and (b) is the corresponding speed profiles The interval ofeach prediction step is 05 s current speed is 12ms2 and the speed limit is 20ms2 The bold black line is one policy In the first 3 secondsautonomous vehicles decelerate in minus35ms2 then accelerate at 2ms2 for 4 seconds and finally stop in the last one second In this case 109policies were generated which is suitable for replanning fast
AB
Figure 7 Testing scenario Autonomous vehicle B and human-driven vehicle A are both approaching the uncontrolled intersection To goacross the intersection successfully autonomous vehicle should interact with human-driven vehicle
53 Time Step Selection Another problem is the predic-tion time step The intention prediction algorithm and thePOMDP are computed in each step If the time step is 119905stepthe total computation timeswill be119867119905stepThus smaller timestep leads to more computation time To solve this problemwe use a simple adaptive time step calculation mechanism togive a final value The time step is selected based on the TTCof autonomous vehicle If the host vehicle is far away from theintersection we can use a very large time step But if the TTCis quite small the low 119905step is applied to ensure safety
6 Experiment and Results
61 Settings In this paper we evaluate our approach throughPreScan 710 [21] a simulation tool for autonomous drivingand connected vehicles Using this software we can build thetesting scenarios (Figure 7) and add vehicles with dynamicmodel In order to get a similar scenario considering social
interaction the driver simulator is added in our experiment(Figure 8) The human-driven vehicle is driven by severalpeople during the experiment and the autonomous vehiclemakes decisions based on the human-driven vehiclersquos drivingbehavior The reference trajectory for autonomous vehicleis generated from path planning module and the human-driven vehiclersquos data (eg position velocity and heading)are transferred through V2V communication sensor Thedecision-making module sends desired velocity command tothe PID controlled to follow the reference path All policies inthe experiment part use a planning horizon119867 = 8 s which isdiscretized into the time step of 05 s
62 Results It is difficult to compare different approachesin the same scenario because the environment is dynamicand not exactly the same However we select two typicalsituations and special settings to make it possible The sameinitial conditions including position orientation and velocity
Mathematical Problems in Engineering 11
Figure 8 Logitech G27 driving simulator
T = 0 1 2 6 s T = 5 s
T = 4 s
T = 3 s
T = 2 s
T = 1 s
T = 0 s
T = 6 s
(a)
T = 0 1 2 9 s
T = 4 s
T = 5 s
T = 6 sT = 7 sT = 8 s
T = 9 s
T = 3 s
T = 2 s
T = 1 s
T = 0 s
(b)
Figure 9 The visualized passing sequence (a) is the result of our approach and (b) represents the result of reactive approach withoutconsidering intention The black vehicle is an autonomous vehicle while the red car is the human-driven vehicle Each vehicle representsthe position in a specific time 119879 with an interval of 1 second
for each vehicle are used in different tests Besides twotypical situations including human-driven vehicle gettingthrough before or after autonomous vehicle are comparedin this section With the same initial state different reac-tions will occur based on various methods We compareour approach and reactive-based method [6] in this sec-tion The key difference for these two methods is that ourapproach considers human-driven vehiclersquos driving inten-tion
The first experiment is that human-driven vehicle triesto yield autonomous vehicle in the interaction process Theresults are shown in Figures 9 and 10 Firstly Figure 9 givesus a visual comparison of the different approaches Fromalmost the same initial state (eg position and velocity) ourapproach could lead to autonomous vehicle passing throughthe intersection more quickly and reasonable
Then let us look at Figure 10 for detailed explanationIn the first 12 s in Figures 10(a) and 10(c) autonomousvehicle maintains speed and understands that human-drivenvehicle will not perform yielding actions Then autonomousvehicle gets yielding intention of human-driven vehicle andunderstands that human-driven vehiclersquos lateral intentionis to go straight Based on candidate policies autonomousvehicle selects acceleration strategy with maximum rewardand finally crosses the intersection In this process we canobviously find that autonomous vehicle understands human-driven vehiclersquos yielding intention Figure 10(c) is an exampleof understand human-driven vehiclersquos behavior based on egovehiclersquos future actions in a specific time Our strategy pre-dicts the future actions of human-driven vehicle Althoughthe velocity curves after 1 s do not correspond it does notaffect the performance of our methods The reason is that
12 Mathematical Problems in Engineering
0 1 2 3 4 5 6 7 80
1020
Time (s)
Velo
city
(ms
) Velocity profile
0 1 2 3 4 5 6 7 8024
Time (s)
Inte
ntio
n Longitudinal intention
0 1 2 3 4 5 6 7 8024
Time (s)
Inte
ntio
n Lateral intention
(a)
0 1 2 3 4 5 6 7 8 90
1020
Time (s)
Velo
city
(ms
) Velocity profile
0 1 2 3 4 5 6 7 8 9024
Time (s)
Inte
ntio
n
Longitudinal intention
0 1 2 3 4 5 6 7 8 9024
Time (s)
Inte
ntio
n
Lateral intention
(b)
0 1 2 3 4 5 6 7 8minus60
minus40
minus20
0
20
40
60
80
Time (s)
Distance to collision region
Dist
ance
(m)
(c)
0 1 2 3 4 5 6 7 8 9minus10
0
10
20
30
40
50
60
70
80
Time (s)
Dist
ance
(m)
Distance to collision region
(d)
0 1 2 3 4 5 6 7 87
8
9
10
11
12
13
14
15
Time (s)
Velo
city
(ms
)
(e)
Time (s)0 1 2 3 4 5 6 7 8
0
2
4
6
8
10
12
14
Velo
city
(ms
)
(f)
Figure 10 Case test 1 In this case human-driven vehicle passes through intersection after autonomous vehicle (a) (c) and (e) are theperformance of our method while (b) (d) and (f) are from the strategy without considering the driving intention (a) and (b) are the velocityprofiles and the corresponding driving intention For longitudinal intention label 1 means yielding and label 2 means not yielding In lateralintention 1 means turning left 2 means turning right 3 means going straight and 4 means stop The intentions in (b) are not used in thatmethod but for detailed analysis (c) and (d) are the distance to collision area for autonomous vehicle and human-driven vehicle respectively(e) and (f) are the prediction and true motions of human-driven vehicles in time 15 s with a prediction length of 8 s The red curves in thesesubfigures are from autonomous vehicle while blue lines are from human-driven vehicle The green lines in (e) and (f) are the predictionvelocity curves of human-driven vehicle
Mathematical Problems in Engineering 13
T = 4 s
T = 1 s
T = 2 s
T = 3 s
T = 5 s
T = 0 s
T = 0 1 2 5 s
(a)
T = 4 s
T = 1 s
T = 2 s
T = 3 s
T = 5 s
T = 0 s
T = 0 1 2 5 s
(b)
Figure 11 The visualized passing sequence for the case of human-driven vehicle first getting through (a) is the result of our approach and(b) is the reactive-based approach
we use a deterministic model in the prediction process andthe prediction value is inside two boundaries to ensure safetyBesides the whole actions of autonomous vehicle in thisprocess could also help human-driven vehicle to understandnot-yielding intention of autonomous vehicles In this casecooperative driving behaviors are performed by both vehi-cles
However if the intention is not considered in this processwe can find the results in Figures 10(b) 10(d) and 10(f)After 2 s in Figure 10(b) while the human-driven vehiclegives a yielding intention autonomous vehicle could notunderstand and they find a potential collision based on theconstant velocity assumptions Then it decreases the speedbut the human-driven vehicle also slows down The puzzledbehavior leads both vehicles to slow down near intersectionFinally human-driven vehicle stops at the stop line and thenautonomous vehicle could pass the intersection In this strat-egy the human-driven vehiclersquos future motion is assumed tobe constant (Figure 10(f))Without understanding of human-driven vehiclersquos intentions this strategy can increase conges-tion problem
Another experiment is that human-driven vehicle tries toget through the intersection first The results are shown inFigures 11 and 12This case is quite typical because many traf-fic accidents in real world are happening in this situation Indetail if one vehicle tries to cross an intersection while violat-ing the law another vehicle will be in great danger if it doesnot understand its behavior From the visualized perform-ance in Figure 11 our method is a little more safe than otherapproaches as there is nearly collision situation in Figure11(b) In detail we can see from Figure 12(a) that our strategycould perform deceleration actions after we understand thenot-yielding intention in 08 sHowever without understand-ing human-driven vehiclersquos motion intention the responsetime has a 1-second delay which may be quite dangerous
In addition it is shown that good performance is in thepredictions of human-driven vehiclersquos future motion in ourmethods (Figure 12(e))
The results of these two cases demonstrate that our algo-rithm could deal with typical scenarios and have better per-formance than traditional reactive controller Autonomousvehicle could be driven more safely fast and comfortablythrough our strategy
7 Conclusion and Future Work
In this paper we proposed an autonomous driving decision-making algorithm considering human-driven vehiclersquos uncer-tain intentions in an uncontrolled intersection The lat-eral and longitudinal intentions are recognized by a con-tinuous HMM Based on HMM and POMDP we modelgeneral decision-making process and then use an approxi-mate approach to solve this complex problem Finally weuse PreScan software and a driving simulator to emulatesocial interaction process The experiment results show thatautonomous vehicles with our approach can pass throughuncontrolled intersections more safely and efficiently thanusing the strategy without considering human-driven vehi-clesrsquo driving intentions
In the near future we aim to implement our approach intoa real autonomous vehicle and perform real world experi-ments In additionmore precious intention recognition algo-rithm aims to be figured out Somemethods like probabilisticgraphic model can be used to get a distribution of each inten-tion Finally designing online POMDP planning algorithmsis also valuable
Competing Interests
The authors declare that there are no competing interestsregarding the publication of this paper
14 Mathematical Problems in Engineering
0 05 1 15 2 25 3 35 4 45 510
20
30
Time (s)
Velo
city
(ms
) Velocity profile
0 05 1 15 2 25 3 35 4 45 50
2
4
Time (s)
Inte
ntio
n
Longitudinal intention
0 05 1 15 2 25 3 35 4 45 5024
Inte
ntio
n Lateral intention
Time (s)
(a)
0 05 1 15 2 25 3 35 4 45 5102030
Time (s)
Velo
city
(ms
) Velocity profile
0 05 1 15 2 25 3 35 4 45 5024
Time (s)
Inte
ntio
n
Longitudinal intention
0 05 1 15 2 25 3 35 4 45 5024
Time (s)
Inte
ntio
n
Lateral intention
(b)
0 05 1 15 2 25 3 35 4 45 5
0
20
40
60
80
Dist
ance
(m)
Distance to collision region
Time (s)
minus40
minus20
(c)
0 05 1 15 2 25 3 35 4 45 5
0
20
40
60
80D
istan
ce (m
)
Distance to collision region
Time (s)
minus40
minus20
(d)
0 1 2 3 4 5 6 7 816
17
18
19
20
21
22
23
24
25
Velo
city
(ms
)
Time (s)
(e)
0 1 2 3 4 5 6 7 817
18
19
20
21
22
23
24
25
Time (s)
Velo
city
(ms
)
(f)
Figure 12 Case test 2 In this case human-driven vehicle passes through intersection before autonomous vehicle through different strategiesThe definition of each subfigure is the same as in Figure 10
Mathematical Problems in Engineering 15
Acknowledgments
This study is supported by the National Natural ScienceFoundation of China (no 91420203)
References
[1] C Urmson J Anhalt D Bagnell et al ldquoAutonomous driving inurban environments boss and the urban challengerdquo Journal ofField Robotics vol 25 no 8 pp 425ndash466 2008
[2] J Markoff ldquoGoogle cars drive themselves in trafficrdquo New YorkTimes vol 9 2010
[3] L R Rabiner and B-H Juang ldquoAn introduction to hiddenMarkov modelsrdquo IEEE ASSP Magazine vol 3 no 1 pp 4ndash161986
[4] M Buehler K Iagnemma and S Singh The DARPA UrbanChallenge Autonomous Vehicles in City Traffic vol 56 Springer2009
[5] S Kammel J Ziegler B Pitzer et al ldquoTeam AnnieWAYrsquos auto-nomous system for the 2007 DARPAUrban Challengerdquo Journalof Field Robotics vol 25 no 9 pp 615ndash639 2008
[6] C R Baker and J M Dolan ldquoTraffic interaction in the urbanchallenge putting boss on its best behaviorrdquo in Proceedingsof the IEEERSJ International Conference on Intelligent Robotsand Systems (IROS rsquo08) pp 1752ndash1758 IEEE Nice FranceSeptember 2008
[7] M Montemerlo J Becker S Shat et al ldquoJunior the Stanfordentry in the urban challengerdquo Journal of Field Robotics vol 25no 9 pp 569ndash597 2008
[8] L Fletcher S Teller E Olson et al ldquoTheMIT-Cornell collisionand why it happenedrdquo Journal of Field Robotics vol 25 no 10pp 775ndash807 2008
[9] M Bahram A Wolf M Aeberhard and D Wollherr ldquoAprediction-based reactive driving strategy for highly automateddriving function on freewaysrdquo in Proceedings of the 25th IEEEIntelligent Vehicles Symposium pp 400ndash406 IEEE DearbornMich USA June 2014
[10] J Wei J M Dolan and B Litkouhi ldquoA prediction- and costfunction-based algorithm for robust autonomous freeway driv-ingrdquo in Proceedings of the IEEE Intelligent Vehicles Symposium(IV rsquo10) pp 512ndash517 San Diego Calif USA June 2010
[11] J Wei J M Dolan and B Litkouhi ldquoAutonomous vehiclesocial behavior for highway entrance ramp managementrdquo inProceedings of the IEEE Intelligent Vehicles Symposium (IV rsquo13)pp 201ndash207 IEEE Gold Coast Australia June 2013
[12] H Bai S Cai N Ye D Hsu and W S Lee ldquoIntention-awareonline POMDP planning for autonomous driving in a crowdrdquoin Proceedings of the IEEE International Conference on Roboticsand Automation (ICRA rsquo15) pp 454ndash460 IEEE Seattle WashUSA May 2015
[13] A Somani N Ye D Hsu and W S Lee ldquoDespot onlinepomdp planning with regularizationrdquo in Advances in NeuralInformation Processing Systems pp 1772ndash1780 2013
[14] S Brechtel T Gindele and R Dillmann ldquoProbabilistic deci-sion-making under uncertainty for autonomous driving usingcontinuous POMDPsrdquo in Proceedings of the 17th IEEE Interna-tional Conference on Intelligent Transportation Systems (ITSCrsquo14) pp 392ndash399 Qingdao China October 2014
[15] C H Papadimitriou and J N Tsitsiklis ldquoThe complexity ofMarkov decision processesrdquo Mathematics of Operations Re-search vol 12 no 3 pp 441ndash450 1987
[16] O Madani S Hanks and A Condon ldquoOn the undecidabilityof probabilistic planning and related stochastic optimizationproblemsrdquoArtificial Intelligence vol 147 no 1-2 pp 5ndash34 2003
[17] S Ulbrich and M Maurer ldquoProbabilistic online POMDP deci-sion making for lane changes in fully automated drivingrdquo inProceedings of the 16th International IEEE Conference on Intelli-gent Transportation Systems (ITSC rsquo13) pp 2063ndash2067 TheHague The Netherlands October 2013
[18] A G Cunningham E Galceran R M Eustice and E OlsonldquoMPDM multipolicy decision-making in dynamic uncertainenvironments for autonomous drivingrdquo in Proceedings of theIEEE International Conference on Robotics and Automation(ICRA rsquo15) pp 1670ndash1677 Seattle Wash USA May 2015
[19] D A Reynolds and R C Rose ldquoRobust text-independentspeaker identification using Gaussian mixture speaker modelsrdquoIEEE Transactions on Speech and Audio Processing vol 3 no 1pp 72ndash83 1995
[20] L E Baum and T Petrie ldquoStatistical inference for probabilisticfunctions of finite state Markov chainsrdquo The Annals of Mathe-matical Statistics vol 37 pp 1554ndash1563 1966
[21] M Tideman and M Van Noort ldquoA simulation tool suite fordeveloping connected vehicle systemsrdquo in Proceedings of theIEEE Intelligent Vehicles Symposium (IEEE IV rsquo13) pp 713ndash718Queensland Australia June 2013
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
8 Mathematical Problems in Engineering
Human-driven vehicle
Autonomous vehicle
1
4
3
2
Figure 5 One typical scenario for calculating safety reward
Table 1 Safe condition judgments in the intersection
Human-driven vehicle Left side Right side Opposite side Same sideDriving direction TL LK TR S TL LK TR S TL LK TR S TL LK TR S
Turn left permil permil I I permil permil I I permil permil permil I I I I ILane keeping permil permil I I permil permil permil I permil I I I I I I ITurn right I permil I I I I I I permil I I I I I I IStop I I I I I I I I I I I I I I I I
permil indicates potential collision I indicates no potential collision
where 119879119909and 119879
119910are vehiclesrsquo maneuver 119879
119860119894119878119895
This functionLaw(119879
119909 119879119910) is formulized as shown in Algorithm 1
If the behavior will break the law a large penalty is appliedand the behavior of obeying traffic laws will get a zero reward
473 Time Reward The time cost is based on the time to thedestination for the targeted vehicles in the intersection area
Costtime =DTGVhost
(20)
DTG is the distance to the driving goal In addition wealso need to consider the speed limit which is discussed inpolicy generation part in Section 5
5 Approximations on SolvingPOMDP Problem
Solving POMDP is quite difficult The complexity of search-ing total brief space is O(|A|119867|Z|
119867
) [12] where 119867 is theprediction horizon In this paper we model the intentionrecognition process as a deterministic model and use com-munication sensors to ignore the perception error and thusthe size of |Z| is reduced to 1 in the simplified problem Tosolve this problem we first generate the suitable potentialpolicies according to the property of driving tasks and then
select the reasonable total predicting interval time and totalhorizon After that the approximate optimal policy canbe calculated through searching all possible policies withmaximum total rewardThe policy selection process is shownin Algorithm 2 and some detailed explanations are discussedin the subsections
51 Policy Generation For autonomous driving near inter-section the desired velocity curves need to satisfy severalconstraints Firstly except for emergency braking the accel-eration constraints are applied to ensure comfort Secondlythe speed limit constraints should be used in this process Weaim to avoid the acceleration commands when autonomousvehicle is reaching maximum speed limit Thirdly for thecomfort purpose the acceleration command should not bealways changed In otherwords we need tominimize the jerk
Similar to [11] the candidate policies are divided intothree time segments The first two segments are like ldquokeepconstant accelerationdeceleration actionsrdquo while keepingconstant velocity in the third segment We use 119905
1 1199052 and 119905
3
to represent the time periods of these three segments Toguarantee comfort the acceleration is limited to the rangefrom minus4ms2 to 2ms2 and we discrete acceleration actioninto a multiple of [minus05 05 0] Then the action space can berepresented by a discretizing acceleration setThenwe can set
Mathematical Problems in Engineering 9
Law(119879119909 119879119910) larr 0
119905119909larr 1198891199092119868V119909
119905119910larr 1198891199102119868V119910
if 119905119909lt 119905119910minus Δ119905 then
Law(119879119909 119879119910) larr 1
else if |119905119909minus 119905119910| lt Δ119905 then
statuslarr 119865(119879119909 119879119910)
if status = 1 thenif 119868lat119909 = lanekeeping and 119868lat119910 ltgt lanekeeping thenLaw(119879
119909 119879119910) larr 1
else if 119868lat119909 119868lat119910 = lanekeeping and 119860119909minus 119860119910= 1 or minus3 then
Law(119879119909 119879119910) larr 1
else if 119868lat119909 = turnleft and 119868lat119910 = turnright thenLaw(119879
119909 119879119910) larr 1
end ifend if
end ifreturn Law(119879
119909 119879119910)
Algorithm 1 Traffic law formulization
InputPredict horizon119867 time step Δ119905Current states 119904host = 119909host 119904human = [119909human 119868human]
(1) 119875 larr genenratepolicyset()(2) for each 120587
119896isin 119875 do
(3) for 119894 = 1 to119867Δ119905 do(4) 119886host larr 120587
119896(119894)
(5) 1199041015840
host larr updatestate(119904host 119886host)(6) 119886human larr predictactions(1199041015840host 119904human 119868human)
(7) 1199091015840
human larr updatestate(119909human 119886human)
(8) 1198681015840
human larr updateintention(1199041015840host 1199091015840
human)
(9) 1199041015840
human larr [1199091015840
human 1198681015840
human]
(10) 119877(119894) = calculatereward(1199041015840host 1199041015840
human)
(11) 119904host larr 1199041015840
host(12) 119904human larr 119904
1015840
human(13) end(14) 119877
total119896
larr sum119894sum(119877(119894))
(15) end(16) 119896lowast larr argmax
119896(119877
total119896
)
(17) 120587lowast larr 120587119896lowast
(18) return 120587lowast
Algorithm 2 Policy selection process
the value of 1199051 1199052 and 119905
3and the prediction period of single
step An example of policy generation is shown in Figure 6
52 Planning Horizon Selection After building policy gener-ation model the next problem is to select a suitable planninghorizon Longer horizon can lead to a better solution but con-suming more computing resources However as our purpose
is to deal with the interaction problem in the uncontrolledintersection we only need to consider the situation beforeautonomous vehicle gets through In our algorithmwe set theprediction horizon as 8 seconds In addition in the process ofupdating the future state of each vehicle using each policy thecar following mode is used after autonomous vehicle passesthrough the intersection area
10 Mathematical Problems in Engineering
0
1
2Candidate policies
minus4
minus3
minus2
minus1
Acce
lera
tion
(ms2)
1 2 3 4 5 6 7 80Time (s)
(a)
Time (s)0 1 2 3 4 5 6 7 8
0
2
4
6
8
10
12
14
16
18
20
Velo
city
(ms
)
Velocity strategies
(b)
Figure 6 An example of policy generation process (a) is the generated policies and (b) is the corresponding speed profiles The interval ofeach prediction step is 05 s current speed is 12ms2 and the speed limit is 20ms2 The bold black line is one policy In the first 3 secondsautonomous vehicles decelerate in minus35ms2 then accelerate at 2ms2 for 4 seconds and finally stop in the last one second In this case 109policies were generated which is suitable for replanning fast
AB
Figure 7 Testing scenario Autonomous vehicle B and human-driven vehicle A are both approaching the uncontrolled intersection To goacross the intersection successfully autonomous vehicle should interact with human-driven vehicle
53 Time Step Selection Another problem is the predic-tion time step The intention prediction algorithm and thePOMDP are computed in each step If the time step is 119905stepthe total computation timeswill be119867119905stepThus smaller timestep leads to more computation time To solve this problemwe use a simple adaptive time step calculation mechanism togive a final value The time step is selected based on the TTCof autonomous vehicle If the host vehicle is far away from theintersection we can use a very large time step But if the TTCis quite small the low 119905step is applied to ensure safety
6 Experiment and Results
61 Settings In this paper we evaluate our approach throughPreScan 710 [21] a simulation tool for autonomous drivingand connected vehicles Using this software we can build thetesting scenarios (Figure 7) and add vehicles with dynamicmodel In order to get a similar scenario considering social
interaction the driver simulator is added in our experiment(Figure 8) The human-driven vehicle is driven by severalpeople during the experiment and the autonomous vehiclemakes decisions based on the human-driven vehiclersquos drivingbehavior The reference trajectory for autonomous vehicleis generated from path planning module and the human-driven vehiclersquos data (eg position velocity and heading)are transferred through V2V communication sensor Thedecision-making module sends desired velocity command tothe PID controlled to follow the reference path All policies inthe experiment part use a planning horizon119867 = 8 s which isdiscretized into the time step of 05 s
62 Results It is difficult to compare different approachesin the same scenario because the environment is dynamicand not exactly the same However we select two typicalsituations and special settings to make it possible The sameinitial conditions including position orientation and velocity
Mathematical Problems in Engineering 11
Figure 8 Logitech G27 driving simulator
T = 0 1 2 6 s T = 5 s
T = 4 s
T = 3 s
T = 2 s
T = 1 s
T = 0 s
T = 6 s
(a)
T = 0 1 2 9 s
T = 4 s
T = 5 s
T = 6 sT = 7 sT = 8 s
T = 9 s
T = 3 s
T = 2 s
T = 1 s
T = 0 s
(b)
Figure 9 The visualized passing sequence (a) is the result of our approach and (b) represents the result of reactive approach withoutconsidering intention The black vehicle is an autonomous vehicle while the red car is the human-driven vehicle Each vehicle representsthe position in a specific time 119879 with an interval of 1 second
for each vehicle are used in different tests Besides twotypical situations including human-driven vehicle gettingthrough before or after autonomous vehicle are comparedin this section With the same initial state different reac-tions will occur based on various methods We compareour approach and reactive-based method [6] in this sec-tion The key difference for these two methods is that ourapproach considers human-driven vehiclersquos driving inten-tion
The first experiment is that human-driven vehicle triesto yield autonomous vehicle in the interaction process Theresults are shown in Figures 9 and 10 Firstly Figure 9 givesus a visual comparison of the different approaches Fromalmost the same initial state (eg position and velocity) ourapproach could lead to autonomous vehicle passing throughthe intersection more quickly and reasonable
Then let us look at Figure 10 for detailed explanationIn the first 12 s in Figures 10(a) and 10(c) autonomousvehicle maintains speed and understands that human-drivenvehicle will not perform yielding actions Then autonomousvehicle gets yielding intention of human-driven vehicle andunderstands that human-driven vehiclersquos lateral intentionis to go straight Based on candidate policies autonomousvehicle selects acceleration strategy with maximum rewardand finally crosses the intersection In this process we canobviously find that autonomous vehicle understands human-driven vehiclersquos yielding intention Figure 10(c) is an exampleof understand human-driven vehiclersquos behavior based on egovehiclersquos future actions in a specific time Our strategy pre-dicts the future actions of human-driven vehicle Althoughthe velocity curves after 1 s do not correspond it does notaffect the performance of our methods The reason is that
12 Mathematical Problems in Engineering
0 1 2 3 4 5 6 7 80
1020
Time (s)
Velo
city
(ms
) Velocity profile
0 1 2 3 4 5 6 7 8024
Time (s)
Inte
ntio
n Longitudinal intention
0 1 2 3 4 5 6 7 8024
Time (s)
Inte
ntio
n Lateral intention
(a)
0 1 2 3 4 5 6 7 8 90
1020
Time (s)
Velo
city
(ms
) Velocity profile
0 1 2 3 4 5 6 7 8 9024
Time (s)
Inte
ntio
n
Longitudinal intention
0 1 2 3 4 5 6 7 8 9024
Time (s)
Inte
ntio
n
Lateral intention
(b)
0 1 2 3 4 5 6 7 8minus60
minus40
minus20
0
20
40
60
80
Time (s)
Distance to collision region
Dist
ance
(m)
(c)
0 1 2 3 4 5 6 7 8 9minus10
0
10
20
30
40
50
60
70
80
Time (s)
Dist
ance
(m)
Distance to collision region
(d)
0 1 2 3 4 5 6 7 87
8
9
10
11
12
13
14
15
Time (s)
Velo
city
(ms
)
(e)
Time (s)0 1 2 3 4 5 6 7 8
0
2
4
6
8
10
12
14
Velo
city
(ms
)
(f)
Figure 10 Case test 1 In this case human-driven vehicle passes through intersection after autonomous vehicle (a) (c) and (e) are theperformance of our method while (b) (d) and (f) are from the strategy without considering the driving intention (a) and (b) are the velocityprofiles and the corresponding driving intention For longitudinal intention label 1 means yielding and label 2 means not yielding In lateralintention 1 means turning left 2 means turning right 3 means going straight and 4 means stop The intentions in (b) are not used in thatmethod but for detailed analysis (c) and (d) are the distance to collision area for autonomous vehicle and human-driven vehicle respectively(e) and (f) are the prediction and true motions of human-driven vehicles in time 15 s with a prediction length of 8 s The red curves in thesesubfigures are from autonomous vehicle while blue lines are from human-driven vehicle The green lines in (e) and (f) are the predictionvelocity curves of human-driven vehicle
Mathematical Problems in Engineering 13
T = 4 s
T = 1 s
T = 2 s
T = 3 s
T = 5 s
T = 0 s
T = 0 1 2 5 s
(a)
T = 4 s
T = 1 s
T = 2 s
T = 3 s
T = 5 s
T = 0 s
T = 0 1 2 5 s
(b)
Figure 11 The visualized passing sequence for the case of human-driven vehicle first getting through (a) is the result of our approach and(b) is the reactive-based approach
we use a deterministic model in the prediction process andthe prediction value is inside two boundaries to ensure safetyBesides the whole actions of autonomous vehicle in thisprocess could also help human-driven vehicle to understandnot-yielding intention of autonomous vehicles In this casecooperative driving behaviors are performed by both vehi-cles
However if the intention is not considered in this processwe can find the results in Figures 10(b) 10(d) and 10(f)After 2 s in Figure 10(b) while the human-driven vehiclegives a yielding intention autonomous vehicle could notunderstand and they find a potential collision based on theconstant velocity assumptions Then it decreases the speedbut the human-driven vehicle also slows down The puzzledbehavior leads both vehicles to slow down near intersectionFinally human-driven vehicle stops at the stop line and thenautonomous vehicle could pass the intersection In this strat-egy the human-driven vehiclersquos future motion is assumed tobe constant (Figure 10(f))Without understanding of human-driven vehiclersquos intentions this strategy can increase conges-tion problem
Another experiment is that human-driven vehicle tries toget through the intersection first The results are shown inFigures 11 and 12This case is quite typical because many traf-fic accidents in real world are happening in this situation Indetail if one vehicle tries to cross an intersection while violat-ing the law another vehicle will be in great danger if it doesnot understand its behavior From the visualized perform-ance in Figure 11 our method is a little more safe than otherapproaches as there is nearly collision situation in Figure11(b) In detail we can see from Figure 12(a) that our strategycould perform deceleration actions after we understand thenot-yielding intention in 08 sHowever without understand-ing human-driven vehiclersquos motion intention the responsetime has a 1-second delay which may be quite dangerous
In addition it is shown that good performance is in thepredictions of human-driven vehiclersquos future motion in ourmethods (Figure 12(e))
The results of these two cases demonstrate that our algo-rithm could deal with typical scenarios and have better per-formance than traditional reactive controller Autonomousvehicle could be driven more safely fast and comfortablythrough our strategy
7 Conclusion and Future Work
In this paper we proposed an autonomous driving decision-making algorithm considering human-driven vehiclersquos uncer-tain intentions in an uncontrolled intersection The lat-eral and longitudinal intentions are recognized by a con-tinuous HMM Based on HMM and POMDP we modelgeneral decision-making process and then use an approxi-mate approach to solve this complex problem Finally weuse PreScan software and a driving simulator to emulatesocial interaction process The experiment results show thatautonomous vehicles with our approach can pass throughuncontrolled intersections more safely and efficiently thanusing the strategy without considering human-driven vehi-clesrsquo driving intentions
In the near future we aim to implement our approach intoa real autonomous vehicle and perform real world experi-ments In additionmore precious intention recognition algo-rithm aims to be figured out Somemethods like probabilisticgraphic model can be used to get a distribution of each inten-tion Finally designing online POMDP planning algorithmsis also valuable
Competing Interests
The authors declare that there are no competing interestsregarding the publication of this paper
14 Mathematical Problems in Engineering
0 05 1 15 2 25 3 35 4 45 510
20
30
Time (s)
Velo
city
(ms
) Velocity profile
0 05 1 15 2 25 3 35 4 45 50
2
4
Time (s)
Inte
ntio
n
Longitudinal intention
0 05 1 15 2 25 3 35 4 45 5024
Inte
ntio
n Lateral intention
Time (s)
(a)
0 05 1 15 2 25 3 35 4 45 5102030
Time (s)
Velo
city
(ms
) Velocity profile
0 05 1 15 2 25 3 35 4 45 5024
Time (s)
Inte
ntio
n
Longitudinal intention
0 05 1 15 2 25 3 35 4 45 5024
Time (s)
Inte
ntio
n
Lateral intention
(b)
0 05 1 15 2 25 3 35 4 45 5
0
20
40
60
80
Dist
ance
(m)
Distance to collision region
Time (s)
minus40
minus20
(c)
0 05 1 15 2 25 3 35 4 45 5
0
20
40
60
80D
istan
ce (m
)
Distance to collision region
Time (s)
minus40
minus20
(d)
0 1 2 3 4 5 6 7 816
17
18
19
20
21
22
23
24
25
Velo
city
(ms
)
Time (s)
(e)
0 1 2 3 4 5 6 7 817
18
19
20
21
22
23
24
25
Time (s)
Velo
city
(ms
)
(f)
Figure 12 Case test 2 In this case human-driven vehicle passes through intersection before autonomous vehicle through different strategiesThe definition of each subfigure is the same as in Figure 10
Mathematical Problems in Engineering 15
Acknowledgments
This study is supported by the National Natural ScienceFoundation of China (no 91420203)
References
[1] C Urmson J Anhalt D Bagnell et al ldquoAutonomous driving inurban environments boss and the urban challengerdquo Journal ofField Robotics vol 25 no 8 pp 425ndash466 2008
[2] J Markoff ldquoGoogle cars drive themselves in trafficrdquo New YorkTimes vol 9 2010
[3] L R Rabiner and B-H Juang ldquoAn introduction to hiddenMarkov modelsrdquo IEEE ASSP Magazine vol 3 no 1 pp 4ndash161986
[4] M Buehler K Iagnemma and S Singh The DARPA UrbanChallenge Autonomous Vehicles in City Traffic vol 56 Springer2009
[5] S Kammel J Ziegler B Pitzer et al ldquoTeam AnnieWAYrsquos auto-nomous system for the 2007 DARPAUrban Challengerdquo Journalof Field Robotics vol 25 no 9 pp 615ndash639 2008
[6] C R Baker and J M Dolan ldquoTraffic interaction in the urbanchallenge putting boss on its best behaviorrdquo in Proceedingsof the IEEERSJ International Conference on Intelligent Robotsand Systems (IROS rsquo08) pp 1752ndash1758 IEEE Nice FranceSeptember 2008
[7] M Montemerlo J Becker S Shat et al ldquoJunior the Stanfordentry in the urban challengerdquo Journal of Field Robotics vol 25no 9 pp 569ndash597 2008
[8] L Fletcher S Teller E Olson et al ldquoTheMIT-Cornell collisionand why it happenedrdquo Journal of Field Robotics vol 25 no 10pp 775ndash807 2008
[9] M Bahram A Wolf M Aeberhard and D Wollherr ldquoAprediction-based reactive driving strategy for highly automateddriving function on freewaysrdquo in Proceedings of the 25th IEEEIntelligent Vehicles Symposium pp 400ndash406 IEEE DearbornMich USA June 2014
[10] J Wei J M Dolan and B Litkouhi ldquoA prediction- and costfunction-based algorithm for robust autonomous freeway driv-ingrdquo in Proceedings of the IEEE Intelligent Vehicles Symposium(IV rsquo10) pp 512ndash517 San Diego Calif USA June 2010
[11] J Wei J M Dolan and B Litkouhi ldquoAutonomous vehiclesocial behavior for highway entrance ramp managementrdquo inProceedings of the IEEE Intelligent Vehicles Symposium (IV rsquo13)pp 201ndash207 IEEE Gold Coast Australia June 2013
[12] H Bai S Cai N Ye D Hsu and W S Lee ldquoIntention-awareonline POMDP planning for autonomous driving in a crowdrdquoin Proceedings of the IEEE International Conference on Roboticsand Automation (ICRA rsquo15) pp 454ndash460 IEEE Seattle WashUSA May 2015
[13] A Somani N Ye D Hsu and W S Lee ldquoDespot onlinepomdp planning with regularizationrdquo in Advances in NeuralInformation Processing Systems pp 1772ndash1780 2013
[14] S Brechtel T Gindele and R Dillmann ldquoProbabilistic deci-sion-making under uncertainty for autonomous driving usingcontinuous POMDPsrdquo in Proceedings of the 17th IEEE Interna-tional Conference on Intelligent Transportation Systems (ITSCrsquo14) pp 392ndash399 Qingdao China October 2014
[15] C H Papadimitriou and J N Tsitsiklis ldquoThe complexity ofMarkov decision processesrdquo Mathematics of Operations Re-search vol 12 no 3 pp 441ndash450 1987
[16] O Madani S Hanks and A Condon ldquoOn the undecidabilityof probabilistic planning and related stochastic optimizationproblemsrdquoArtificial Intelligence vol 147 no 1-2 pp 5ndash34 2003
[17] S Ulbrich and M Maurer ldquoProbabilistic online POMDP deci-sion making for lane changes in fully automated drivingrdquo inProceedings of the 16th International IEEE Conference on Intelli-gent Transportation Systems (ITSC rsquo13) pp 2063ndash2067 TheHague The Netherlands October 2013
[18] A G Cunningham E Galceran R M Eustice and E OlsonldquoMPDM multipolicy decision-making in dynamic uncertainenvironments for autonomous drivingrdquo in Proceedings of theIEEE International Conference on Robotics and Automation(ICRA rsquo15) pp 1670ndash1677 Seattle Wash USA May 2015
[19] D A Reynolds and R C Rose ldquoRobust text-independentspeaker identification using Gaussian mixture speaker modelsrdquoIEEE Transactions on Speech and Audio Processing vol 3 no 1pp 72ndash83 1995
[20] L E Baum and T Petrie ldquoStatistical inference for probabilisticfunctions of finite state Markov chainsrdquo The Annals of Mathe-matical Statistics vol 37 pp 1554ndash1563 1966
[21] M Tideman and M Van Noort ldquoA simulation tool suite fordeveloping connected vehicle systemsrdquo in Proceedings of theIEEE Intelligent Vehicles Symposium (IEEE IV rsquo13) pp 713ndash718Queensland Australia June 2013
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Mathematical Problems in Engineering 9
Law(119879119909 119879119910) larr 0
119905119909larr 1198891199092119868V119909
119905119910larr 1198891199102119868V119910
if 119905119909lt 119905119910minus Δ119905 then
Law(119879119909 119879119910) larr 1
else if |119905119909minus 119905119910| lt Δ119905 then
statuslarr 119865(119879119909 119879119910)
if status = 1 thenif 119868lat119909 = lanekeeping and 119868lat119910 ltgt lanekeeping thenLaw(119879
119909 119879119910) larr 1
else if 119868lat119909 119868lat119910 = lanekeeping and 119860119909minus 119860119910= 1 or minus3 then
Law(119879119909 119879119910) larr 1
else if 119868lat119909 = turnleft and 119868lat119910 = turnright thenLaw(119879
119909 119879119910) larr 1
end ifend if
end ifreturn Law(119879
119909 119879119910)
Algorithm 1 Traffic law formulization
InputPredict horizon119867 time step Δ119905Current states 119904host = 119909host 119904human = [119909human 119868human]
(1) 119875 larr genenratepolicyset()(2) for each 120587
119896isin 119875 do
(3) for 119894 = 1 to119867Δ119905 do(4) 119886host larr 120587
119896(119894)
(5) 1199041015840
host larr updatestate(119904host 119886host)(6) 119886human larr predictactions(1199041015840host 119904human 119868human)
(7) 1199091015840
human larr updatestate(119909human 119886human)
(8) 1198681015840
human larr updateintention(1199041015840host 1199091015840
human)
(9) 1199041015840
human larr [1199091015840
human 1198681015840
human]
(10) 119877(119894) = calculatereward(1199041015840host 1199041015840
human)
(11) 119904host larr 1199041015840
host(12) 119904human larr 119904
1015840
human(13) end(14) 119877
total119896
larr sum119894sum(119877(119894))
(15) end(16) 119896lowast larr argmax
119896(119877
total119896
)
(17) 120587lowast larr 120587119896lowast
(18) return 120587lowast
Algorithm 2 Policy selection process
the value of 1199051 1199052 and 119905
3and the prediction period of single
step An example of policy generation is shown in Figure 6
52 Planning Horizon Selection After building policy gener-ation model the next problem is to select a suitable planninghorizon Longer horizon can lead to a better solution but con-suming more computing resources However as our purpose
is to deal with the interaction problem in the uncontrolledintersection we only need to consider the situation beforeautonomous vehicle gets through In our algorithmwe set theprediction horizon as 8 seconds In addition in the process ofupdating the future state of each vehicle using each policy thecar following mode is used after autonomous vehicle passesthrough the intersection area
10 Mathematical Problems in Engineering
0
1
2Candidate policies
minus4
minus3
minus2
minus1
Acce
lera
tion
(ms2)
1 2 3 4 5 6 7 80Time (s)
(a)
Time (s)0 1 2 3 4 5 6 7 8
0
2
4
6
8
10
12
14
16
18
20
Velo
city
(ms
)
Velocity strategies
(b)
Figure 6 An example of policy generation process (a) is the generated policies and (b) is the corresponding speed profiles The interval ofeach prediction step is 05 s current speed is 12ms2 and the speed limit is 20ms2 The bold black line is one policy In the first 3 secondsautonomous vehicles decelerate in minus35ms2 then accelerate at 2ms2 for 4 seconds and finally stop in the last one second In this case 109policies were generated which is suitable for replanning fast
AB
Figure 7 Testing scenario Autonomous vehicle B and human-driven vehicle A are both approaching the uncontrolled intersection To goacross the intersection successfully autonomous vehicle should interact with human-driven vehicle
53 Time Step Selection Another problem is the predic-tion time step The intention prediction algorithm and thePOMDP are computed in each step If the time step is 119905stepthe total computation timeswill be119867119905stepThus smaller timestep leads to more computation time To solve this problemwe use a simple adaptive time step calculation mechanism togive a final value The time step is selected based on the TTCof autonomous vehicle If the host vehicle is far away from theintersection we can use a very large time step But if the TTCis quite small the low 119905step is applied to ensure safety
6 Experiment and Results
61 Settings In this paper we evaluate our approach throughPreScan 710 [21] a simulation tool for autonomous drivingand connected vehicles Using this software we can build thetesting scenarios (Figure 7) and add vehicles with dynamicmodel In order to get a similar scenario considering social
interaction the driver simulator is added in our experiment(Figure 8) The human-driven vehicle is driven by severalpeople during the experiment and the autonomous vehiclemakes decisions based on the human-driven vehiclersquos drivingbehavior The reference trajectory for autonomous vehicleis generated from path planning module and the human-driven vehiclersquos data (eg position velocity and heading)are transferred through V2V communication sensor Thedecision-making module sends desired velocity command tothe PID controlled to follow the reference path All policies inthe experiment part use a planning horizon119867 = 8 s which isdiscretized into the time step of 05 s
62 Results It is difficult to compare different approachesin the same scenario because the environment is dynamicand not exactly the same However we select two typicalsituations and special settings to make it possible The sameinitial conditions including position orientation and velocity
Mathematical Problems in Engineering 11
Figure 8 Logitech G27 driving simulator
T = 0 1 2 6 s T = 5 s
T = 4 s
T = 3 s
T = 2 s
T = 1 s
T = 0 s
T = 6 s
(a)
T = 0 1 2 9 s
T = 4 s
T = 5 s
T = 6 sT = 7 sT = 8 s
T = 9 s
T = 3 s
T = 2 s
T = 1 s
T = 0 s
(b)
Figure 9 The visualized passing sequence (a) is the result of our approach and (b) represents the result of reactive approach withoutconsidering intention The black vehicle is an autonomous vehicle while the red car is the human-driven vehicle Each vehicle representsthe position in a specific time 119879 with an interval of 1 second
for each vehicle are used in different tests Besides twotypical situations including human-driven vehicle gettingthrough before or after autonomous vehicle are comparedin this section With the same initial state different reac-tions will occur based on various methods We compareour approach and reactive-based method [6] in this sec-tion The key difference for these two methods is that ourapproach considers human-driven vehiclersquos driving inten-tion
The first experiment is that human-driven vehicle triesto yield autonomous vehicle in the interaction process Theresults are shown in Figures 9 and 10 Firstly Figure 9 givesus a visual comparison of the different approaches Fromalmost the same initial state (eg position and velocity) ourapproach could lead to autonomous vehicle passing throughthe intersection more quickly and reasonable
Then let us look at Figure 10 for detailed explanationIn the first 12 s in Figures 10(a) and 10(c) autonomousvehicle maintains speed and understands that human-drivenvehicle will not perform yielding actions Then autonomousvehicle gets yielding intention of human-driven vehicle andunderstands that human-driven vehiclersquos lateral intentionis to go straight Based on candidate policies autonomousvehicle selects acceleration strategy with maximum rewardand finally crosses the intersection In this process we canobviously find that autonomous vehicle understands human-driven vehiclersquos yielding intention Figure 10(c) is an exampleof understand human-driven vehiclersquos behavior based on egovehiclersquos future actions in a specific time Our strategy pre-dicts the future actions of human-driven vehicle Althoughthe velocity curves after 1 s do not correspond it does notaffect the performance of our methods The reason is that
12 Mathematical Problems in Engineering
0 1 2 3 4 5 6 7 80
1020
Time (s)
Velo
city
(ms
) Velocity profile
0 1 2 3 4 5 6 7 8024
Time (s)
Inte
ntio
n Longitudinal intention
0 1 2 3 4 5 6 7 8024
Time (s)
Inte
ntio
n Lateral intention
(a)
0 1 2 3 4 5 6 7 8 90
1020
Time (s)
Velo
city
(ms
) Velocity profile
0 1 2 3 4 5 6 7 8 9024
Time (s)
Inte
ntio
n
Longitudinal intention
0 1 2 3 4 5 6 7 8 9024
Time (s)
Inte
ntio
n
Lateral intention
(b)
0 1 2 3 4 5 6 7 8minus60
minus40
minus20
0
20
40
60
80
Time (s)
Distance to collision region
Dist
ance
(m)
(c)
0 1 2 3 4 5 6 7 8 9minus10
0
10
20
30
40
50
60
70
80
Time (s)
Dist
ance
(m)
Distance to collision region
(d)
0 1 2 3 4 5 6 7 87
8
9
10
11
12
13
14
15
Time (s)
Velo
city
(ms
)
(e)
Time (s)0 1 2 3 4 5 6 7 8
0
2
4
6
8
10
12
14
Velo
city
(ms
)
(f)
Figure 10 Case test 1 In this case human-driven vehicle passes through intersection after autonomous vehicle (a) (c) and (e) are theperformance of our method while (b) (d) and (f) are from the strategy without considering the driving intention (a) and (b) are the velocityprofiles and the corresponding driving intention For longitudinal intention label 1 means yielding and label 2 means not yielding In lateralintention 1 means turning left 2 means turning right 3 means going straight and 4 means stop The intentions in (b) are not used in thatmethod but for detailed analysis (c) and (d) are the distance to collision area for autonomous vehicle and human-driven vehicle respectively(e) and (f) are the prediction and true motions of human-driven vehicles in time 15 s with a prediction length of 8 s The red curves in thesesubfigures are from autonomous vehicle while blue lines are from human-driven vehicle The green lines in (e) and (f) are the predictionvelocity curves of human-driven vehicle
Mathematical Problems in Engineering 13
T = 4 s
T = 1 s
T = 2 s
T = 3 s
T = 5 s
T = 0 s
T = 0 1 2 5 s
(a)
T = 4 s
T = 1 s
T = 2 s
T = 3 s
T = 5 s
T = 0 s
T = 0 1 2 5 s
(b)
Figure 11 The visualized passing sequence for the case of human-driven vehicle first getting through (a) is the result of our approach and(b) is the reactive-based approach
we use a deterministic model in the prediction process andthe prediction value is inside two boundaries to ensure safetyBesides the whole actions of autonomous vehicle in thisprocess could also help human-driven vehicle to understandnot-yielding intention of autonomous vehicles In this casecooperative driving behaviors are performed by both vehi-cles
However if the intention is not considered in this processwe can find the results in Figures 10(b) 10(d) and 10(f)After 2 s in Figure 10(b) while the human-driven vehiclegives a yielding intention autonomous vehicle could notunderstand and they find a potential collision based on theconstant velocity assumptions Then it decreases the speedbut the human-driven vehicle also slows down The puzzledbehavior leads both vehicles to slow down near intersectionFinally human-driven vehicle stops at the stop line and thenautonomous vehicle could pass the intersection In this strat-egy the human-driven vehiclersquos future motion is assumed tobe constant (Figure 10(f))Without understanding of human-driven vehiclersquos intentions this strategy can increase conges-tion problem
Another experiment is that human-driven vehicle tries toget through the intersection first The results are shown inFigures 11 and 12This case is quite typical because many traf-fic accidents in real world are happening in this situation Indetail if one vehicle tries to cross an intersection while violat-ing the law another vehicle will be in great danger if it doesnot understand its behavior From the visualized perform-ance in Figure 11 our method is a little more safe than otherapproaches as there is nearly collision situation in Figure11(b) In detail we can see from Figure 12(a) that our strategycould perform deceleration actions after we understand thenot-yielding intention in 08 sHowever without understand-ing human-driven vehiclersquos motion intention the responsetime has a 1-second delay which may be quite dangerous
In addition it is shown that good performance is in thepredictions of human-driven vehiclersquos future motion in ourmethods (Figure 12(e))
The results of these two cases demonstrate that our algo-rithm could deal with typical scenarios and have better per-formance than traditional reactive controller Autonomousvehicle could be driven more safely fast and comfortablythrough our strategy
7 Conclusion and Future Work
In this paper we proposed an autonomous driving decision-making algorithm considering human-driven vehiclersquos uncer-tain intentions in an uncontrolled intersection The lat-eral and longitudinal intentions are recognized by a con-tinuous HMM Based on HMM and POMDP we modelgeneral decision-making process and then use an approxi-mate approach to solve this complex problem Finally weuse PreScan software and a driving simulator to emulatesocial interaction process The experiment results show thatautonomous vehicles with our approach can pass throughuncontrolled intersections more safely and efficiently thanusing the strategy without considering human-driven vehi-clesrsquo driving intentions
In the near future we aim to implement our approach intoa real autonomous vehicle and perform real world experi-ments In additionmore precious intention recognition algo-rithm aims to be figured out Somemethods like probabilisticgraphic model can be used to get a distribution of each inten-tion Finally designing online POMDP planning algorithmsis also valuable
Competing Interests
The authors declare that there are no competing interestsregarding the publication of this paper
14 Mathematical Problems in Engineering
0 05 1 15 2 25 3 35 4 45 510
20
30
Time (s)
Velo
city
(ms
) Velocity profile
0 05 1 15 2 25 3 35 4 45 50
2
4
Time (s)
Inte
ntio
n
Longitudinal intention
0 05 1 15 2 25 3 35 4 45 5024
Inte
ntio
n Lateral intention
Time (s)
(a)
0 05 1 15 2 25 3 35 4 45 5102030
Time (s)
Velo
city
(ms
) Velocity profile
0 05 1 15 2 25 3 35 4 45 5024
Time (s)
Inte
ntio
n
Longitudinal intention
0 05 1 15 2 25 3 35 4 45 5024
Time (s)
Inte
ntio
n
Lateral intention
(b)
0 05 1 15 2 25 3 35 4 45 5
0
20
40
60
80
Dist
ance
(m)
Distance to collision region
Time (s)
minus40
minus20
(c)
0 05 1 15 2 25 3 35 4 45 5
0
20
40
60
80D
istan
ce (m
)
Distance to collision region
Time (s)
minus40
minus20
(d)
0 1 2 3 4 5 6 7 816
17
18
19
20
21
22
23
24
25
Velo
city
(ms
)
Time (s)
(e)
0 1 2 3 4 5 6 7 817
18
19
20
21
22
23
24
25
Time (s)
Velo
city
(ms
)
(f)
Figure 12 Case test 2 In this case human-driven vehicle passes through intersection before autonomous vehicle through different strategiesThe definition of each subfigure is the same as in Figure 10
Mathematical Problems in Engineering 15
Acknowledgments
This study is supported by the National Natural ScienceFoundation of China (no 91420203)
References
[1] C Urmson J Anhalt D Bagnell et al ldquoAutonomous driving inurban environments boss and the urban challengerdquo Journal ofField Robotics vol 25 no 8 pp 425ndash466 2008
[2] J Markoff ldquoGoogle cars drive themselves in trafficrdquo New YorkTimes vol 9 2010
[3] L R Rabiner and B-H Juang ldquoAn introduction to hiddenMarkov modelsrdquo IEEE ASSP Magazine vol 3 no 1 pp 4ndash161986
[4] M Buehler K Iagnemma and S Singh The DARPA UrbanChallenge Autonomous Vehicles in City Traffic vol 56 Springer2009
[5] S Kammel J Ziegler B Pitzer et al ldquoTeam AnnieWAYrsquos auto-nomous system for the 2007 DARPAUrban Challengerdquo Journalof Field Robotics vol 25 no 9 pp 615ndash639 2008
[6] C R Baker and J M Dolan ldquoTraffic interaction in the urbanchallenge putting boss on its best behaviorrdquo in Proceedingsof the IEEERSJ International Conference on Intelligent Robotsand Systems (IROS rsquo08) pp 1752ndash1758 IEEE Nice FranceSeptember 2008
[7] M Montemerlo J Becker S Shat et al ldquoJunior the Stanfordentry in the urban challengerdquo Journal of Field Robotics vol 25no 9 pp 569ndash597 2008
[8] L Fletcher S Teller E Olson et al ldquoTheMIT-Cornell collisionand why it happenedrdquo Journal of Field Robotics vol 25 no 10pp 775ndash807 2008
[9] M Bahram A Wolf M Aeberhard and D Wollherr ldquoAprediction-based reactive driving strategy for highly automateddriving function on freewaysrdquo in Proceedings of the 25th IEEEIntelligent Vehicles Symposium pp 400ndash406 IEEE DearbornMich USA June 2014
[10] J Wei J M Dolan and B Litkouhi ldquoA prediction- and costfunction-based algorithm for robust autonomous freeway driv-ingrdquo in Proceedings of the IEEE Intelligent Vehicles Symposium(IV rsquo10) pp 512ndash517 San Diego Calif USA June 2010
[11] J Wei J M Dolan and B Litkouhi ldquoAutonomous vehiclesocial behavior for highway entrance ramp managementrdquo inProceedings of the IEEE Intelligent Vehicles Symposium (IV rsquo13)pp 201ndash207 IEEE Gold Coast Australia June 2013
[12] H Bai S Cai N Ye D Hsu and W S Lee ldquoIntention-awareonline POMDP planning for autonomous driving in a crowdrdquoin Proceedings of the IEEE International Conference on Roboticsand Automation (ICRA rsquo15) pp 454ndash460 IEEE Seattle WashUSA May 2015
[13] A Somani N Ye D Hsu and W S Lee ldquoDespot onlinepomdp planning with regularizationrdquo in Advances in NeuralInformation Processing Systems pp 1772ndash1780 2013
[14] S Brechtel T Gindele and R Dillmann ldquoProbabilistic deci-sion-making under uncertainty for autonomous driving usingcontinuous POMDPsrdquo in Proceedings of the 17th IEEE Interna-tional Conference on Intelligent Transportation Systems (ITSCrsquo14) pp 392ndash399 Qingdao China October 2014
[15] C H Papadimitriou and J N Tsitsiklis ldquoThe complexity ofMarkov decision processesrdquo Mathematics of Operations Re-search vol 12 no 3 pp 441ndash450 1987
[16] O Madani S Hanks and A Condon ldquoOn the undecidabilityof probabilistic planning and related stochastic optimizationproblemsrdquoArtificial Intelligence vol 147 no 1-2 pp 5ndash34 2003
[17] S Ulbrich and M Maurer ldquoProbabilistic online POMDP deci-sion making for lane changes in fully automated drivingrdquo inProceedings of the 16th International IEEE Conference on Intelli-gent Transportation Systems (ITSC rsquo13) pp 2063ndash2067 TheHague The Netherlands October 2013
[18] A G Cunningham E Galceran R M Eustice and E OlsonldquoMPDM multipolicy decision-making in dynamic uncertainenvironments for autonomous drivingrdquo in Proceedings of theIEEE International Conference on Robotics and Automation(ICRA rsquo15) pp 1670ndash1677 Seattle Wash USA May 2015
[19] D A Reynolds and R C Rose ldquoRobust text-independentspeaker identification using Gaussian mixture speaker modelsrdquoIEEE Transactions on Speech and Audio Processing vol 3 no 1pp 72ndash83 1995
[20] L E Baum and T Petrie ldquoStatistical inference for probabilisticfunctions of finite state Markov chainsrdquo The Annals of Mathe-matical Statistics vol 37 pp 1554ndash1563 1966
[21] M Tideman and M Van Noort ldquoA simulation tool suite fordeveloping connected vehicle systemsrdquo in Proceedings of theIEEE Intelligent Vehicles Symposium (IEEE IV rsquo13) pp 713ndash718Queensland Australia June 2013
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
10 Mathematical Problems in Engineering
0
1
2Candidate policies
minus4
minus3
minus2
minus1
Acce
lera
tion
(ms2)
1 2 3 4 5 6 7 80Time (s)
(a)
Time (s)0 1 2 3 4 5 6 7 8
0
2
4
6
8
10
12
14
16
18
20
Velo
city
(ms
)
Velocity strategies
(b)
Figure 6 An example of policy generation process (a) is the generated policies and (b) is the corresponding speed profiles The interval ofeach prediction step is 05 s current speed is 12ms2 and the speed limit is 20ms2 The bold black line is one policy In the first 3 secondsautonomous vehicles decelerate in minus35ms2 then accelerate at 2ms2 for 4 seconds and finally stop in the last one second In this case 109policies were generated which is suitable for replanning fast
AB
Figure 7 Testing scenario Autonomous vehicle B and human-driven vehicle A are both approaching the uncontrolled intersection To goacross the intersection successfully autonomous vehicle should interact with human-driven vehicle
53 Time Step Selection Another problem is the predic-tion time step The intention prediction algorithm and thePOMDP are computed in each step If the time step is 119905stepthe total computation timeswill be119867119905stepThus smaller timestep leads to more computation time To solve this problemwe use a simple adaptive time step calculation mechanism togive a final value The time step is selected based on the TTCof autonomous vehicle If the host vehicle is far away from theintersection we can use a very large time step But if the TTCis quite small the low 119905step is applied to ensure safety
6 Experiment and Results
61 Settings In this paper we evaluate our approach throughPreScan 710 [21] a simulation tool for autonomous drivingand connected vehicles Using this software we can build thetesting scenarios (Figure 7) and add vehicles with dynamicmodel In order to get a similar scenario considering social
interaction the driver simulator is added in our experiment(Figure 8) The human-driven vehicle is driven by severalpeople during the experiment and the autonomous vehiclemakes decisions based on the human-driven vehiclersquos drivingbehavior The reference trajectory for autonomous vehicleis generated from path planning module and the human-driven vehiclersquos data (eg position velocity and heading)are transferred through V2V communication sensor Thedecision-making module sends desired velocity command tothe PID controlled to follow the reference path All policies inthe experiment part use a planning horizon119867 = 8 s which isdiscretized into the time step of 05 s
62 Results It is difficult to compare different approachesin the same scenario because the environment is dynamicand not exactly the same However we select two typicalsituations and special settings to make it possible The sameinitial conditions including position orientation and velocity
Mathematical Problems in Engineering 11
Figure 8 Logitech G27 driving simulator
T = 0 1 2 6 s T = 5 s
T = 4 s
T = 3 s
T = 2 s
T = 1 s
T = 0 s
T = 6 s
(a)
T = 0 1 2 9 s
T = 4 s
T = 5 s
T = 6 sT = 7 sT = 8 s
T = 9 s
T = 3 s
T = 2 s
T = 1 s
T = 0 s
(b)
Figure 9 The visualized passing sequence (a) is the result of our approach and (b) represents the result of reactive approach withoutconsidering intention The black vehicle is an autonomous vehicle while the red car is the human-driven vehicle Each vehicle representsthe position in a specific time 119879 with an interval of 1 second
for each vehicle are used in different tests Besides twotypical situations including human-driven vehicle gettingthrough before or after autonomous vehicle are comparedin this section With the same initial state different reac-tions will occur based on various methods We compareour approach and reactive-based method [6] in this sec-tion The key difference for these two methods is that ourapproach considers human-driven vehiclersquos driving inten-tion
The first experiment is that human-driven vehicle triesto yield autonomous vehicle in the interaction process Theresults are shown in Figures 9 and 10 Firstly Figure 9 givesus a visual comparison of the different approaches Fromalmost the same initial state (eg position and velocity) ourapproach could lead to autonomous vehicle passing throughthe intersection more quickly and reasonable
Then let us look at Figure 10 for detailed explanationIn the first 12 s in Figures 10(a) and 10(c) autonomousvehicle maintains speed and understands that human-drivenvehicle will not perform yielding actions Then autonomousvehicle gets yielding intention of human-driven vehicle andunderstands that human-driven vehiclersquos lateral intentionis to go straight Based on candidate policies autonomousvehicle selects acceleration strategy with maximum rewardand finally crosses the intersection In this process we canobviously find that autonomous vehicle understands human-driven vehiclersquos yielding intention Figure 10(c) is an exampleof understand human-driven vehiclersquos behavior based on egovehiclersquos future actions in a specific time Our strategy pre-dicts the future actions of human-driven vehicle Althoughthe velocity curves after 1 s do not correspond it does notaffect the performance of our methods The reason is that
12 Mathematical Problems in Engineering
0 1 2 3 4 5 6 7 80
1020
Time (s)
Velo
city
(ms
) Velocity profile
0 1 2 3 4 5 6 7 8024
Time (s)
Inte
ntio
n Longitudinal intention
0 1 2 3 4 5 6 7 8024
Time (s)
Inte
ntio
n Lateral intention
(a)
0 1 2 3 4 5 6 7 8 90
1020
Time (s)
Velo
city
(ms
) Velocity profile
0 1 2 3 4 5 6 7 8 9024
Time (s)
Inte
ntio
n
Longitudinal intention
0 1 2 3 4 5 6 7 8 9024
Time (s)
Inte
ntio
n
Lateral intention
(b)
0 1 2 3 4 5 6 7 8minus60
minus40
minus20
0
20
40
60
80
Time (s)
Distance to collision region
Dist
ance
(m)
(c)
0 1 2 3 4 5 6 7 8 9minus10
0
10
20
30
40
50
60
70
80
Time (s)
Dist
ance
(m)
Distance to collision region
(d)
0 1 2 3 4 5 6 7 87
8
9
10
11
12
13
14
15
Time (s)
Velo
city
(ms
)
(e)
Time (s)0 1 2 3 4 5 6 7 8
0
2
4
6
8
10
12
14
Velo
city
(ms
)
(f)
Figure 10 Case test 1 In this case human-driven vehicle passes through intersection after autonomous vehicle (a) (c) and (e) are theperformance of our method while (b) (d) and (f) are from the strategy without considering the driving intention (a) and (b) are the velocityprofiles and the corresponding driving intention For longitudinal intention label 1 means yielding and label 2 means not yielding In lateralintention 1 means turning left 2 means turning right 3 means going straight and 4 means stop The intentions in (b) are not used in thatmethod but for detailed analysis (c) and (d) are the distance to collision area for autonomous vehicle and human-driven vehicle respectively(e) and (f) are the prediction and true motions of human-driven vehicles in time 15 s with a prediction length of 8 s The red curves in thesesubfigures are from autonomous vehicle while blue lines are from human-driven vehicle The green lines in (e) and (f) are the predictionvelocity curves of human-driven vehicle
Mathematical Problems in Engineering 13
T = 4 s
T = 1 s
T = 2 s
T = 3 s
T = 5 s
T = 0 s
T = 0 1 2 5 s
(a)
T = 4 s
T = 1 s
T = 2 s
T = 3 s
T = 5 s
T = 0 s
T = 0 1 2 5 s
(b)
Figure 11 The visualized passing sequence for the case of human-driven vehicle first getting through (a) is the result of our approach and(b) is the reactive-based approach
we use a deterministic model in the prediction process andthe prediction value is inside two boundaries to ensure safetyBesides the whole actions of autonomous vehicle in thisprocess could also help human-driven vehicle to understandnot-yielding intention of autonomous vehicles In this casecooperative driving behaviors are performed by both vehi-cles
However if the intention is not considered in this processwe can find the results in Figures 10(b) 10(d) and 10(f)After 2 s in Figure 10(b) while the human-driven vehiclegives a yielding intention autonomous vehicle could notunderstand and they find a potential collision based on theconstant velocity assumptions Then it decreases the speedbut the human-driven vehicle also slows down The puzzledbehavior leads both vehicles to slow down near intersectionFinally human-driven vehicle stops at the stop line and thenautonomous vehicle could pass the intersection In this strat-egy the human-driven vehiclersquos future motion is assumed tobe constant (Figure 10(f))Without understanding of human-driven vehiclersquos intentions this strategy can increase conges-tion problem
Another experiment is that human-driven vehicle tries toget through the intersection first The results are shown inFigures 11 and 12This case is quite typical because many traf-fic accidents in real world are happening in this situation Indetail if one vehicle tries to cross an intersection while violat-ing the law another vehicle will be in great danger if it doesnot understand its behavior From the visualized perform-ance in Figure 11 our method is a little more safe than otherapproaches as there is nearly collision situation in Figure11(b) In detail we can see from Figure 12(a) that our strategycould perform deceleration actions after we understand thenot-yielding intention in 08 sHowever without understand-ing human-driven vehiclersquos motion intention the responsetime has a 1-second delay which may be quite dangerous
In addition it is shown that good performance is in thepredictions of human-driven vehiclersquos future motion in ourmethods (Figure 12(e))
The results of these two cases demonstrate that our algo-rithm could deal with typical scenarios and have better per-formance than traditional reactive controller Autonomousvehicle could be driven more safely fast and comfortablythrough our strategy
7 Conclusion and Future Work
In this paper we proposed an autonomous driving decision-making algorithm considering human-driven vehiclersquos uncer-tain intentions in an uncontrolled intersection The lat-eral and longitudinal intentions are recognized by a con-tinuous HMM Based on HMM and POMDP we modelgeneral decision-making process and then use an approxi-mate approach to solve this complex problem Finally weuse PreScan software and a driving simulator to emulatesocial interaction process The experiment results show thatautonomous vehicles with our approach can pass throughuncontrolled intersections more safely and efficiently thanusing the strategy without considering human-driven vehi-clesrsquo driving intentions
In the near future we aim to implement our approach intoa real autonomous vehicle and perform real world experi-ments In additionmore precious intention recognition algo-rithm aims to be figured out Somemethods like probabilisticgraphic model can be used to get a distribution of each inten-tion Finally designing online POMDP planning algorithmsis also valuable
Competing Interests
The authors declare that there are no competing interestsregarding the publication of this paper
14 Mathematical Problems in Engineering
0 05 1 15 2 25 3 35 4 45 510
20
30
Time (s)
Velo
city
(ms
) Velocity profile
0 05 1 15 2 25 3 35 4 45 50
2
4
Time (s)
Inte
ntio
n
Longitudinal intention
0 05 1 15 2 25 3 35 4 45 5024
Inte
ntio
n Lateral intention
Time (s)
(a)
0 05 1 15 2 25 3 35 4 45 5102030
Time (s)
Velo
city
(ms
) Velocity profile
0 05 1 15 2 25 3 35 4 45 5024
Time (s)
Inte
ntio
n
Longitudinal intention
0 05 1 15 2 25 3 35 4 45 5024
Time (s)
Inte
ntio
n
Lateral intention
(b)
0 05 1 15 2 25 3 35 4 45 5
0
20
40
60
80
Dist
ance
(m)
Distance to collision region
Time (s)
minus40
minus20
(c)
0 05 1 15 2 25 3 35 4 45 5
0
20
40
60
80D
istan
ce (m
)
Distance to collision region
Time (s)
minus40
minus20
(d)
0 1 2 3 4 5 6 7 816
17
18
19
20
21
22
23
24
25
Velo
city
(ms
)
Time (s)
(e)
0 1 2 3 4 5 6 7 817
18
19
20
21
22
23
24
25
Time (s)
Velo
city
(ms
)
(f)
Figure 12 Case test 2 In this case human-driven vehicle passes through intersection before autonomous vehicle through different strategiesThe definition of each subfigure is the same as in Figure 10
Mathematical Problems in Engineering 15
Acknowledgments
This study is supported by the National Natural ScienceFoundation of China (no 91420203)
References
[1] C Urmson J Anhalt D Bagnell et al ldquoAutonomous driving inurban environments boss and the urban challengerdquo Journal ofField Robotics vol 25 no 8 pp 425ndash466 2008
[2] J Markoff ldquoGoogle cars drive themselves in trafficrdquo New YorkTimes vol 9 2010
[3] L R Rabiner and B-H Juang ldquoAn introduction to hiddenMarkov modelsrdquo IEEE ASSP Magazine vol 3 no 1 pp 4ndash161986
[4] M Buehler K Iagnemma and S Singh The DARPA UrbanChallenge Autonomous Vehicles in City Traffic vol 56 Springer2009
[5] S Kammel J Ziegler B Pitzer et al ldquoTeam AnnieWAYrsquos auto-nomous system for the 2007 DARPAUrban Challengerdquo Journalof Field Robotics vol 25 no 9 pp 615ndash639 2008
[6] C R Baker and J M Dolan ldquoTraffic interaction in the urbanchallenge putting boss on its best behaviorrdquo in Proceedingsof the IEEERSJ International Conference on Intelligent Robotsand Systems (IROS rsquo08) pp 1752ndash1758 IEEE Nice FranceSeptember 2008
[7] M Montemerlo J Becker S Shat et al ldquoJunior the Stanfordentry in the urban challengerdquo Journal of Field Robotics vol 25no 9 pp 569ndash597 2008
[8] L Fletcher S Teller E Olson et al ldquoTheMIT-Cornell collisionand why it happenedrdquo Journal of Field Robotics vol 25 no 10pp 775ndash807 2008
[9] M Bahram A Wolf M Aeberhard and D Wollherr ldquoAprediction-based reactive driving strategy for highly automateddriving function on freewaysrdquo in Proceedings of the 25th IEEEIntelligent Vehicles Symposium pp 400ndash406 IEEE DearbornMich USA June 2014
[10] J Wei J M Dolan and B Litkouhi ldquoA prediction- and costfunction-based algorithm for robust autonomous freeway driv-ingrdquo in Proceedings of the IEEE Intelligent Vehicles Symposium(IV rsquo10) pp 512ndash517 San Diego Calif USA June 2010
[11] J Wei J M Dolan and B Litkouhi ldquoAutonomous vehiclesocial behavior for highway entrance ramp managementrdquo inProceedings of the IEEE Intelligent Vehicles Symposium (IV rsquo13)pp 201ndash207 IEEE Gold Coast Australia June 2013
[12] H Bai S Cai N Ye D Hsu and W S Lee ldquoIntention-awareonline POMDP planning for autonomous driving in a crowdrdquoin Proceedings of the IEEE International Conference on Roboticsand Automation (ICRA rsquo15) pp 454ndash460 IEEE Seattle WashUSA May 2015
[13] A Somani N Ye D Hsu and W S Lee ldquoDespot onlinepomdp planning with regularizationrdquo in Advances in NeuralInformation Processing Systems pp 1772ndash1780 2013
[14] S Brechtel T Gindele and R Dillmann ldquoProbabilistic deci-sion-making under uncertainty for autonomous driving usingcontinuous POMDPsrdquo in Proceedings of the 17th IEEE Interna-tional Conference on Intelligent Transportation Systems (ITSCrsquo14) pp 392ndash399 Qingdao China October 2014
[15] C H Papadimitriou and J N Tsitsiklis ldquoThe complexity ofMarkov decision processesrdquo Mathematics of Operations Re-search vol 12 no 3 pp 441ndash450 1987
[16] O Madani S Hanks and A Condon ldquoOn the undecidabilityof probabilistic planning and related stochastic optimizationproblemsrdquoArtificial Intelligence vol 147 no 1-2 pp 5ndash34 2003
[17] S Ulbrich and M Maurer ldquoProbabilistic online POMDP deci-sion making for lane changes in fully automated drivingrdquo inProceedings of the 16th International IEEE Conference on Intelli-gent Transportation Systems (ITSC rsquo13) pp 2063ndash2067 TheHague The Netherlands October 2013
[18] A G Cunningham E Galceran R M Eustice and E OlsonldquoMPDM multipolicy decision-making in dynamic uncertainenvironments for autonomous drivingrdquo in Proceedings of theIEEE International Conference on Robotics and Automation(ICRA rsquo15) pp 1670ndash1677 Seattle Wash USA May 2015
[19] D A Reynolds and R C Rose ldquoRobust text-independentspeaker identification using Gaussian mixture speaker modelsrdquoIEEE Transactions on Speech and Audio Processing vol 3 no 1pp 72ndash83 1995
[20] L E Baum and T Petrie ldquoStatistical inference for probabilisticfunctions of finite state Markov chainsrdquo The Annals of Mathe-matical Statistics vol 37 pp 1554ndash1563 1966
[21] M Tideman and M Van Noort ldquoA simulation tool suite fordeveloping connected vehicle systemsrdquo in Proceedings of theIEEE Intelligent Vehicles Symposium (IEEE IV rsquo13) pp 713ndash718Queensland Australia June 2013
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Mathematical Problems in Engineering 11
Figure 8 Logitech G27 driving simulator
T = 0 1 2 6 s T = 5 s
T = 4 s
T = 3 s
T = 2 s
T = 1 s
T = 0 s
T = 6 s
(a)
T = 0 1 2 9 s
T = 4 s
T = 5 s
T = 6 sT = 7 sT = 8 s
T = 9 s
T = 3 s
T = 2 s
T = 1 s
T = 0 s
(b)
Figure 9 The visualized passing sequence (a) is the result of our approach and (b) represents the result of reactive approach withoutconsidering intention The black vehicle is an autonomous vehicle while the red car is the human-driven vehicle Each vehicle representsthe position in a specific time 119879 with an interval of 1 second
for each vehicle are used in different tests Besides twotypical situations including human-driven vehicle gettingthrough before or after autonomous vehicle are comparedin this section With the same initial state different reac-tions will occur based on various methods We compareour approach and reactive-based method [6] in this sec-tion The key difference for these two methods is that ourapproach considers human-driven vehiclersquos driving inten-tion
The first experiment is that human-driven vehicle triesto yield autonomous vehicle in the interaction process Theresults are shown in Figures 9 and 10 Firstly Figure 9 givesus a visual comparison of the different approaches Fromalmost the same initial state (eg position and velocity) ourapproach could lead to autonomous vehicle passing throughthe intersection more quickly and reasonable
Then let us look at Figure 10 for detailed explanationIn the first 12 s in Figures 10(a) and 10(c) autonomousvehicle maintains speed and understands that human-drivenvehicle will not perform yielding actions Then autonomousvehicle gets yielding intention of human-driven vehicle andunderstands that human-driven vehiclersquos lateral intentionis to go straight Based on candidate policies autonomousvehicle selects acceleration strategy with maximum rewardand finally crosses the intersection In this process we canobviously find that autonomous vehicle understands human-driven vehiclersquos yielding intention Figure 10(c) is an exampleof understand human-driven vehiclersquos behavior based on egovehiclersquos future actions in a specific time Our strategy pre-dicts the future actions of human-driven vehicle Althoughthe velocity curves after 1 s do not correspond it does notaffect the performance of our methods The reason is that
12 Mathematical Problems in Engineering
0 1 2 3 4 5 6 7 80
1020
Time (s)
Velo
city
(ms
) Velocity profile
0 1 2 3 4 5 6 7 8024
Time (s)
Inte
ntio
n Longitudinal intention
0 1 2 3 4 5 6 7 8024
Time (s)
Inte
ntio
n Lateral intention
(a)
0 1 2 3 4 5 6 7 8 90
1020
Time (s)
Velo
city
(ms
) Velocity profile
0 1 2 3 4 5 6 7 8 9024
Time (s)
Inte
ntio
n
Longitudinal intention
0 1 2 3 4 5 6 7 8 9024
Time (s)
Inte
ntio
n
Lateral intention
(b)
0 1 2 3 4 5 6 7 8minus60
minus40
minus20
0
20
40
60
80
Time (s)
Distance to collision region
Dist
ance
(m)
(c)
0 1 2 3 4 5 6 7 8 9minus10
0
10
20
30
40
50
60
70
80
Time (s)
Dist
ance
(m)
Distance to collision region
(d)
0 1 2 3 4 5 6 7 87
8
9
10
11
12
13
14
15
Time (s)
Velo
city
(ms
)
(e)
Time (s)0 1 2 3 4 5 6 7 8
0
2
4
6
8
10
12
14
Velo
city
(ms
)
(f)
Figure 10 Case test 1 In this case human-driven vehicle passes through intersection after autonomous vehicle (a) (c) and (e) are theperformance of our method while (b) (d) and (f) are from the strategy without considering the driving intention (a) and (b) are the velocityprofiles and the corresponding driving intention For longitudinal intention label 1 means yielding and label 2 means not yielding In lateralintention 1 means turning left 2 means turning right 3 means going straight and 4 means stop The intentions in (b) are not used in thatmethod but for detailed analysis (c) and (d) are the distance to collision area for autonomous vehicle and human-driven vehicle respectively(e) and (f) are the prediction and true motions of human-driven vehicles in time 15 s with a prediction length of 8 s The red curves in thesesubfigures are from autonomous vehicle while blue lines are from human-driven vehicle The green lines in (e) and (f) are the predictionvelocity curves of human-driven vehicle
Mathematical Problems in Engineering 13
T = 4 s
T = 1 s
T = 2 s
T = 3 s
T = 5 s
T = 0 s
T = 0 1 2 5 s
(a)
T = 4 s
T = 1 s
T = 2 s
T = 3 s
T = 5 s
T = 0 s
T = 0 1 2 5 s
(b)
Figure 11 The visualized passing sequence for the case of human-driven vehicle first getting through (a) is the result of our approach and(b) is the reactive-based approach
we use a deterministic model in the prediction process andthe prediction value is inside two boundaries to ensure safetyBesides the whole actions of autonomous vehicle in thisprocess could also help human-driven vehicle to understandnot-yielding intention of autonomous vehicles In this casecooperative driving behaviors are performed by both vehi-cles
However if the intention is not considered in this processwe can find the results in Figures 10(b) 10(d) and 10(f)After 2 s in Figure 10(b) while the human-driven vehiclegives a yielding intention autonomous vehicle could notunderstand and they find a potential collision based on theconstant velocity assumptions Then it decreases the speedbut the human-driven vehicle also slows down The puzzledbehavior leads both vehicles to slow down near intersectionFinally human-driven vehicle stops at the stop line and thenautonomous vehicle could pass the intersection In this strat-egy the human-driven vehiclersquos future motion is assumed tobe constant (Figure 10(f))Without understanding of human-driven vehiclersquos intentions this strategy can increase conges-tion problem
Another experiment is that human-driven vehicle tries toget through the intersection first The results are shown inFigures 11 and 12This case is quite typical because many traf-fic accidents in real world are happening in this situation Indetail if one vehicle tries to cross an intersection while violat-ing the law another vehicle will be in great danger if it doesnot understand its behavior From the visualized perform-ance in Figure 11 our method is a little more safe than otherapproaches as there is nearly collision situation in Figure11(b) In detail we can see from Figure 12(a) that our strategycould perform deceleration actions after we understand thenot-yielding intention in 08 sHowever without understand-ing human-driven vehiclersquos motion intention the responsetime has a 1-second delay which may be quite dangerous
In addition it is shown that good performance is in thepredictions of human-driven vehiclersquos future motion in ourmethods (Figure 12(e))
The results of these two cases demonstrate that our algo-rithm could deal with typical scenarios and have better per-formance than traditional reactive controller Autonomousvehicle could be driven more safely fast and comfortablythrough our strategy
7 Conclusion and Future Work
In this paper we proposed an autonomous driving decision-making algorithm considering human-driven vehiclersquos uncer-tain intentions in an uncontrolled intersection The lat-eral and longitudinal intentions are recognized by a con-tinuous HMM Based on HMM and POMDP we modelgeneral decision-making process and then use an approxi-mate approach to solve this complex problem Finally weuse PreScan software and a driving simulator to emulatesocial interaction process The experiment results show thatautonomous vehicles with our approach can pass throughuncontrolled intersections more safely and efficiently thanusing the strategy without considering human-driven vehi-clesrsquo driving intentions
In the near future we aim to implement our approach intoa real autonomous vehicle and perform real world experi-ments In additionmore precious intention recognition algo-rithm aims to be figured out Somemethods like probabilisticgraphic model can be used to get a distribution of each inten-tion Finally designing online POMDP planning algorithmsis also valuable
Competing Interests
The authors declare that there are no competing interestsregarding the publication of this paper
14 Mathematical Problems in Engineering
0 05 1 15 2 25 3 35 4 45 510
20
30
Time (s)
Velo
city
(ms
) Velocity profile
0 05 1 15 2 25 3 35 4 45 50
2
4
Time (s)
Inte
ntio
n
Longitudinal intention
0 05 1 15 2 25 3 35 4 45 5024
Inte
ntio
n Lateral intention
Time (s)
(a)
0 05 1 15 2 25 3 35 4 45 5102030
Time (s)
Velo
city
(ms
) Velocity profile
0 05 1 15 2 25 3 35 4 45 5024
Time (s)
Inte
ntio
n
Longitudinal intention
0 05 1 15 2 25 3 35 4 45 5024
Time (s)
Inte
ntio
n
Lateral intention
(b)
0 05 1 15 2 25 3 35 4 45 5
0
20
40
60
80
Dist
ance
(m)
Distance to collision region
Time (s)
minus40
minus20
(c)
0 05 1 15 2 25 3 35 4 45 5
0
20
40
60
80D
istan
ce (m
)
Distance to collision region
Time (s)
minus40
minus20
(d)
0 1 2 3 4 5 6 7 816
17
18
19
20
21
22
23
24
25
Velo
city
(ms
)
Time (s)
(e)
0 1 2 3 4 5 6 7 817
18
19
20
21
22
23
24
25
Time (s)
Velo
city
(ms
)
(f)
Figure 12 Case test 2 In this case human-driven vehicle passes through intersection before autonomous vehicle through different strategiesThe definition of each subfigure is the same as in Figure 10
Mathematical Problems in Engineering 15
Acknowledgments
This study is supported by the National Natural ScienceFoundation of China (no 91420203)
References
[1] C Urmson J Anhalt D Bagnell et al ldquoAutonomous driving inurban environments boss and the urban challengerdquo Journal ofField Robotics vol 25 no 8 pp 425ndash466 2008
[2] J Markoff ldquoGoogle cars drive themselves in trafficrdquo New YorkTimes vol 9 2010
[3] L R Rabiner and B-H Juang ldquoAn introduction to hiddenMarkov modelsrdquo IEEE ASSP Magazine vol 3 no 1 pp 4ndash161986
[4] M Buehler K Iagnemma and S Singh The DARPA UrbanChallenge Autonomous Vehicles in City Traffic vol 56 Springer2009
[5] S Kammel J Ziegler B Pitzer et al ldquoTeam AnnieWAYrsquos auto-nomous system for the 2007 DARPAUrban Challengerdquo Journalof Field Robotics vol 25 no 9 pp 615ndash639 2008
[6] C R Baker and J M Dolan ldquoTraffic interaction in the urbanchallenge putting boss on its best behaviorrdquo in Proceedingsof the IEEERSJ International Conference on Intelligent Robotsand Systems (IROS rsquo08) pp 1752ndash1758 IEEE Nice FranceSeptember 2008
[7] M Montemerlo J Becker S Shat et al ldquoJunior the Stanfordentry in the urban challengerdquo Journal of Field Robotics vol 25no 9 pp 569ndash597 2008
[8] L Fletcher S Teller E Olson et al ldquoTheMIT-Cornell collisionand why it happenedrdquo Journal of Field Robotics vol 25 no 10pp 775ndash807 2008
[9] M Bahram A Wolf M Aeberhard and D Wollherr ldquoAprediction-based reactive driving strategy for highly automateddriving function on freewaysrdquo in Proceedings of the 25th IEEEIntelligent Vehicles Symposium pp 400ndash406 IEEE DearbornMich USA June 2014
[10] J Wei J M Dolan and B Litkouhi ldquoA prediction- and costfunction-based algorithm for robust autonomous freeway driv-ingrdquo in Proceedings of the IEEE Intelligent Vehicles Symposium(IV rsquo10) pp 512ndash517 San Diego Calif USA June 2010
[11] J Wei J M Dolan and B Litkouhi ldquoAutonomous vehiclesocial behavior for highway entrance ramp managementrdquo inProceedings of the IEEE Intelligent Vehicles Symposium (IV rsquo13)pp 201ndash207 IEEE Gold Coast Australia June 2013
[12] H Bai S Cai N Ye D Hsu and W S Lee ldquoIntention-awareonline POMDP planning for autonomous driving in a crowdrdquoin Proceedings of the IEEE International Conference on Roboticsand Automation (ICRA rsquo15) pp 454ndash460 IEEE Seattle WashUSA May 2015
[13] A Somani N Ye D Hsu and W S Lee ldquoDespot onlinepomdp planning with regularizationrdquo in Advances in NeuralInformation Processing Systems pp 1772ndash1780 2013
[14] S Brechtel T Gindele and R Dillmann ldquoProbabilistic deci-sion-making under uncertainty for autonomous driving usingcontinuous POMDPsrdquo in Proceedings of the 17th IEEE Interna-tional Conference on Intelligent Transportation Systems (ITSCrsquo14) pp 392ndash399 Qingdao China October 2014
[15] C H Papadimitriou and J N Tsitsiklis ldquoThe complexity ofMarkov decision processesrdquo Mathematics of Operations Re-search vol 12 no 3 pp 441ndash450 1987
[16] O Madani S Hanks and A Condon ldquoOn the undecidabilityof probabilistic planning and related stochastic optimizationproblemsrdquoArtificial Intelligence vol 147 no 1-2 pp 5ndash34 2003
[17] S Ulbrich and M Maurer ldquoProbabilistic online POMDP deci-sion making for lane changes in fully automated drivingrdquo inProceedings of the 16th International IEEE Conference on Intelli-gent Transportation Systems (ITSC rsquo13) pp 2063ndash2067 TheHague The Netherlands October 2013
[18] A G Cunningham E Galceran R M Eustice and E OlsonldquoMPDM multipolicy decision-making in dynamic uncertainenvironments for autonomous drivingrdquo in Proceedings of theIEEE International Conference on Robotics and Automation(ICRA rsquo15) pp 1670ndash1677 Seattle Wash USA May 2015
[19] D A Reynolds and R C Rose ldquoRobust text-independentspeaker identification using Gaussian mixture speaker modelsrdquoIEEE Transactions on Speech and Audio Processing vol 3 no 1pp 72ndash83 1995
[20] L E Baum and T Petrie ldquoStatistical inference for probabilisticfunctions of finite state Markov chainsrdquo The Annals of Mathe-matical Statistics vol 37 pp 1554ndash1563 1966
[21] M Tideman and M Van Noort ldquoA simulation tool suite fordeveloping connected vehicle systemsrdquo in Proceedings of theIEEE Intelligent Vehicles Symposium (IEEE IV rsquo13) pp 713ndash718Queensland Australia June 2013
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
12 Mathematical Problems in Engineering
0 1 2 3 4 5 6 7 80
1020
Time (s)
Velo
city
(ms
) Velocity profile
0 1 2 3 4 5 6 7 8024
Time (s)
Inte
ntio
n Longitudinal intention
0 1 2 3 4 5 6 7 8024
Time (s)
Inte
ntio
n Lateral intention
(a)
0 1 2 3 4 5 6 7 8 90
1020
Time (s)
Velo
city
(ms
) Velocity profile
0 1 2 3 4 5 6 7 8 9024
Time (s)
Inte
ntio
n
Longitudinal intention
0 1 2 3 4 5 6 7 8 9024
Time (s)
Inte
ntio
n
Lateral intention
(b)
0 1 2 3 4 5 6 7 8minus60
minus40
minus20
0
20
40
60
80
Time (s)
Distance to collision region
Dist
ance
(m)
(c)
0 1 2 3 4 5 6 7 8 9minus10
0
10
20
30
40
50
60
70
80
Time (s)
Dist
ance
(m)
Distance to collision region
(d)
0 1 2 3 4 5 6 7 87
8
9
10
11
12
13
14
15
Time (s)
Velo
city
(ms
)
(e)
Time (s)0 1 2 3 4 5 6 7 8
0
2
4
6
8
10
12
14
Velo
city
(ms
)
(f)
Figure 10 Case test 1 In this case human-driven vehicle passes through intersection after autonomous vehicle (a) (c) and (e) are theperformance of our method while (b) (d) and (f) are from the strategy without considering the driving intention (a) and (b) are the velocityprofiles and the corresponding driving intention For longitudinal intention label 1 means yielding and label 2 means not yielding In lateralintention 1 means turning left 2 means turning right 3 means going straight and 4 means stop The intentions in (b) are not used in thatmethod but for detailed analysis (c) and (d) are the distance to collision area for autonomous vehicle and human-driven vehicle respectively(e) and (f) are the prediction and true motions of human-driven vehicles in time 15 s with a prediction length of 8 s The red curves in thesesubfigures are from autonomous vehicle while blue lines are from human-driven vehicle The green lines in (e) and (f) are the predictionvelocity curves of human-driven vehicle
Mathematical Problems in Engineering 13
T = 4 s
T = 1 s
T = 2 s
T = 3 s
T = 5 s
T = 0 s
T = 0 1 2 5 s
(a)
T = 4 s
T = 1 s
T = 2 s
T = 3 s
T = 5 s
T = 0 s
T = 0 1 2 5 s
(b)
Figure 11 The visualized passing sequence for the case of human-driven vehicle first getting through (a) is the result of our approach and(b) is the reactive-based approach
we use a deterministic model in the prediction process andthe prediction value is inside two boundaries to ensure safetyBesides the whole actions of autonomous vehicle in thisprocess could also help human-driven vehicle to understandnot-yielding intention of autonomous vehicles In this casecooperative driving behaviors are performed by both vehi-cles
However if the intention is not considered in this processwe can find the results in Figures 10(b) 10(d) and 10(f)After 2 s in Figure 10(b) while the human-driven vehiclegives a yielding intention autonomous vehicle could notunderstand and they find a potential collision based on theconstant velocity assumptions Then it decreases the speedbut the human-driven vehicle also slows down The puzzledbehavior leads both vehicles to slow down near intersectionFinally human-driven vehicle stops at the stop line and thenautonomous vehicle could pass the intersection In this strat-egy the human-driven vehiclersquos future motion is assumed tobe constant (Figure 10(f))Without understanding of human-driven vehiclersquos intentions this strategy can increase conges-tion problem
Another experiment is that human-driven vehicle tries toget through the intersection first The results are shown inFigures 11 and 12This case is quite typical because many traf-fic accidents in real world are happening in this situation Indetail if one vehicle tries to cross an intersection while violat-ing the law another vehicle will be in great danger if it doesnot understand its behavior From the visualized perform-ance in Figure 11 our method is a little more safe than otherapproaches as there is nearly collision situation in Figure11(b) In detail we can see from Figure 12(a) that our strategycould perform deceleration actions after we understand thenot-yielding intention in 08 sHowever without understand-ing human-driven vehiclersquos motion intention the responsetime has a 1-second delay which may be quite dangerous
In addition it is shown that good performance is in thepredictions of human-driven vehiclersquos future motion in ourmethods (Figure 12(e))
The results of these two cases demonstrate that our algo-rithm could deal with typical scenarios and have better per-formance than traditional reactive controller Autonomousvehicle could be driven more safely fast and comfortablythrough our strategy
7 Conclusion and Future Work
In this paper we proposed an autonomous driving decision-making algorithm considering human-driven vehiclersquos uncer-tain intentions in an uncontrolled intersection The lat-eral and longitudinal intentions are recognized by a con-tinuous HMM Based on HMM and POMDP we modelgeneral decision-making process and then use an approxi-mate approach to solve this complex problem Finally weuse PreScan software and a driving simulator to emulatesocial interaction process The experiment results show thatautonomous vehicles with our approach can pass throughuncontrolled intersections more safely and efficiently thanusing the strategy without considering human-driven vehi-clesrsquo driving intentions
In the near future we aim to implement our approach intoa real autonomous vehicle and perform real world experi-ments In additionmore precious intention recognition algo-rithm aims to be figured out Somemethods like probabilisticgraphic model can be used to get a distribution of each inten-tion Finally designing online POMDP planning algorithmsis also valuable
Competing Interests
The authors declare that there are no competing interestsregarding the publication of this paper
14 Mathematical Problems in Engineering
0 05 1 15 2 25 3 35 4 45 510
20
30
Time (s)
Velo
city
(ms
) Velocity profile
0 05 1 15 2 25 3 35 4 45 50
2
4
Time (s)
Inte
ntio
n
Longitudinal intention
0 05 1 15 2 25 3 35 4 45 5024
Inte
ntio
n Lateral intention
Time (s)
(a)
0 05 1 15 2 25 3 35 4 45 5102030
Time (s)
Velo
city
(ms
) Velocity profile
0 05 1 15 2 25 3 35 4 45 5024
Time (s)
Inte
ntio
n
Longitudinal intention
0 05 1 15 2 25 3 35 4 45 5024
Time (s)
Inte
ntio
n
Lateral intention
(b)
0 05 1 15 2 25 3 35 4 45 5
0
20
40
60
80
Dist
ance
(m)
Distance to collision region
Time (s)
minus40
minus20
(c)
0 05 1 15 2 25 3 35 4 45 5
0
20
40
60
80D
istan
ce (m
)
Distance to collision region
Time (s)
minus40
minus20
(d)
0 1 2 3 4 5 6 7 816
17
18
19
20
21
22
23
24
25
Velo
city
(ms
)
Time (s)
(e)
0 1 2 3 4 5 6 7 817
18
19
20
21
22
23
24
25
Time (s)
Velo
city
(ms
)
(f)
Figure 12 Case test 2 In this case human-driven vehicle passes through intersection before autonomous vehicle through different strategiesThe definition of each subfigure is the same as in Figure 10
Mathematical Problems in Engineering 15
Acknowledgments
This study is supported by the National Natural ScienceFoundation of China (no 91420203)
References
[1] C Urmson J Anhalt D Bagnell et al ldquoAutonomous driving inurban environments boss and the urban challengerdquo Journal ofField Robotics vol 25 no 8 pp 425ndash466 2008
[2] J Markoff ldquoGoogle cars drive themselves in trafficrdquo New YorkTimes vol 9 2010
[3] L R Rabiner and B-H Juang ldquoAn introduction to hiddenMarkov modelsrdquo IEEE ASSP Magazine vol 3 no 1 pp 4ndash161986
[4] M Buehler K Iagnemma and S Singh The DARPA UrbanChallenge Autonomous Vehicles in City Traffic vol 56 Springer2009
[5] S Kammel J Ziegler B Pitzer et al ldquoTeam AnnieWAYrsquos auto-nomous system for the 2007 DARPAUrban Challengerdquo Journalof Field Robotics vol 25 no 9 pp 615ndash639 2008
[6] C R Baker and J M Dolan ldquoTraffic interaction in the urbanchallenge putting boss on its best behaviorrdquo in Proceedingsof the IEEERSJ International Conference on Intelligent Robotsand Systems (IROS rsquo08) pp 1752ndash1758 IEEE Nice FranceSeptember 2008
[7] M Montemerlo J Becker S Shat et al ldquoJunior the Stanfordentry in the urban challengerdquo Journal of Field Robotics vol 25no 9 pp 569ndash597 2008
[8] L Fletcher S Teller E Olson et al ldquoTheMIT-Cornell collisionand why it happenedrdquo Journal of Field Robotics vol 25 no 10pp 775ndash807 2008
[9] M Bahram A Wolf M Aeberhard and D Wollherr ldquoAprediction-based reactive driving strategy for highly automateddriving function on freewaysrdquo in Proceedings of the 25th IEEEIntelligent Vehicles Symposium pp 400ndash406 IEEE DearbornMich USA June 2014
[10] J Wei J M Dolan and B Litkouhi ldquoA prediction- and costfunction-based algorithm for robust autonomous freeway driv-ingrdquo in Proceedings of the IEEE Intelligent Vehicles Symposium(IV rsquo10) pp 512ndash517 San Diego Calif USA June 2010
[11] J Wei J M Dolan and B Litkouhi ldquoAutonomous vehiclesocial behavior for highway entrance ramp managementrdquo inProceedings of the IEEE Intelligent Vehicles Symposium (IV rsquo13)pp 201ndash207 IEEE Gold Coast Australia June 2013
[12] H Bai S Cai N Ye D Hsu and W S Lee ldquoIntention-awareonline POMDP planning for autonomous driving in a crowdrdquoin Proceedings of the IEEE International Conference on Roboticsand Automation (ICRA rsquo15) pp 454ndash460 IEEE Seattle WashUSA May 2015
[13] A Somani N Ye D Hsu and W S Lee ldquoDespot onlinepomdp planning with regularizationrdquo in Advances in NeuralInformation Processing Systems pp 1772ndash1780 2013
[14] S Brechtel T Gindele and R Dillmann ldquoProbabilistic deci-sion-making under uncertainty for autonomous driving usingcontinuous POMDPsrdquo in Proceedings of the 17th IEEE Interna-tional Conference on Intelligent Transportation Systems (ITSCrsquo14) pp 392ndash399 Qingdao China October 2014
[15] C H Papadimitriou and J N Tsitsiklis ldquoThe complexity ofMarkov decision processesrdquo Mathematics of Operations Re-search vol 12 no 3 pp 441ndash450 1987
[16] O Madani S Hanks and A Condon ldquoOn the undecidabilityof probabilistic planning and related stochastic optimizationproblemsrdquoArtificial Intelligence vol 147 no 1-2 pp 5ndash34 2003
[17] S Ulbrich and M Maurer ldquoProbabilistic online POMDP deci-sion making for lane changes in fully automated drivingrdquo inProceedings of the 16th International IEEE Conference on Intelli-gent Transportation Systems (ITSC rsquo13) pp 2063ndash2067 TheHague The Netherlands October 2013
[18] A G Cunningham E Galceran R M Eustice and E OlsonldquoMPDM multipolicy decision-making in dynamic uncertainenvironments for autonomous drivingrdquo in Proceedings of theIEEE International Conference on Robotics and Automation(ICRA rsquo15) pp 1670ndash1677 Seattle Wash USA May 2015
[19] D A Reynolds and R C Rose ldquoRobust text-independentspeaker identification using Gaussian mixture speaker modelsrdquoIEEE Transactions on Speech and Audio Processing vol 3 no 1pp 72ndash83 1995
[20] L E Baum and T Petrie ldquoStatistical inference for probabilisticfunctions of finite state Markov chainsrdquo The Annals of Mathe-matical Statistics vol 37 pp 1554ndash1563 1966
[21] M Tideman and M Van Noort ldquoA simulation tool suite fordeveloping connected vehicle systemsrdquo in Proceedings of theIEEE Intelligent Vehicles Symposium (IEEE IV rsquo13) pp 713ndash718Queensland Australia June 2013
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Mathematical Problems in Engineering 13
T = 4 s
T = 1 s
T = 2 s
T = 3 s
T = 5 s
T = 0 s
T = 0 1 2 5 s
(a)
T = 4 s
T = 1 s
T = 2 s
T = 3 s
T = 5 s
T = 0 s
T = 0 1 2 5 s
(b)
Figure 11 The visualized passing sequence for the case of human-driven vehicle first getting through (a) is the result of our approach and(b) is the reactive-based approach
we use a deterministic model in the prediction process andthe prediction value is inside two boundaries to ensure safetyBesides the whole actions of autonomous vehicle in thisprocess could also help human-driven vehicle to understandnot-yielding intention of autonomous vehicles In this casecooperative driving behaviors are performed by both vehi-cles
However if the intention is not considered in this processwe can find the results in Figures 10(b) 10(d) and 10(f)After 2 s in Figure 10(b) while the human-driven vehiclegives a yielding intention autonomous vehicle could notunderstand and they find a potential collision based on theconstant velocity assumptions Then it decreases the speedbut the human-driven vehicle also slows down The puzzledbehavior leads both vehicles to slow down near intersectionFinally human-driven vehicle stops at the stop line and thenautonomous vehicle could pass the intersection In this strat-egy the human-driven vehiclersquos future motion is assumed tobe constant (Figure 10(f))Without understanding of human-driven vehiclersquos intentions this strategy can increase conges-tion problem
Another experiment is that human-driven vehicle tries toget through the intersection first The results are shown inFigures 11 and 12This case is quite typical because many traf-fic accidents in real world are happening in this situation Indetail if one vehicle tries to cross an intersection while violat-ing the law another vehicle will be in great danger if it doesnot understand its behavior From the visualized perform-ance in Figure 11 our method is a little more safe than otherapproaches as there is nearly collision situation in Figure11(b) In detail we can see from Figure 12(a) that our strategycould perform deceleration actions after we understand thenot-yielding intention in 08 sHowever without understand-ing human-driven vehiclersquos motion intention the responsetime has a 1-second delay which may be quite dangerous
In addition it is shown that good performance is in thepredictions of human-driven vehiclersquos future motion in ourmethods (Figure 12(e))
The results of these two cases demonstrate that our algo-rithm could deal with typical scenarios and have better per-formance than traditional reactive controller Autonomousvehicle could be driven more safely fast and comfortablythrough our strategy
7 Conclusion and Future Work
In this paper we proposed an autonomous driving decision-making algorithm considering human-driven vehiclersquos uncer-tain intentions in an uncontrolled intersection The lat-eral and longitudinal intentions are recognized by a con-tinuous HMM Based on HMM and POMDP we modelgeneral decision-making process and then use an approxi-mate approach to solve this complex problem Finally weuse PreScan software and a driving simulator to emulatesocial interaction process The experiment results show thatautonomous vehicles with our approach can pass throughuncontrolled intersections more safely and efficiently thanusing the strategy without considering human-driven vehi-clesrsquo driving intentions
In the near future we aim to implement our approach intoa real autonomous vehicle and perform real world experi-ments In additionmore precious intention recognition algo-rithm aims to be figured out Somemethods like probabilisticgraphic model can be used to get a distribution of each inten-tion Finally designing online POMDP planning algorithmsis also valuable
Competing Interests
The authors declare that there are no competing interestsregarding the publication of this paper
14 Mathematical Problems in Engineering
0 05 1 15 2 25 3 35 4 45 510
20
30
Time (s)
Velo
city
(ms
) Velocity profile
0 05 1 15 2 25 3 35 4 45 50
2
4
Time (s)
Inte
ntio
n
Longitudinal intention
0 05 1 15 2 25 3 35 4 45 5024
Inte
ntio
n Lateral intention
Time (s)
(a)
0 05 1 15 2 25 3 35 4 45 5102030
Time (s)
Velo
city
(ms
) Velocity profile
0 05 1 15 2 25 3 35 4 45 5024
Time (s)
Inte
ntio
n
Longitudinal intention
0 05 1 15 2 25 3 35 4 45 5024
Time (s)
Inte
ntio
n
Lateral intention
(b)
0 05 1 15 2 25 3 35 4 45 5
0
20
40
60
80
Dist
ance
(m)
Distance to collision region
Time (s)
minus40
minus20
(c)
0 05 1 15 2 25 3 35 4 45 5
0
20
40
60
80D
istan
ce (m
)
Distance to collision region
Time (s)
minus40
minus20
(d)
0 1 2 3 4 5 6 7 816
17
18
19
20
21
22
23
24
25
Velo
city
(ms
)
Time (s)
(e)
0 1 2 3 4 5 6 7 817
18
19
20
21
22
23
24
25
Time (s)
Velo
city
(ms
)
(f)
Figure 12 Case test 2 In this case human-driven vehicle passes through intersection before autonomous vehicle through different strategiesThe definition of each subfigure is the same as in Figure 10
Mathematical Problems in Engineering 15
Acknowledgments
This study is supported by the National Natural ScienceFoundation of China (no 91420203)
References
[1] C Urmson J Anhalt D Bagnell et al ldquoAutonomous driving inurban environments boss and the urban challengerdquo Journal ofField Robotics vol 25 no 8 pp 425ndash466 2008
[2] J Markoff ldquoGoogle cars drive themselves in trafficrdquo New YorkTimes vol 9 2010
[3] L R Rabiner and B-H Juang ldquoAn introduction to hiddenMarkov modelsrdquo IEEE ASSP Magazine vol 3 no 1 pp 4ndash161986
[4] M Buehler K Iagnemma and S Singh The DARPA UrbanChallenge Autonomous Vehicles in City Traffic vol 56 Springer2009
[5] S Kammel J Ziegler B Pitzer et al ldquoTeam AnnieWAYrsquos auto-nomous system for the 2007 DARPAUrban Challengerdquo Journalof Field Robotics vol 25 no 9 pp 615ndash639 2008
[6] C R Baker and J M Dolan ldquoTraffic interaction in the urbanchallenge putting boss on its best behaviorrdquo in Proceedingsof the IEEERSJ International Conference on Intelligent Robotsand Systems (IROS rsquo08) pp 1752ndash1758 IEEE Nice FranceSeptember 2008
[7] M Montemerlo J Becker S Shat et al ldquoJunior the Stanfordentry in the urban challengerdquo Journal of Field Robotics vol 25no 9 pp 569ndash597 2008
[8] L Fletcher S Teller E Olson et al ldquoTheMIT-Cornell collisionand why it happenedrdquo Journal of Field Robotics vol 25 no 10pp 775ndash807 2008
[9] M Bahram A Wolf M Aeberhard and D Wollherr ldquoAprediction-based reactive driving strategy for highly automateddriving function on freewaysrdquo in Proceedings of the 25th IEEEIntelligent Vehicles Symposium pp 400ndash406 IEEE DearbornMich USA June 2014
[10] J Wei J M Dolan and B Litkouhi ldquoA prediction- and costfunction-based algorithm for robust autonomous freeway driv-ingrdquo in Proceedings of the IEEE Intelligent Vehicles Symposium(IV rsquo10) pp 512ndash517 San Diego Calif USA June 2010
[11] J Wei J M Dolan and B Litkouhi ldquoAutonomous vehiclesocial behavior for highway entrance ramp managementrdquo inProceedings of the IEEE Intelligent Vehicles Symposium (IV rsquo13)pp 201ndash207 IEEE Gold Coast Australia June 2013
[12] H Bai S Cai N Ye D Hsu and W S Lee ldquoIntention-awareonline POMDP planning for autonomous driving in a crowdrdquoin Proceedings of the IEEE International Conference on Roboticsand Automation (ICRA rsquo15) pp 454ndash460 IEEE Seattle WashUSA May 2015
[13] A Somani N Ye D Hsu and W S Lee ldquoDespot onlinepomdp planning with regularizationrdquo in Advances in NeuralInformation Processing Systems pp 1772ndash1780 2013
[14] S Brechtel T Gindele and R Dillmann ldquoProbabilistic deci-sion-making under uncertainty for autonomous driving usingcontinuous POMDPsrdquo in Proceedings of the 17th IEEE Interna-tional Conference on Intelligent Transportation Systems (ITSCrsquo14) pp 392ndash399 Qingdao China October 2014
[15] C H Papadimitriou and J N Tsitsiklis ldquoThe complexity ofMarkov decision processesrdquo Mathematics of Operations Re-search vol 12 no 3 pp 441ndash450 1987
[16] O Madani S Hanks and A Condon ldquoOn the undecidabilityof probabilistic planning and related stochastic optimizationproblemsrdquoArtificial Intelligence vol 147 no 1-2 pp 5ndash34 2003
[17] S Ulbrich and M Maurer ldquoProbabilistic online POMDP deci-sion making for lane changes in fully automated drivingrdquo inProceedings of the 16th International IEEE Conference on Intelli-gent Transportation Systems (ITSC rsquo13) pp 2063ndash2067 TheHague The Netherlands October 2013
[18] A G Cunningham E Galceran R M Eustice and E OlsonldquoMPDM multipolicy decision-making in dynamic uncertainenvironments for autonomous drivingrdquo in Proceedings of theIEEE International Conference on Robotics and Automation(ICRA rsquo15) pp 1670ndash1677 Seattle Wash USA May 2015
[19] D A Reynolds and R C Rose ldquoRobust text-independentspeaker identification using Gaussian mixture speaker modelsrdquoIEEE Transactions on Speech and Audio Processing vol 3 no 1pp 72ndash83 1995
[20] L E Baum and T Petrie ldquoStatistical inference for probabilisticfunctions of finite state Markov chainsrdquo The Annals of Mathe-matical Statistics vol 37 pp 1554ndash1563 1966
[21] M Tideman and M Van Noort ldquoA simulation tool suite fordeveloping connected vehicle systemsrdquo in Proceedings of theIEEE Intelligent Vehicles Symposium (IEEE IV rsquo13) pp 713ndash718Queensland Australia June 2013
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
14 Mathematical Problems in Engineering
0 05 1 15 2 25 3 35 4 45 510
20
30
Time (s)
Velo
city
(ms
) Velocity profile
0 05 1 15 2 25 3 35 4 45 50
2
4
Time (s)
Inte
ntio
n
Longitudinal intention
0 05 1 15 2 25 3 35 4 45 5024
Inte
ntio
n Lateral intention
Time (s)
(a)
0 05 1 15 2 25 3 35 4 45 5102030
Time (s)
Velo
city
(ms
) Velocity profile
0 05 1 15 2 25 3 35 4 45 5024
Time (s)
Inte
ntio
n
Longitudinal intention
0 05 1 15 2 25 3 35 4 45 5024
Time (s)
Inte
ntio
n
Lateral intention
(b)
0 05 1 15 2 25 3 35 4 45 5
0
20
40
60
80
Dist
ance
(m)
Distance to collision region
Time (s)
minus40
minus20
(c)
0 05 1 15 2 25 3 35 4 45 5
0
20
40
60
80D
istan
ce (m
)
Distance to collision region
Time (s)
minus40
minus20
(d)
0 1 2 3 4 5 6 7 816
17
18
19
20
21
22
23
24
25
Velo
city
(ms
)
Time (s)
(e)
0 1 2 3 4 5 6 7 817
18
19
20
21
22
23
24
25
Time (s)
Velo
city
(ms
)
(f)
Figure 12 Case test 2 In this case human-driven vehicle passes through intersection before autonomous vehicle through different strategiesThe definition of each subfigure is the same as in Figure 10
Mathematical Problems in Engineering 15
Acknowledgments
This study is supported by the National Natural ScienceFoundation of China (no 91420203)
References
[1] C Urmson J Anhalt D Bagnell et al ldquoAutonomous driving inurban environments boss and the urban challengerdquo Journal ofField Robotics vol 25 no 8 pp 425ndash466 2008
[2] J Markoff ldquoGoogle cars drive themselves in trafficrdquo New YorkTimes vol 9 2010
[3] L R Rabiner and B-H Juang ldquoAn introduction to hiddenMarkov modelsrdquo IEEE ASSP Magazine vol 3 no 1 pp 4ndash161986
[4] M Buehler K Iagnemma and S Singh The DARPA UrbanChallenge Autonomous Vehicles in City Traffic vol 56 Springer2009
[5] S Kammel J Ziegler B Pitzer et al ldquoTeam AnnieWAYrsquos auto-nomous system for the 2007 DARPAUrban Challengerdquo Journalof Field Robotics vol 25 no 9 pp 615ndash639 2008
[6] C R Baker and J M Dolan ldquoTraffic interaction in the urbanchallenge putting boss on its best behaviorrdquo in Proceedingsof the IEEERSJ International Conference on Intelligent Robotsand Systems (IROS rsquo08) pp 1752ndash1758 IEEE Nice FranceSeptember 2008
[7] M Montemerlo J Becker S Shat et al ldquoJunior the Stanfordentry in the urban challengerdquo Journal of Field Robotics vol 25no 9 pp 569ndash597 2008
[8] L Fletcher S Teller E Olson et al ldquoTheMIT-Cornell collisionand why it happenedrdquo Journal of Field Robotics vol 25 no 10pp 775ndash807 2008
[9] M Bahram A Wolf M Aeberhard and D Wollherr ldquoAprediction-based reactive driving strategy for highly automateddriving function on freewaysrdquo in Proceedings of the 25th IEEEIntelligent Vehicles Symposium pp 400ndash406 IEEE DearbornMich USA June 2014
[10] J Wei J M Dolan and B Litkouhi ldquoA prediction- and costfunction-based algorithm for robust autonomous freeway driv-ingrdquo in Proceedings of the IEEE Intelligent Vehicles Symposium(IV rsquo10) pp 512ndash517 San Diego Calif USA June 2010
[11] J Wei J M Dolan and B Litkouhi ldquoAutonomous vehiclesocial behavior for highway entrance ramp managementrdquo inProceedings of the IEEE Intelligent Vehicles Symposium (IV rsquo13)pp 201ndash207 IEEE Gold Coast Australia June 2013
[12] H Bai S Cai N Ye D Hsu and W S Lee ldquoIntention-awareonline POMDP planning for autonomous driving in a crowdrdquoin Proceedings of the IEEE International Conference on Roboticsand Automation (ICRA rsquo15) pp 454ndash460 IEEE Seattle WashUSA May 2015
[13] A Somani N Ye D Hsu and W S Lee ldquoDespot onlinepomdp planning with regularizationrdquo in Advances in NeuralInformation Processing Systems pp 1772ndash1780 2013
[14] S Brechtel T Gindele and R Dillmann ldquoProbabilistic deci-sion-making under uncertainty for autonomous driving usingcontinuous POMDPsrdquo in Proceedings of the 17th IEEE Interna-tional Conference on Intelligent Transportation Systems (ITSCrsquo14) pp 392ndash399 Qingdao China October 2014
[15] C H Papadimitriou and J N Tsitsiklis ldquoThe complexity ofMarkov decision processesrdquo Mathematics of Operations Re-search vol 12 no 3 pp 441ndash450 1987
[16] O Madani S Hanks and A Condon ldquoOn the undecidabilityof probabilistic planning and related stochastic optimizationproblemsrdquoArtificial Intelligence vol 147 no 1-2 pp 5ndash34 2003
[17] S Ulbrich and M Maurer ldquoProbabilistic online POMDP deci-sion making for lane changes in fully automated drivingrdquo inProceedings of the 16th International IEEE Conference on Intelli-gent Transportation Systems (ITSC rsquo13) pp 2063ndash2067 TheHague The Netherlands October 2013
[18] A G Cunningham E Galceran R M Eustice and E OlsonldquoMPDM multipolicy decision-making in dynamic uncertainenvironments for autonomous drivingrdquo in Proceedings of theIEEE International Conference on Robotics and Automation(ICRA rsquo15) pp 1670ndash1677 Seattle Wash USA May 2015
[19] D A Reynolds and R C Rose ldquoRobust text-independentspeaker identification using Gaussian mixture speaker modelsrdquoIEEE Transactions on Speech and Audio Processing vol 3 no 1pp 72ndash83 1995
[20] L E Baum and T Petrie ldquoStatistical inference for probabilisticfunctions of finite state Markov chainsrdquo The Annals of Mathe-matical Statistics vol 37 pp 1554ndash1563 1966
[21] M Tideman and M Van Noort ldquoA simulation tool suite fordeveloping connected vehicle systemsrdquo in Proceedings of theIEEE Intelligent Vehicles Symposium (IEEE IV rsquo13) pp 713ndash718Queensland Australia June 2013
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Mathematical Problems in Engineering 15
Acknowledgments
This study is supported by the National Natural ScienceFoundation of China (no 91420203)
References
[1] C Urmson J Anhalt D Bagnell et al ldquoAutonomous driving inurban environments boss and the urban challengerdquo Journal ofField Robotics vol 25 no 8 pp 425ndash466 2008
[2] J Markoff ldquoGoogle cars drive themselves in trafficrdquo New YorkTimes vol 9 2010
[3] L R Rabiner and B-H Juang ldquoAn introduction to hiddenMarkov modelsrdquo IEEE ASSP Magazine vol 3 no 1 pp 4ndash161986
[4] M Buehler K Iagnemma and S Singh The DARPA UrbanChallenge Autonomous Vehicles in City Traffic vol 56 Springer2009
[5] S Kammel J Ziegler B Pitzer et al ldquoTeam AnnieWAYrsquos auto-nomous system for the 2007 DARPAUrban Challengerdquo Journalof Field Robotics vol 25 no 9 pp 615ndash639 2008
[6] C R Baker and J M Dolan ldquoTraffic interaction in the urbanchallenge putting boss on its best behaviorrdquo in Proceedingsof the IEEERSJ International Conference on Intelligent Robotsand Systems (IROS rsquo08) pp 1752ndash1758 IEEE Nice FranceSeptember 2008
[7] M Montemerlo J Becker S Shat et al ldquoJunior the Stanfordentry in the urban challengerdquo Journal of Field Robotics vol 25no 9 pp 569ndash597 2008
[8] L Fletcher S Teller E Olson et al ldquoTheMIT-Cornell collisionand why it happenedrdquo Journal of Field Robotics vol 25 no 10pp 775ndash807 2008
[9] M Bahram A Wolf M Aeberhard and D Wollherr ldquoAprediction-based reactive driving strategy for highly automateddriving function on freewaysrdquo in Proceedings of the 25th IEEEIntelligent Vehicles Symposium pp 400ndash406 IEEE DearbornMich USA June 2014
[10] J Wei J M Dolan and B Litkouhi ldquoA prediction- and costfunction-based algorithm for robust autonomous freeway driv-ingrdquo in Proceedings of the IEEE Intelligent Vehicles Symposium(IV rsquo10) pp 512ndash517 San Diego Calif USA June 2010
[11] J Wei J M Dolan and B Litkouhi ldquoAutonomous vehiclesocial behavior for highway entrance ramp managementrdquo inProceedings of the IEEE Intelligent Vehicles Symposium (IV rsquo13)pp 201ndash207 IEEE Gold Coast Australia June 2013
[12] H Bai S Cai N Ye D Hsu and W S Lee ldquoIntention-awareonline POMDP planning for autonomous driving in a crowdrdquoin Proceedings of the IEEE International Conference on Roboticsand Automation (ICRA rsquo15) pp 454ndash460 IEEE Seattle WashUSA May 2015
[13] A Somani N Ye D Hsu and W S Lee ldquoDespot onlinepomdp planning with regularizationrdquo in Advances in NeuralInformation Processing Systems pp 1772ndash1780 2013
[14] S Brechtel T Gindele and R Dillmann ldquoProbabilistic deci-sion-making under uncertainty for autonomous driving usingcontinuous POMDPsrdquo in Proceedings of the 17th IEEE Interna-tional Conference on Intelligent Transportation Systems (ITSCrsquo14) pp 392ndash399 Qingdao China October 2014
[15] C H Papadimitriou and J N Tsitsiklis ldquoThe complexity ofMarkov decision processesrdquo Mathematics of Operations Re-search vol 12 no 3 pp 441ndash450 1987
[16] O Madani S Hanks and A Condon ldquoOn the undecidabilityof probabilistic planning and related stochastic optimizationproblemsrdquoArtificial Intelligence vol 147 no 1-2 pp 5ndash34 2003
[17] S Ulbrich and M Maurer ldquoProbabilistic online POMDP deci-sion making for lane changes in fully automated drivingrdquo inProceedings of the 16th International IEEE Conference on Intelli-gent Transportation Systems (ITSC rsquo13) pp 2063ndash2067 TheHague The Netherlands October 2013
[18] A G Cunningham E Galceran R M Eustice and E OlsonldquoMPDM multipolicy decision-making in dynamic uncertainenvironments for autonomous drivingrdquo in Proceedings of theIEEE International Conference on Robotics and Automation(ICRA rsquo15) pp 1670ndash1677 Seattle Wash USA May 2015
[19] D A Reynolds and R C Rose ldquoRobust text-independentspeaker identification using Gaussian mixture speaker modelsrdquoIEEE Transactions on Speech and Audio Processing vol 3 no 1pp 72ndash83 1995
[20] L E Baum and T Petrie ldquoStatistical inference for probabilisticfunctions of finite state Markov chainsrdquo The Annals of Mathe-matical Statistics vol 37 pp 1554ndash1563 1966
[21] M Tideman and M Van Noort ldquoA simulation tool suite fordeveloping connected vehicle systemsrdquo in Proceedings of theIEEE Intelligent Vehicles Symposium (IEEE IV rsquo13) pp 713ndash718Queensland Australia June 2013
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Submit your manuscripts athttpwwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Discrete MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of