1

PERFORMANCE MEASUREMENT AND MONITORING IN TSM&O - CURRENT PRACTICE AND FUTURE

DEVELOPMENTS

Speakers:

Tony Kratofil – Michigan DOT

Aleksandar Stevanovic – FAU

Michael Pack – UMD CATT Lab

Sponsored by TRB NCHRP and AASHTO STSMO Performance Measures Working Group

NOCoE Webinar, April 22 2016

2

• NCHRP 20‐07/Task 366 ‐ Accessing Information aboutTransportation Systems Management and OperationsPerformance Measurement – Aleksandar Stevanovic, FAULATOM

• Answering Questions Both Known & Unknown: TSM&OPerformance Measurement & Monitoring with Big Data ‐Michael Pack, UMD CATT Lab

• Two methods to estimate traffic performance in urbannetworks – Aleksandar Stevanovic, FAU LATOM

• Estimating Signal Performance based on Link Travel Times• Estimating Network Congestion based on Google Traffic

Maps

Webinar Agenda ‐ Presentations

3

• Establish a framework for organizing information about research andpractices for TSMO performance measurement and monitoring to assessthe impacts of TSMO strategies

Less well developed Most difficult to measure

• Facilitate access to TSMO performance measurement and monitoringinformation by developing a guide to the most relevant recent literature

• Identify and describe the problems, opportunities, and consequences forpractitioner adoption of specific measures and setting targets for TSMOperformance management, with particular attention to federal rulemakingunder MAP‐21 legislation

NCHRP 20‐07/Task 366 Research Objectives

4

• Develop a methodology to categorize existing performance measurementstudies by creating a comprehensive list of potential categories

Start from existing categories, i.e., “Elements of Success” Review and edit existing categories as necessary, and suggest new

categories and their subcategories for specificity

• In collaboration with NCHRP, SHRP2, and TRB, identify the best way tointegrate research outputs into either :

Existing TSMO web platform (www.transportationops.org) One or more other existing websites serving the intended audience A brand new, custom‐designed website

• Discuss an analysis of the various problems, opportunities, andconsequences of the adoption of each measure

Reach out to leading performance measures implementers toevaluate the impacts of the methodologies and metrics

Research Approach

5

Information Organization Framework and Literature Search

• Develop Information Organization Framework

• Develop preliminary categories to classify/filter various performancemeasurement‐related studies

• Propose how to integrate such categories into existing ‘tsmoinfo’ webportal or into a new web site

• Conduct Literature Search

• Identify ongoing and past research efforts relevant to the project

• Create a list of studies and the categories (elements of success)

6

Former tsmoinfo.org Structure

7

Current transportationops.org Structure

8

Considered TSM&O Strategies in this Study TSM&O Strategies

1‐ Access Management

2‐ Active Parking Management

3‐ Active Traffic Management

4‐ Adaptive Traffic Signal Technology5‐ Bicycle and Pedestrian Management

6‐ Corridor and Arterial Traffic Management7‐ Freeway Management

8‐ High Occupancy Vehicle (HOV) Lanes

9‐ Pricing/Toll Roads

10‐ Ramp Metering

11‐ Geometric Design

12‐ Traffic Signal Program Management

13‐ Signal Timing

14‐ Transit Operation

15‐ Transit Signal Priority

16‐ Travel Demand Management

17‐ Freight

18‐ Road Weather Management

9

TSM&O Performance Measures Categories  

Performance Measures

Operations Safety Environment Economics

10

Descriptions of Topic Indexing MethodsMethod Alternative Name Working Definition

Text categorization Text classificationVery specific general categories, like Planning orOperations, are assigned from usually a small vocabulary inthe context of performance measures.

Term assignment Subject indexingMain topics are expressed using terms from a largevocabulary, e.g. a thesaurus. The list of categories createdin this Task 2, can serve as our thesaurus

Key‐phrase extraction Keyword extraction, Key term extractionMain topics are expressed using the most prominent wordsand phrases in a document

Terminology extraction Back‐of‐the‐book (BOB) indexingAll domain relevant words and phrases are extracted from adocument

Full‐text indexing Full indexing, Free‐text indexingAll words and phrases, sometimes excluding stop‐words ,are extracted from a document

Key‐phrase indexing Full indexing, Free‐text indexingAll words and phrases, sometimes excluding stop‐words,are extracted from a document

Key‐phrase indexing Key‐phrase assignmentA general term, which refers to both term assignment andkey‐phrase extraction

TaggingCollaborative tagging, Social tagging, Auto‐tagging, Automatic tagging

The user defines as many topics as desired. Any word orphrase can serve as a tag. Applies mainly to collaborativewebsites

11

Framework for Retrieval of Information

12

Finalized Organizational Framework

13

Example of Performance Measures Categorization

14

Example of Literature Overview 

15

Problems and Opportunities for Common TSMO‐Specific Performance Measures 

• A comprehensive list of performance measures forvarious subareas of TSMO

• A set of matrices which will be used to categorizethese performance measures according to variousaspects of deployment

16

Example of TSM&O Performance MeasuresTSMO Strategy Sub‐category Performance Measure

Access Management

Operation

Travel TimeSpeedDelayQueue Length

Safety

Crash Rate (Crash per million VMT)

Number of CrashesNumber of Injury and FatalitiesNumber of Property Damages (Number of property damages only (PDO))

Time‐to‐Collision (Time takes for a vehicle to collide into another if they continue at the same speed without trying to avoid each other)

EconomicBusiness TurnoverCommercial Land ValuesProperty Value

17

Use of Various Performance MeasuresPerformance Measures Organization

Sub‐category: OperationTravel Time Reliability (Variation in travel time) NCFRP, FHWA VDOT, ODOT, NYSDOT, CDOT, University Transportation Center for 

Alabama,Travel Time Index NCDOT, ODOT, FDOT, FHWA, Delaware Valley Regional Planning Commission,  Buffer Time Index FHWA, Delaware Valley Regional Planning Commission,  Planning Time Index Delaware Valley Regional Planning Commission,  

Sub‐category: SafetyCrash Rate NCHRP, FHWA, ODOT, VDOT, NCDOT, TxDOT, FDOT, DDOTCrash Severity ODOT, University Transportation Center for Alabama,Fatality Rate NCFRP, TxDOT, ODOT, FDOT, DDOT, NCHRPRate of Injuries VDOT, 

Sub‐category: EconomicBusiness Turnover VDOTCommercial Land Values VDOTProperty Value NCHRP, FDOT, ODOT, Deployment Costs DOTFuel Consumption Nevada DOT, MnDOT, TxDOT, ITS‐CT, FHWA

Sub‐category: EnvironmentEmission (hydrocarbons, carbon monoxide,  nitrogen oxides and volatile organic compounds)

TxDOT, FDOT, NYSDOT, WSDOT, ODOT, University Transportation Center for Alabama, NCHRP, NCFRP, FHWA, Texas Transportation Institute 

Carbon dioxide Emission TxDOT, FDOT, NYSDOT, WSDOT, ODOT, NCHRP, NCFRP, FHWAEmission of Greenhouse Gas WDOT, FDOT, ODOT

18

Example of Performance Measure MatricesPerformance Measure Definition Units Spatial Scope  Time ScaleSub‐category: Operational

Travel TimeAverage time consumed by vehicles travelling a fixed distance

Minutes

Specific points on a section or a representative trip; separate for GP and HOV lanes

Peak hour, a.m./p.m. peak period, midday, daily

Travel Time ReliabilityMeasure of dispersion or spread of travel time distribution

MinutesSpecific section or a representative trip only

Peak hour, a.m./p.m. peak period

Travel Time Index Ratio of actual travel rate to ideal travel rateNone; Minimum value = 1.00

Section and area wide as a minimum; separately for GP and HOV lanes

As needed

SpeedAverage speed obtained by the vehicles in a fixed distance

Mile per hour

Specific points on a section or a representative trip only; separate for GP and HOV lanes

Peak hour, a.m./p.m. peak period, midday, daily

DelayExcess travel time used on a trip, facility, or freeway segment beyond what would occur under ideal conditions

Vehicle hours

Section and area wide as a minimum; separate for GP and HOV lanes

Peak hour, a.m./p.m. peak period, midday, daily

19

Data Collection, Advantages and DisadvantagesPerformance Measures

Techniques to Collect Data

Instrumentation Level

Advantages Disadvantages

Travel Time  Travel Time 

Reliability (Variation in travel time)

Travel Time Index Buffer Time Index Planning Time Index Speed Delay Intersection Delay Congestion Hours

Probe‐vehicle techniques

Manual

Low initial cost No special equipment needed Low required skill level 

High operating cost (high labor requirements)

Greater potential for human error Limited travel time/delay information 

available Limited sample of motorists

GPS

Moderate initial cost Reduction in human error Data easily integrated into GIS Detailed speed/delay data available No vehicle calibration is necessary as 

with the DMI method

Reception problems in urban “canyons”, trees

Limited sample of motorists Due to rapidly changing area, difficult 

to stay updated on what equipment to purchase

Electronic DMI

Moderate initial cost Proven technology Reduction in human error Very detailed speed/delay data 

available Commercially available software 

provides a variety of collection and analysis features

Not readily adaptable to a geographic information system

Limited sample of motorists

20

Future Steps Review of Project‐related MAP‐21 Efforts

• MAP‐21 has six primary goals which include enhancing safety,improving infrastructure condition, reduction of congestion,increasing system reliability, development of freightmovement and economic vitality, and improvingenvironmental sustainability.

• Identify Impact of MAP‐21 Legislation on TSMO‐specificPerformance Measures

• Review of FHWA MAP‐21 congestion‐related rulemakingefforts, AASHTO activities, and ongoing research anddevelopment work

21

MAP‐21 Goals and Performance MeasuresGoals Performance Measures Definition Units

Safety

Serious injuries per VMT Total crashes divided by the total vehicle miles traveled for which a police accident report form is generated, where at least one injury occurred.

Persons per mile

Fatalities per VMT Total fatal crashes divided by the total vehicle miles travelled, for which a police accident report form is generated, where at least one fatality occurred.

Persons per mile

Number of serious injuries Total crashes for which a police accident report form is generated, where at least one injury occurred.

Persons

Number of fatalities Total fatal crashes for which a police accident report form is generated, where at least one fatality occurred.

Persons

Number of transit‐related fatalities

Total fatal crashes related to transit system for which a police accident report form is generated, where at least one fatality occurred.

Persons

Infrastructure Conditions

IRI (International roughness index)

Ride Quality Parameter (IRI) IRI is the International Roughness Index and measures pavement smoothness.

m/km or mm/m (from 0 to 170)

Pavement structural health index

Percentage of pavement which meet minimum criteria for pavement faulting, rutting and cracking.

Percentage 

22

Answering Questions Both Known & Unknown: TSM&O Performance 

Measurement & Monitoring with Big Data

Michael Pack, UMD CATT Lab

23

Estimating Signal Performance based on Link Travel Times

24

Purpose of Research• Many agencies deploy ITS equipment to measure arterial travel times

• Very few studies (if any) investigated if travel time information can be used to estimate performance of traffic signals

• We present a method to estimate performance of traffic signals (their major coordinated movements) based on point‐to‐point travel time measurements

• The core of this method is based on well‐known volume‐delay functions which have been used in transportation planning for decades

• Use of these relationships has been reversed to estimate some fundamental signal performance measures of the downstream signal (e.g. V/C ratio, Level of Service (LOS), number of cycles to pass through the signal)  based on travel time between pairs of signalized intersections

25

Overall Framework• Input Data

• Signal Performance Measures

‐ Upstream travel time

‐ Volume‐to‐capacity ratio‐ Level of service‐ Number of cycles

Volume‐delay function (VDF) to establish this relationship

26

Defining Travel Time‐V/C Relationship1)  Retrieving travel time from Acyclica (or BlueTOAD)2)  Retrieving signal timing parameters from ATMS.now

Aggregating travel time by every cycle

3)  Traffic volume count using CCTV Count # of vehicles – through/queued vehicles

4)  Estimating capacity and saturation flow rate Estimating hourly volume using the observed # of vehicles and cycle length Saturation flow rate = capacity * (effective green time/cycle length)

5) Estimating V/C Retrieve occupancy rates from Sensys and find free‐flow conditions Defining free‐flow travel time

6) VDF Curve plotting and calibrationMATLAB Curve fitting toolbox finds the best combinations of calibration parameters

7) VDF validation Validate the calibrated VDF with a new set of data collected a different link

27

Study Area & Data Collection

28

Data Collection: Volume• Volume

=   V1 (Passing Vehicles) + V2 (Queued Vehicles)

(Passing Vehicles)

Counting passing vehicles during  at the stop line

(Queued Vehicles)

Counting queue at the beginning of 

29

Data Collection: Acyclica Travel Time• User Interface & Output Data Format

30

VDF Formulas UsedCategory VDF Formula

Newly‐developed · · ∗

Conventional(Most common VDFs)

Bureau of Public Roads (BPR) ∗ ∗

Conical

Akcelik . ∗

Conventional(VISUM)

BPR2

BPR3

Conical_Marginal

Logistic ∗ ∗

Quadratic t = 

Exponential ⁄⁄

Inrets..

..

Lohse

31

Example: Data Points Collected

0

50

100

150

200

250

300

350

400

0.00 0.20 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80

Travel Time (Secon

ds)

V/C

11‐Feb 16‐Feb 17‐Feb 18‐Feb 31‐Mar 1‐Apr

• NW 10th Int. to Airport Rd. ‐ 313 Data Points

32

Example: VDF Parameter Optimization

0

5

10

15

20

25

Occupa

ncy R

ate (

%)

Time

April 1st April 2nd April 3rd

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Freq

uency (%)

Bins (Travel time in seconds)

Optimizing ParametersUsing MATLABCurve FittingToolbox

OptimizationAlgorithm“Least‐square Method”

33

Example: VDF Parameter Optimization

0

50

100

150

200

250

300

350

400

0.00 0.20 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80Travel Tim

e (Secon

ds)

V/CObserved BPR BPR2 BPR3Conical Conical_M Akcelik LogisticQuadratic Exponential Inrets Lohse

0

50

100

150

200

250

300

350

400

0.00 0.20 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80

Travel Tim

e (Secon

ds)

V/C

Observed Newly‐derived function Calibrated BPR

• VDF Calibration and Plotting Results

Other VDFsNew VDF & BPR (Calibrated)

34

Example: Calibrated VDF ParametersVDF

FunctionsCalibrated Parameters

RMSE R‐squaredc d f

BPR 1.581 2.608 0.954 ‐ ‐ 37.85 0.78

BPR2 1.724 2.989 1.026 ‐ ‐ 50.23 0.64

BPR3 1.730 2.632 0.991 ‐0.600 ‐ 37.93 0.78

Conical 4.491 ‐ 0.950 ‐ ‐ 41.01 0.74

Conical_Marginal 1.210 0.497 1.050 ‐ ‐ 41.83 0.73

Akcelik 8.587 ‐ 1.100 ‐ ‐ 49.88 0.66

Logistic 24.140 ‐2.409 1.027 ‐0.460 ‐5.163 50.52 0.61

Quadratic 16.320 ‐82.180 0.950 146.000 ‐ 38.10 0.78

Exponential 3.485 0.343 0.974 ‐98.140 ‐ 50.35 0.62

Inrets 0.603 ‐ 1.050 ‐ ‐ 54.50 0.60

Lohse 2.130 1.483 0.950 ‐ ‐ 65.65 0.48

New VDF 0.648 ‐ ‐ ‐ ‐ 37.69 0.78

35

VDFs on Glades Rd

1 NW 13th St. 2 NW 10th Ave. 3 Airport Rd. 4 I95 NB Off‐ramp 5 I95 SB Off‐ramp

6 Renaissance Way 7 Butts Rd. 8 Town Center Mall Entrance 9 St. Andrews Blvd

Free‐flow T.T.: 50 s

BPR (R2 = 0.78)

New VDF (R2 = 0.78)

Free‐flow T.T.: 25 s

BPR (R2 = 0.55)

New VDF (R2 = 0.57)

∗ . ∗ .

. · · ∗ .

∗ . ∗ .

. · · ∗ .

Free‐flow T.T.: 25 s

BPR (R2 = 0.59)

New VDF (R2 = 0.58)

. · · ∗ .

Free‐flow T.T.: 25 s

BPR (R2 = 0.70)

New VDF (R2 = 0.71)

∗ . ∗ .

. · · ∗ .

Free‐flow T.T.: 12 s

BPR (R2 = 0.79)

New VDF (R2 = 0.79)

∗ . ∗ .

. · · ∗ .

∗ . ∗ .

36

V/C Estimation SimulationTravel

Time (sec)Estimated

V/C

(39.7, 0.29)

TT1

(75.5, 0.56)TT2

(110.0, 0.82)TT3

(157.8, 1.04)TT4

(190.0, 1.16)

TT5

(232.4, 1.29)

TT6

(291.0, 1.42)

TT7

(358.8, 1.59)TT8

39.7 0.29

75.7 0.56

110.0 0.82

157.8 1.04

190.0 1.16

232.4 1.29

291.0 1.42

358.8 1.59

(39.7, 0.29)

TT1

(75.5, 0.56)TT2

(110.0, 0.82)TT3

(157.8, 1.04)TT4

(190.0, 1.16)

TT5

(232.4, 1.29)

TT6

(291.0, 1.42)

TT7

(358.8, 1.59)TT8

37

V/C Estimation Simulation

39.7 0.29 100% ‐ ‐ ‐ ‐ ‐ 100% ‐ ‐ ‐

75.5 0.56 16% 30% 42% 10% 2% ‐ 100% ‐ ‐ ‐

110.0 0.82 ‐ ‐ 12% 33% 45% 7% 100% ‐ ‐ ‐

157.0 1.04 ‐ ‐ 1% 21% 54% 24% 60% 40% ‐ ‐

190.0 1.16 ‐ ‐ ‐ ‐ 10% 90% 21% 63% 16% ‐

232.4 1.29 ‐ ‐ ‐ ‐ 7% 93% 7% 76% 17% ‐

291.0 1.42 ‐ ‐ ‐ ‐ ‐ 100% ‐ 69% 31% ‐

358.8 1.59 ‐ ‐ ‐ ‐ ‐ 100% ‐ 50% 25% 25%

• LOS is F in 90% and E in 10%.

• The vehicles are expected to pass the intersection within 1 cycle (21%), 2 cycles (63%), and 3 cycles (16%).

38

Visual Validations

39

Practical Uses

0.00

0.20

0.40

0.60

0.80

1.00

1.20

1.40

1.60

40 75 125 175 225 275 325

V/C Ra

tio

Travel Time (Seconds)

Oversaturation

Near Saturation

Normal

0.00

0.20

0.40

0.60

0.80

1.00

1.20

1.40

40 50 60 70 80 90 100 110

V/C Ra

tio

Travel Time (Seconds)

Oversaturation

Near Saturation

Normal

40

Current Interface & Demonstration• Functionalities & Demonstration

• Only using the upstream travel time

• Estimating three signal performance measures‐ V/C‐ Level of Service‐ Number of Cycles

• Showing the summarized information (Default Setting)

• Showing the detailed information by clicking the Info. Box.

41

Estimating Network Congestion based on Google Traffic Maps

42

Assessing congestion based on Google traffic maps

Application based on Google traffic maps‐ Initiated from a question – can we use information from Google 

maps to assess network‐wide congestion?‐ Benefits for TMCs that do not have enough ITS infrastructure ‐ A simple idea: program counts # of pixels of certain color and 

compares with total # of pixels in a specific link‐ Links are defined manually (only once)

43

View the map

Create a map using latitude/longitude obtained

Load a map and open it in a browser

Wait until first screenshot is taken

Create/save links, intersections, and corridors on the screenshot

Trigger the (pixel) analysis process

Insert time interval for monitoring

Selected links in the link list?&

Time interval inserted?

Insert time interval for monitoring

Start the analysis 

Indicate the results on the interface

Generate output files

User decides when to stop

Yes

No

Program Architecture

44

Color Scheme & Outputs

• Google Color SchemeLegend Colors Traffic Condition

Green Normal OperationYellow Moderate CongestionRed High CongestionBlack Severe Congestion

• Output File FormatDate Time Link # of Total Pixels 

captures# of Pixels for 

Green# of Pixels for

Yellow

# of Pixels for Red

# of Pixels forBlack % of Green % of Yellow % of Red % of Black

45

Current Interface & Demonstration

• Selecting part of the map and saving it as a new map.

• Creating multiple links/ intersections/corridors by drawing on the map.

• Adjusting the data collection interval.

• Setting the congestion warning threshold.

• Detecting the congestion level by capturing the number of pixels (Black/Red/Yellow/Green)

• Creating output files

46

Defining Congestion Threshold

Threshold reached

55% of this link is very congested

47

Textual Outputs of the Congestion

48

Thank You!

Questions & Comments?

Aleks Stevanovic, astevano@fau.eduMichael L. Pack, packml@umd.edu

Visit NOCoE @http://transportationops.org/