Design of New Generation

Roadway and Traffic

Characteristics (RCI / TCI)

Database

August 12, 2013

AuthorsJeffrey Altbush / Brian Murphy

Findings and Recommendations

Goals of Study

Goals

• Provide support to the Traffic and Highway Data Section by evaluating

options for a next-generation database to store highway, rail, traffic,

freight and other transportation information

• Assess the current state of RCI and recommend improvements in

advance of a request by the Innovation Committee

– Recognition that RCI is an essential business system with FDOT but that it needs

to be “fixed”

– Prepare thoughtful solution based on input by those closest to the problem

RCI needs to be improved. What is the innovative solution?

3

Approach

• Phase 1: Familiarization with RCI and TCI

– Interviews with RCI/TCI support staff

• Focus on system architecture, database design, maintenance procedures

– Interviews with RCI/TCI stakeholders

• Focus on data usage, data deficiencies

• Phase 2: Research transportation databases used by other States

– What are other States doing to address similar requirements for highway inventory

/ traffic inventory?

• Phase 3: Prepare recommendations on improved database design

• Phase 4: Discuss recommendations with FDOT

4

Assess RCI design; interview stakeholders; research databases in other States

Executive Summary

Overview of Findings

• Database design restricts the ability to collect a complete set of

roadway data

• There are Federal or ad-hoc reporting requirements which require

capturing data on additional roadway features; the current RCI

features/characteristics paradigm makes it difficult to represent these

features in an easy way

• For several key stakeholders, the roadway is not being represented at

the level for which analysis/reporting must be performed

• Working to overcome these limitations has led to inefficient business

processes and additional IT costs

6

RCI has reached its functional limits due to database design

Overview of Findings (cont.)

• Since the inception of RCI, there has been significant research into

database designs for State highway inventory systems

• New Federal Highway legislation (MAP-21) is intended to serve as a

catalyst to adopting a new approach to implementing State highway

inventory systems

• GIS has become mainstream in the software industry and has become

an essential tool for transportation systems

• Recent offerings by software vendors provide the opportunity to

implement a new RCI with COTS software

• Other States are moving to capitalize on these trends

7

Important trends are aligned for FDOT to make a transformative change to RCI

Overview of Recommendations

• ESRI “Roads and Highways” has the potential to be a game-changer

in the economics of implementing a new RCI

– A COTS roadway management solution designed specifically for Departments of

Transportation and Highway Management agencies

– Released in 2012, ESRI released Roads and Highways

• benefits

– lower cost , risk, timeframe

– increase efficiency

– potential for joint funding with other departments

– potential for US DOT funding

8

Implement a new RCI using COTS software

RCI Database

RCI System Overview

10

Source: Phase I of an

Enterprise Geographical

Information System (GIS) for

Transportation;

FDOT contract BDI 40,

January, 2008

RCI System Overview

11

Source: Phase I of an

Enterprise Geographical

Information System

(GIS) for Transportation;

FDOT contract BDI 40,

January, 2008

Core Database Tables

• Key database tables are

shown here

• They implement the LRS

– Roadway ID’s

– Milepoint offsets

• Roadway inventory is

stored as features and

characteristics

12

Highway inventory represented as features and characteristics

Extract Tables

• Interface to most external

systems is addressed with

“Table 1, Table 2”

– Denormalized extract of

features and characteristics

for all roadways

13

External interfaces based on “flat file” table layouts

Observations on Database Design

• Strengths

– No software changes required to add new features or characteristics

• Limitations

– The implied transportation data model is hidden

– Complex relationships cannot be represented

– Complex objects cannot be easily described

– Difficult to reconstitute all of the features for a characteristics

– Attributes for characteristics limit ability to fully describe the object of interest

– Data entry for RCI is tied too closely to database design

– Transportation model limited by reporting requirements of the 1970’s

– Does not provide adequate support to track changes and report on roadways or

roadway inventory at points in the past

14

RCI database optimized for table-drive data maintenance

Limitations on Transportation Modeling

• Without the RCI Handbook, it would not be possible to know what highway features are represented in the database

– Features are treated generically (versus being represented as specific tables)

• Characteristics are defined via meta data

– The list of characteristics for a feature, and expected rules for those characteristics, can only be determined by looking at reference table

– Versus explicit definition as columns in a table

15

The implied transportation data model is hidden

Limitations on Modeling Roadway Features

• All features in RCI are assumed to be

related only to the roadway

– Relationship between features cannot be

distinguished

• No way to express complex

relationships or features

– e.g., sub types, association, inheritance,

composition

16

Source: Designing Geodatabases for Transportation

Complex relationships cannot be represented. Complex objects cannot be easily described

Limitations on Modeling Characteristics

• Each characteristic for a feature is stored in a separate record

– versus storage as a column in a table for a specific feature

17

Difficult to reconstitute all of the features for a characteristics

Limitations on Modeling Characteristics

• Characteristics– There are 3 Types of data

• Number (e.g. 99.99)

• Date (e.g. 02/04/2019)

• Text (e.g. Apalachee Parkway) (limited to 20 characters)

– Only one data field per characteristic

• Leads to compromises– Text field too short to record data (e.g.,

street names)

– Related data must be “stored” as a feature name (e.g. SUFLAY1, SUFLAY2)

• Leads to limitations– Cannot store more complex data types (e.g.,

documents, XML)

18

Attributes for characteristics limit ability to fully describe the object of interest

Source: RCI Handbook

Limitations on Data Entry

• Database design takes precedence

over ease of use

• Leads to cumbersome, inefficient

data entry process

19

Data entry for RCI is tied too closely to database design

Limitations on Government Reporting

• RCI data model based upon early set of

Federal highway reporting requirements

– Data model has not evolved as reporting

requirements have changed

• While it is possible to add additional

features and characteristics, the

fundamental nature of the database has

not changed

– The LRS and implied transportation model

reflect “simpler times” for highway reporting

20

Transportation model limited by reporting requirements of the 1970’s

"The form of transportation data is determined by a sequence of federal data reporting requirements that resulted in the creation in the 1970s and subsequent years of a number of fairly standard data sets. The 1972 act reauthorizing federal aid for highways established a requirement for state reporting of highway data through the Highway Performance Monitoring System (HPMS). This system is based on unique roadway identifiers and one-dimension measurement mechanisms to locate major roadway attributes and minor roadway sample sections where data are collected. A later act established a related requirement for data from the states to create a National Highway Planning Network map using a defined data structure and roadway identifiers that are unique within each state. HPMS sample sections are to be identified within the new network structure.”

Source: “Transportation Networks in ArcGIS: An Alternative to Geometric Networks”

Limitations on Tracking Changes

• Unable to “roll back” the clock to

see the status of roadways or

inventory at times in the past

• Not able to enter pre-enter

planned changes

21

Cannot report on roadways or inventory at points in the past

Data Gap Analysis

Stakeholder Interviews

23

Ref #

DescriptionProduct

EvaluationEMO Safety

Roadway Design

Work Programs

Traffic Ops

Maint.Office

RCICoordinator

a. Requested capability already exists in RCI X X X

b.

RCI has the necessary data but the current RCI database design and the way data is encoded makes it hard to perform analysis

X X

c.The roadway is not being represented at the level for which analysis/reporting must be performed

X X X

d.

There are Federal or ad-hoc reporting requirements which require capturing data on additional roadway features; the current RCI features/characteristics paradigm make it difficult to represent these features in an easy way

X X X X X

e.Entering more detailed data into RCI would be hampered by the current limitations in data entry

X X

f.There are Federal reporting requirements which require an expansion of the roadways covered by RCI.

X X

g.There should be tighter integration between RCI and other application systems

X X

h.There should be additional methods to locate events along the roadway

X

During the week of April 8th, 2013, interviews were held with major stakeholders of RCI. The following themes emerged.

Related to limitations in database design

Stakeholder Interviews (cont.)

24

Ref #

DescriptionProduct

EvaluationEMO Safety

Roadway Design

Work Programs

Traffic Ops

Maint.Office

RCICoordinator

i.

What is the right balance of data to be stored within RCIwhile still being able to view a complete set of roadway information

X X X

j.

There are ad-hoc reporting requirements which require geo-spatial, but non-roadway, features which are unavailable in RCI

X

k.

While ownership is clearly defined for all RCI features and characteristics, data usage for business systems has not been cataloged

X X

l. Some characteristic data within RCI is missing or incomplete X

m.There should be tighter integration between RCI and other GIS systems

X

n.Database design restricts the ability to collect a complete set of data

x

o.Search capabilities of the RCI query screens could be improved

x

p.

Currently the update of RCI is manual-intensive even when data for updates is available in electronic form. This can result in data entry errors, invalid data being entered, and an inefficient update process.

x

q.

The RCI database has limited ability to track changes, restore changed data and to report on the roadway status at time points in the past.

x

Related to limitations in database design

Observations on Data Gaps

25

• The current approach to transportation data modeling has reached its limit

– Currently RCI is a collection of attributes. There is an implied data model but it is not easily seen or understood. Difficult to collect attributes for the same object.

– Currently only one method for locating objects on a road (e.g, milepoints)

– Temporal support is weak

– However, an improved relational model alone will not provide a solution without substantial revisions to the front-end systems used to maintain and deliver that data

• The current architecture of RCI has reached its limit

– The capabilities of the current mainframe-based RCI application are focused on simple data maintenance and query

– Merging information and adding value (e.g., answering analytical questions) to the data is left to the end user

– Sharing / distribution of RCI data relies primarily upon physical movement (e.g., replication)

Recommendations

COTS Recommendation

• Utilize Existing DOT ESRI Software

• ESRI Roads and Highways

– Builds upon ESRI’s research into transportation data modeling and many years of

working with many State and Government DOT

27

Concept of Operations

• Mainframe RCI application to be retired

– Data migrated into new tables and data structures

• TCI database can remain largely unchanged

– Input of data to new RCI database will need to be modified

• Existing GIS application to be augmented by capabilities of Roads and Highways

• Some supporting applications can be replaced with COTS tools

– SLD

– HPMS reporting

– Data maintenance

• RCI legacy extract tables will be created with “reverse migration” scripts

– Over time, applications which rely on relational data can be changed to access new tables

28

Opportunities

• Typically, “buy versus build” solutions provide

– Lower cost

– Lower risk

– Faster time to deployment

– More functional capability

• Modern database concepts and system architecture

– Provides a solid technical foundation for future

• Allows FDOT to conserve internal development resources

– Only develop software to address unique State requirements

– Ongoing fixes and enhancements delivered as part of software maintenance program

– Information currently stored in other system’s geodatabases can be added to (or shared with) RCI

• Provides an opportunity to close identified data gaps

– Resolve LRS and transportation data modeling issues

– Temporal data reporting (previous and future state of LRS and inventory)

– Addition of off-system roads

29

Challenges

• Closing the data gaps

– Changes to LRS and data model will require important discussions to resolve how best to balance modeling “accuracy” and reporting needs

– Also, how best to balance highway inventory reporting versus asset management

• Support of interfacing systems

– RCI data is used extensively within FDOT (and beyond)

– Need to minimize initial impact to other systems so that they remain operational without changes for a period of time

– Funding required to support enhancements for other systems to take advantage of new infrastructure and RCI database

• Data maintenance workflows

– Completely different approach and tools for data maintenance

– ESRI recommends use of separate maintenance and published databases

– Opportunities to develop maintenance workflows to control update and release of changes

30

Why This Solution? Why Now?

• Over the years there have been other studies on how to improve RCI and GIS

enterprise data sharing at FDOT

• All of these studies have produced valuable recommendations but many of the

same issues exist today

• What is different now?

• Timing is everything

• New Federal reporting requirements have reached a point where something must

be done with RCI

• Timing of this study allows FDOT to take advantage of a relatively new COTS

product built specifically for highway data management by an industry leader

• “Faster, cheaper, better”

– Roads and Highways provides FDOT with a solution to move forward rapidly

– Provides a modern system architecture and tools to provide a solid technical

foundation for the future

31

What are other States Doing?

What are other State DOT’s Doing?

• We see a growing trend to adopting the approach we have

recommended

– Colorado, Minnesota, and Georgia are moving forward in the ESRI Roads and

Highways implementation

– Per the GIS-T 2013 Conference, there are a number of states who have vocalized

interest in ESRI’s Roads and Highways: Indiana, Louisiana, Maryland,

Massachusetts, Michigan, New Mexico, Vermont, Alaska, Arizona

• Specific approaches to which vendor and which data model may differ,

but the trend is clear

– Depending on when State DOT initiated their development work

– Depending on GIS vendor preference

33

Which States use Which Software?

34

Roads and Highways Interest Level

• Contracting or

Implementing

- North Carolina

- New York

- Minnesota

- Georgia

- Colorado

- Alabama

- Indiana

- Arizona

- West Virginia

- City/County of

Denver

- County of Boulder

• Involved discussions

- Massachusetts

- Kansas

- Louisiana

- Alaska

- Rhode Island

- Maryland SHA

- New Mexico

- Ohio

• Radar screen

- Utah

- Delaware

- Maine

- California

- Florida

- DC DOT

- WMATA

- FHWA

• Purchased product

- Nevada

- Vermont

- Virginia

- Washington

Minnesota

• Furthest along with Roads and Highways implementation, due to

Statewide Enterprise License Agreement with ESRI

• Roads and Highways implementation includes developing the logical

data model; doing the Gap analysis, and then doing the

implementation.

• Integrating ArcGIS with Stratus

36

Colorado

• Released an RFP looking for Roads and Highways integration

• Currently, CODOT’s data is using both a tabular format, in Oracle,

and a GIS feature class environment, using SQL Server (ArcSDE).

– They are not remotely tied to one another in any meaningful way.

– Oftentimes, there are requests for data, and significant conflicts exist between

both datasets, amounting to confusion in regards to which dataset is correct.

– Their RFP was essentially written to request an ESRI Roads and Highways

solution, which addresses many of the same issues that FDOT is experiencing.

– The proposed system will allow users to do both web-based tabular edits, in

addition to geospatial edits.

– Additionally, they are able to “paint” road segments using a slider-straight line

diagram capability, to assist editors in making determinations as to where to start

and end their attribution edits.

37

Colorado (cont.)

• Developing GIS-based business rules – Will facilitate improving the quality of data and will assist in spatially and a-spatially

resolve conflicts within the data (an with other connected databases).

• Empowering their small team – Only 5 people in their GIS shop. Effort to migrate them from data-maintainers into a

positions of “educators” and “delegators.”

– Their focus will then shift towards training the DOT workforce on how to edit, update the GIS through a web-based editing session (this approach avoids the need for Esri ArcGIS software licensing)

• Starting small – Rolling out to GIS-ready business groups from within the DOT.

– Their pavement condition group is the most “GIS-savvy” they will begin with them, and then work towards incorporating less-GIS-savvy groups into the editing environment.

• Regarding video logs– their pavement condition group currently collects both pavement condition data in

addition to windshield perspective video-logs.

– They are also aiming to incorporate traffic safety GIS-based crash models into their GIS.

38

Georgia

• GIS is already their main business in Teague’s group; Thus, they didn’t have to build customized technologies.

• They chose ESRI as a COTS solution, so they could get system support they needed

• The prefer ESRI’s R&H approach, due to it’s simple LRS approach (one LRS / Datum) plus associated tables

• GaDOT feels that ESRI’s pricing is very accommodating (compared to the expense of trying to do yourself)

• They like the ability to integrate with Agile Assets, to help solve many issues

• There is a very good and extensive knowledge base via ESRI User Groups: Over 10 DOTs are working to implement Roads and Highways, and communicate regularly on how one-another are implementing Roads and Highways

• ESRI Responds to the Speed of Business: They are very proactive in their development, and are paying close attention to Federal policy to ensure that their software meets the needs of their DOT customers

39

Their reasons for adopting Roads and Highways

Georgia

• Cost reduction from adopting a GIS approach

– Georgia was spending a lot of money in driving routes to make distance

calculations; most of which could be done directly in GIS.

– Geometric measurements in GIS were comparable if not better than driven routes.

This approach increases safety and significantly reduces costs to DOT

• Collecting GIS from:

– Aerial imagery, Video Logs, CAD project files / surveys / etc.

– GIS First! Field Collection is used as a last resort, if all else fails

• Developing master repository of DGC / CAD files, to facilitate

accessibility of other GIS-like data

40

Another benefit from adopting a GIS approach

Georgia (cont.)

Per Teague Buchannan:

• Funding is available for the creation of a Geographic Information Office at each DOT. The US DOT wants an effective “TRANSPORTATION FOR THE NATION” Dataset, which will include local roads

• Steve Lewis’s vision is to replace the US Census Tiger roads layer as a more functional and robust nationwide dataset

• Right now they are doing a 100% match for local road collection

• Computer Aided Dispatch / 911 phone levies are often also used as a funding mechanism for local road updates and 911-addressing initiatives by other states

• The underlying intent of MAP-21 is to empower GIS as the driver for Federal data reporting public roads that can be used to geolocate attribute data on a roadway.

41

Funding is available as a result of MAP-21

Next Steps

Next Steps

• Introduction / overview of Roads and Highways by ESRI

• Determine the supporting material required for the Innovation Committee to be produced by NG

• Support implementation of proof of concept by ESRI, e.g.,

– Training in Roads and Highways

– Discussion of transportation data modeling issues. e.g.,

• Routes

• LRS

• Events (e.g, features / characteristics)

• Discussion of how best to adapt the existing RCI GIS

43

Prepare supporting material for presentation to the Innovation Committee to support a refresh of the RCI Database.

Next Steps (cont.)

• Create supporting material, as determined. e.g.,

– Motivation for moving to a new RCI

– Motivation for adopting Roads and Highways

– Concept of operations in use of Roads and Highways

• Data editing / publishing

• Reporting

• Data to be included in RCI versus links to related systems

– High level system architecture

– Concept of conversion plan

– Concept of support of legacy systems

– Estimated cost / time to implement

– Critical success factors for implementation of Roads and Highways. e.g.,

• RCI data governance

• GIS data governance

44

FDOT Involvement

• Attend introduction / overview of Roads and Highways

• Participation in discussions on transportation data modeling and GIS

implementation issues

• Support implementation of proof of concept by ESRI, e.g.,

– Providing computer to run the proof of concept

– Providing RCI data

• Assistance in discussions / fact gathering needed to generate

supporting material

• Support in estimating costs / timeframes for internal FDOT

implementation tasks

45

Adapting the Frameworks to RCI

• Frameworks need to be adapted to each organization and application

• Prototype

– Resolve key functional issues which influence how model is applied

– Resolve issues which influence performance and scalability of application

– Gain experience with strengths and limitations of framework

– Identify issues in converting existing data to new database

Critical Success Factors for RCI 3

• Establish a base map at agreed-upon level of detail

• Well defined migration plan

• Coordination with stakeholders

• Data Governance

• Integration with existing FDOT GIS standardization efforts

• Training / Staffing

47

Next Steps

• Pilot Study

• Prepare a more detailed implementation / roll out plan

• Prepare a cost estimate