Performance Management In Performance Management In The Production of The Production of

Geospatial DataGeospatial DataMark GuttmanMark Guttmanmguttman@costquest.commguttman@costquest.com

The questions:The questions:

There is a perception that the There is a perception that the public sector’s production of public sector’s production of geospatial data is wasteful… geospatial data is wasteful… Is this true?Is this true? Can the perception be clarified Can the perception be clarified

and can a clear, reasoned and can a clear, reasoned response be delivered?response be delivered?

The questions:The questions:

Geospatial data supports multiple Geospatial data supports multiple endeavors within private / public endeavors within private / public sectorssectors

Because it’s development is rarely Because it’s development is rarely the business objective it is difficult to the business objective it is difficult to quantify the value it produces.quantify the value it produces. What is geospatial data?What is geospatial data?

What are the processes of producing it?What are the processes of producing it? Can the downstream value it creates be Can the downstream value it creates be

measured?measured? Can the assets consumed during it’s Can the assets consumed during it’s

production be measured?production be measured?

The questions:The questions:

Geospatial data are specialGeospatial data are special In terms of economic value and In terms of economic value and

organizational strategy how special are organizational strategy how special are they?they?

With respect to the questions and issues With respect to the questions and issues posed earlier, can models and tools posed earlier, can models and tools from other business areas be helpful?from other business areas be helpful?

Or, are these perceptions and Or, are these perceptions and questions not yet significant enough to questions not yet significant enough to drive action?drive action?

As practitioners, what should As practitioners, what should be done?be done?

Maintain Present Mode of Maintain Present Mode of Operation (PMO)Operation (PMO)

Panic, Hide, Obfuscate (PHO)Panic, Hide, Obfuscate (PHO) Develop something before Develop something before

something is developed for us…something is developed for us…

Themes for this presentationThemes for this presentation

Establish the perceptionEstablish the perception Define the problemDefine the problem What questions could be asked?What questions could be asked? What thought models could be What thought models could be

helpful?helpful?

Historical Background-Historical Background-Development of NSDIDevelopment of NSDI Establishment of NSDI (1996)Establishment of NSDI (1996) Executive Order Executive Order 1290612906

““to implement the recommendations of to implement the recommendations of the National Performance Review; to the National Performance Review; to advance the goals of the National advance the goals of the National Information Infrastructure; and to Information Infrastructure; and to avoid avoid wasteful duplication of effort and wasteful duplication of effort and promote effective and economical promote effective and economical management of resourcesmanagement of resources by Federal, by Federal, State, local, and tribal governments”State, local, and tribal governments”

[emphasis added][emphasis added]

Historical BackgroundHistorical Background

NSDINSDI FGDC coordinates FGDC coordinates Two highly visible componentsTwo highly visible components

Data Sharing Data Sharing Geospatial One Stop-allows for Geospatial One Stop-allows for

publication of metadata recordspublication of metadata records

Data Acquisition MarketplaceData Acquisition Marketplace Allows for shared acquisition of dataAllows for shared acquisition of data

NSDINSDI

-Partnership portal yields 3000 -Partnership portal yields 3000 acquisition records with ~1000 acquisition records with ~1000 contactscontacts

FGDC 2006 Annual ReportFGDC 2006 Annual Report

What is the problem?What is the problem?

From GAO, June 2004From GAO, June 2004 “… “… FGDC and GOS have taken FGDC and GOS have taken

actions to coordinate government actions to coordinate government geospatial investments…However geospatial investments…However these efforts have not been fully these efforts have not been fully successful”successful”

““Until these problems are resolved Until these problems are resolved duplicative investments are likely duplicative investments are likely to remain”to remain”

What is the problem?What is the problem?

From From NGPONGPO Geospatial Geospatial Investments (Geospatial Investments (Geospatial Investments) work group (Dec Investments) work group (Dec 2004)2004) ““Little evidence of governmentwide Little evidence of governmentwide

management”management” ““Total annual investment is “guesstimated” to be Total annual investment is “guesstimated” to be

somewhere in the range of $4‑8 billion annually somewhere in the range of $4‑8 billion annually and up to 50 percent of this amount has been and up to 50 percent of this amount has been asserted to be duplication.”asserted to be duplication.”

Views on expenditures and duplication are Views on expenditures and duplication are based on anecdotes or assertions that are based on anecdotes or assertions that are unsubstantiatedunsubstantiated

Summing Up…Summing Up…

A number of different factors make us say A number of different factors make us say spatial data are specialspatial data are special

But, in terms of how we develop the data But, in terms of how we develop the data which drives our industry, there seems to which drives our industry, there seems to be a problembe a problem

……the problem is a perception that we are not the problem is a perception that we are not particularly efficientparticularly efficient

……as geospatial data begins to drive solutions as geospatial data begins to drive solutions outside of the traditional geospatial realm, outside of the traditional geospatial realm, perceptions of quality/value could become perceptions of quality/value could become increasingly negativeincreasingly negative

……for a technology which sells it’s ability to help for a technology which sells it’s ability to help make better decisions as a benefit, why can’t we make better decisions as a benefit, why can’t we consistently quantify and defend benefits?consistently quantify and defend benefits?

Does this sound familiar?Does this sound familiar?

Huge investmentHuge investment Presumption of inefficiency in resource Presumption of inefficiency in resource

utilization. utilization. Benefits are not disputed but are difficult to Benefits are not disputed but are difficult to

quantifyquantify Little ‘real evidence’ for decisions. Lots of Little ‘real evidence’ for decisions. Lots of

normative statements.normative statements. ““In the computer industry…failures are In the computer industry…failures are

covered up, ignored, and/or rationalized. covered up, ignored, and/or rationalized. As a result, we keep making the same As a result, we keep making the same mistakes over and over again” (Standish mistakes over and over again” (Standish Group Group CHAOSCHAOS report, 1994) report, 1994)

Where do we go from here?Where do we go from here?

Practitioners who produce Geospatial Data Practitioners who produce Geospatial Data need a means to…need a means to… Justify initial investments which start projectsJustify initial investments which start projects Compare rival proposals and select those which Compare rival proposals and select those which

produces the greatest benefit in light of produces the greatest benefit in light of organizational goalsorganizational goals

Manage existing resources and demonstrate Manage existing resources and demonstrate increasing efficiency over time increasing efficiency over time

We need to focus not just on the data, the We need to focus not just on the data, the production/distribution technologies and production/distribution technologies and legalities—we need to look at how we legalities—we need to look at how we manage the production process.manage the production process.

The foundational management The foundational management issues issues 1)1) How are we selecting the correct How are we selecting the correct

geospatial data projects?geospatial data projects?A.A. Costs/Benefits, ROI/IRR toolsCosts/Benefits, ROI/IRR tools

2)2) How are we managing our existing How are we managing our existing projects?projects?

A.A. A A performance managementperformance management questionquestion

These are very similar issues but I’ll draw a distinction These are very similar issues but I’ll draw a distinction upon the upon the point in timepoint in time the question is asked and the question is asked and to whomto whom it is relevant. it is relevant.

To answer these questions: To answer these questions: Is it possible to apply lessons, tools and techniques Is it possible to apply lessons, tools and techniques

from other areas to the geospatial data industry?from other areas to the geospatial data industry?

What is Performance What is Performance Management?Management?

A data driven organizational process A data driven organizational process designed to improve how the designed to improve how the organization works.organization works.

Uses transparent methods to Uses transparent methods to objectively measure how we are objectively measure how we are doing today versus how we want to doing today versus how we want to be doing.be doing.

It’s an umbrella term for a number of It’s an umbrella term for a number of different ideas: Quality Management, different ideas: Quality Management, Six Sigma, Balanced Scorecards, Six Sigma, Balanced Scorecards, CMMI, Value Management…CMMI, Value Management…

I’m not selling consulting I’m not selling consulting hours, please, don’t leave nowhours, please, don’t leave now

It’s not reporting for reporting It’s not reporting for reporting sake; not methodology for the sake; not methodology for the sake of the methodologist.sake of the methodologist.

Bottom line: What you can’t Bottom line: What you can’t measure, you can’t manage…measure, you can’t manage…what you can’t manage you what you can’t manage you can’t improve.can’t improve.

In terms of Geospatial Data In terms of Geospatial Data Development…Development… Two questions for my researchTwo questions for my research

1) How do some basic data 1) How do some basic data development processes work? development processes work?

A A process modelingprocess modeling question question Can UML or other process modeling tools be Can UML or other process modeling tools be

used to explore what is happening?used to explore what is happening? With process models, can firms begin to see With process models, can firms begin to see

how assets are consumed, where how assets are consumed, where duplication exists and where there exists duplication exists and where there exists comparative advantage?comparative advantage?

2) Is our work helping to 2) Is our work helping to advance/achieve specific goals?advance/achieve specific goals?

A A process alignmentprocess alignment question question

To understand our data To understand our data development processesdevelopment processes We need to categorize the data We need to categorize the data

development work which consumes development work which consumes geospatial investment: like hardware, geospatial investment: like hardware, human capital, existing geospatial human capital, existing geospatial data, software and networks. data, software and networks.

Proposal: define a geospatial Proposal: define a geospatial process, model it, demonstrate work process, model it, demonstrate work and asset flowsand asset flows

A Business Process ModelA Business Process Model

Quantifying Work Process Quantifying Work Process Improvement Benefits (CMMI Improvement Benefits (CMMI example)example) As our people get better trained, As our people get better trained,

their work ‘quality’ their work ‘quality’ improvesimproves

Is our work helping to Is our work helping to advance/achieve specific goals?advance/achieve specific goals?

Strategy

Plan

Teach\Learn

Outputs

Scope

Design

ModelRequirements

Cost/Benefits

Implementation Plan

Purpose: How are wedemonstratingthis linkage?

Proposal: Within a geospatial data framework, is it possibleto use existing alignment tools to show that the right thingsare being done?

From Tomlinson (2004)

Are we doing the right things?Are we doing the right things?

Doing it right vs. doing the right Doing it right vs. doing the right things?things?

Or, it isn’t if we have x million Or, it isn’t if we have x million more metadata records in our more metadata records in our clearinghouse, it’s what are clearinghouse, it’s what are those metadata records helping those metadata records helping us do.us do.

Project Alignment ExampleProject Alignment Example

Critical To Quality (CTQ) Critical To Quality (CTQ) diagram (ASQ, 4/07) diagram (ASQ, 4/07)

Balanced Scorecard Balanced Scorecard Alignment ExampleAlignment Example

2006-Defense Finance and Accounting Services Strategic Plan 2006-2011

ConclusionsConclusions

In terms of business management, In terms of business management, geospatial data development geospatial data development projects seem to have ample projects seem to have ample opportunities to improve. opportunities to improve.

Although I began exploring issues on Although I began exploring issues on Federal level, the concepts are Federal level, the concepts are applicable for State/Local and applicable for State/Local and Private organizationsPrivate organizations

Not dissimilar to issues that framed Not dissimilar to issues that framed development of PMBok or SWEBokdevelopment of PMBok or SWEBok

ConclusionsConclusions

Industry needs to explore ways to Industry needs to explore ways to demonstrate maturity and efficiency before demonstrate maturity and efficiency before someone else does.someone else does.

Existing tools and practices from other Existing tools and practices from other industries may be helpful.industries may be helpful.

Although the tools may exist elsewhere, is Although the tools may exist elsewhere, is it worthwhile to marry geospatial language it worthwhile to marry geospatial language and work patterns with established and work patterns with established management frameworks? If it’s going to management frameworks? If it’s going to work the incentive has to be created for it work the incentive has to be created for it to be used and the organization’s culture to be used and the organization’s culture needs to accept it.needs to accept it.