“There are special management challenges, and I think that that's an area that we in agencies such as NOAA, need to spend an extra amount of time on. We have very talented workers and very talented employees, many of whom have advanced degrees, and they have been successful because of certain behaviors in their field. As you progress through the system in any organization, you need to develop other skills;…”Vice Adm. Lautenbacher

Innovation, Standards, and NOAA Data ManagementTed HabermannNOAA National Data Centers

Mature Organizations

We all know that new technologies emerge slowly, grow quickly (if they catch on) and then fade away. This common knowledge has been described as the technology S-curve.

Why does it exist?

TIME

The Technology S-Curve

Geoffrey Moore has attributed the S-curve to the technology adoption life cycle where techies and visionaries are early adopters, pragmatists make up the bulk of users, and luddites fill out the tail of the distribution.

TIME

Visionaries

Luddites

Pragmatists

The Adoption Curve

The ChasmMoore has also described the “chasm” in the adoption life cycle. He proposes that many new technologies do not make it across the chasm between visionaries and pragmatists. They fall into the chasm. The technology S-curve with the chasm might look like:

TIME

TIME

Technology CycleTechnologicalDisruption

Era of Ferment

Selection

DominantDesign

Disruption #2(destroys existingcompetence)

Technology Cycle

Types of Innovation - 1Sustaining / Incremental Innovation: generally small innovations in products and processes aimed at existing customers.

Disruptive / Discontinuous Innovation: significant innovations generally aimed at unknown or non-existent customers.

Unidata Objectives (1998):

“These objectives either respond to users' current needs or advance Unidata toward meeting future needs effectively. Most of the "responsive" items are continuations of current Unidata objectives, and their importance is well established. But only by looking beyond present needs to anticipate future ones, and by pursuing the most promising technical advances, can Unidata remain effective. This is true even though some of these advances involve uncertainties, and the demand for them may not be apparent as yet”. Unidata, 2003 Proposal.

Sustaining Innovation

Disruptive InnovationClayton Christensen, The Innovator’s Dilemma

TIME

Cu

sto

me

r M

etr

ic

Unidata (netCDF) EvolutionDisruptive Innovation: Always includes a decrease in metrics for current customers so it is difficult for mature organizations.

C

SustainingInnovation

Java

DisruptiveInnovation

In the Unidata case we are now seeing the disruptive switch to Java play out. The capabilities of the Java version of the netCDF libraries have now surpassed the original C version.

Types of Innovation - 2Component Innovation: Making existing components better.

Architectural Innovation: putting existing components together in new ways.

Architecture = OrganizationStructure in mature organizations tends to evolve to match product architectures. Architectural Innovation, therefore, many times includes elements of organizational change.

TIME

Disruptive Innovation

Product InnovationDesign CompetitionCommunity-driventechnology change

Component,Architectural,Sustaining andProcessInnovation

What do we make? How do we makeit (better)?

Innovation & Technology Cycle

Other Differences

TIME

ResearchPrototypesCustom developmentsNetwork buildingUncertainty

Operational SystemsProduct FamiliesPredictabilityPartnerships

(non-compliancecost increases

with time)

Network EffectsValue = f(N2)

Standards

How Standards Change The Game• Expanded Network Externalities (Network effect turns on)• Reduced Uncertainty and Risk in Technology Decisions• Reduced Consumer Lock-In to Particular Components• Competition in the Market vs. Competition for the Market• Competition on Value vs. Features• Competition to Offer Proprietary Extensions• Component vs. Systems Competition

Standards shift the locus of competition from systems development to component development. Specialists tend to thrive in the mix-and-match environment created by interface standards. Generalists and system (stovepipe) developers tend to thrive in the absence of standards. In the absence of standards:1) there is no architectural innovation (no mix-and-match) and2) the organization can not benefit from component innovation.

Once a standard has been agreed on (selection), the organization benefits from component innovation and architectural innovation.

Andy Grove: Communication Overcomes ComputingThe framework is changing now. The Internet is redefining software. The Internet is redefining the role of computing and communication and their interaction with each other. I still don’t understand the framework. I don’t think any of us really do. But some aspects of it are pretty clear. It’s proven not to be computing based but communications based. In it computing is going to be subordinated to the communications task.

“Decisions Don’t Wait”,

Harvard Management

Update.

Kevin Kelley: The Web and SharingThe revolution launched by Netscape’s IPO was only marginally about hypertext and human knowledge. At its heart was a new kind of participation that has since developed into an emerging culture based on sharing.

“We Are The Web”, Wired, August 2005

DESKTOP

Geographic Information Systems

MULTI-USERGEOSPATIAL DATABASE

+ Relational Databases

NETWORK(ENTERPRISE GIS)

+ World Wide Web

Technology Confluence

Computing Communication

Infrastructural TechnologiesIT is, first of all, a transport mechanism – it carries digital information just as railroads carry goods and power grids carry electricity. And like any infrastructural technology, it is far more valuable when shared than when used in isolation. The history of IT in business has been a history of increased interconnectivity and interoperability, from mainframe time-sharing to minicomputer-based local area networks to broader Ethernet networks and on to the Internet. Each stage in that progression has involved greater standardization of the technology and, at least recently, greater homogenization of its functionality. For most business applications today, the benefits of customization would be overwhelmed by the costs of isolation. Nicholas G. Carr, IT Doesn’t Matter, Harvard Business Review, May, 2003

Corollary to Metcalf’s Law: the cost of non-compliance goes up as the square of the number of members of the network.

Why No Standards?The longer the market takes to determine a standard, the more expensive it will be for firms operating within that market. The more expensive this competition becomes, the greater the tendency for firms to cooperate at the beginning.

The difficulty with this reasoning is that it is difficult for individual firms to determine how expensive or how long it will take the market to determine the dominant standard. Nor are companies willing to cede control of such an important aspect of their market early in a competition. Booz Allen Hamilton, 2005.

The science community generally does not value sharing.

Organizational Approaches / Skills

TIME

Discovery-basedPlanning

Resolving CriticalUnknowns

Management

Detailed Plans w/Accountability

Long-term Plans& Requirements

PPBES

Leadership

TIME

NOAA relies on many technologies which are changing rapidly. To do well in long-term, NOAA must:

2) develop and use mechanisms for standards adoption and selection3) realize that different phases require different organizational structures /strategies,4) support multiple structures / strategies simultaneously.

1) recognize phases of the technology cycle,

Multiple Technologies

Innovation, Standards & NOAAThere is a considerable innovation literature that can help NOAA learn the new skills required to innovate strategically and effectively.

Technology is evolving from a computing tool to a communication tool. It is becoming an infrastructure technology.

Standards are critical to building value of infrastructure technologies.

Standards are critical to organizationally effective component and architectural innovation.

NOAA must develop and use processes for selecting and applying standards.

The requirements and approaches to planning are very different in the different phases of the technology cycle.

Understanding and explicitly recognizing the differences in phases of the technology cycle and the differences in balance between management and leadership skills might help NOAA.

Examples from NGDC

1. NOAA Maps, The NOAA Observing System Database and NOSA WebsiteInternet mapping in NOAA is a great example of an “Era of Ferment”. Many different tools produce maps that look and behave differently. We are collecting capabilities for over 400 NOAA Internet Maps and comparing those capabilities to a single map that includes information for ~100 NOAA Observing Systems. How would a selection event change NOAA’s approach to internet mapping?

2. The Data Processing PipelineAn example of components/architecture for data processing and ingest in an open source Java project initiated at NGDC.

NOAA MapsOver 400 maps have been captured from NOAA Web pages and described in the NOAA Maps Project.

http://www.nosa.noaa.gov/noaa_maps/

Maps at NGDC

NOSA Website @ NGDC

GeospatialDatabase

ArcIMS

WIST

WMS

LAS

SpatialSQL

SQL

FYPStations,

QA

NOSA Website

Map

Metadata

WFS

Desktop GIS

Spreadsheet,E-mail, txt,

etc.Pipeline

The Data Processing PipelineA pipeline executes a sequence of plug-able data processing tasks. The NGDC data processing pipeline provides a set of pipeline utilities designed around work queues that run in parallel to sequentially process data objects. The pipeline is an open source project hosted in the Jakarta Commons Sandbox(http://jakarta.apache.org/commons/sandbox/index.html).

Processing steps are specified as a series of stages in an XML configuration file.

FilesOGC

Simple Featuresin Memory

SpatialDatabase

DMSP Orbit Processing

Stage 1. Find Matching FilesStage 2. Avoid Duplicate ProcessingStage 3. Read Data / Create Spatial ObjectsStage 4. Low-Res ThinningStage 5. High-Res ThinningStage 6. Write Spatial Data to Database

Building on Experience

Louis Liebenberg, The Art of Tracking: The Origin of Science (Wired, June 2003)

Leadership Model: Positive Deviance

Positive deviance says that if you want to create change, you must scale it down to the lowest level of granularity and look for people within the social system who are already manifesting the desired future state. Take only the arrows that are already pointing toward the way you want to go, and ignore the others. Identify and differentiate those people who are headed in the right direction. Give them visibility and resources. Bring them together. Aggregate them. Barbara Waugh

Ted.Habermann@noaa.gov

Booz Allen Hamilton, Geospatial Interoperability Return on Investment Study, 2005, http://gio.gsfc.nasa.gov/docs/ROI%20Study.pdf.

Christensen, C., The Innovator’s Dilemma, Harvard Business School Press, 1997, 225p.

Clark and Wheelwright, Revolutionizing Product Development, The Free Press, New York, 1992, 364p.

Govindarajan, V. and C. Trimble, Building Breakthrough Businesses Within Established Organizations, Harvard Business Review, May 2005, p. 58-68.

Lautenbacher, C., Business of Government Radio Interview, http://www.businessofgovernment.org/main/interviews/bios/conrad_lautenbacher_frt.asp, 2005.

Moore, G., Crossing the Chasm, Marketing and Selling High-Tech Products to Mainstream Customers, Harper Business, 1991, 211p.

O’Reilly, C.A. and Tushman, M.L., The Ambidextrous Organizations, Harvard Business Review, April 2004.

The Positive Deviance Initiative, http://positivedeviance.org/

Pascale, R.T. and J. Sternin, Your Company’s Secret Change Agents, Harvard Business Review, May 2005, p. 72-81.

Tushman, M.L., Anderson, P., and O’Reilly, C.A., Technology Cycles, Innovation Streams, and Ambidextrous Organizations: Organizational Renewal Through Innovation Streams and Strategic Change, in Managing Strategic Innovation and Change, Tushman and Anderson, eds., Oxford University Press, New York, 1997, 657p.

References