Twenty Years of Spatial Vision, But What Does 1987 Look Like in Your GIS? Emerging Issues, Hindsight and Insights from the NC Preservation Partnership

Steve MorrisNCSU Libraries

Abby Smith Library of CongressNC GIS 2007March 2, 2007

National Digital Information and Infrastructure Preservation Program (NDIIPP)To ensure access over time to a rich body of digital content through the establishment of a national network of committed partners, collaborating in a digital preservation architecture with defined roles and responsibilities

NDIIPP ObjectivesDevelop a national digital collection and preservation strategy Build a network of partnershipsExplore protocols and standards to support partnership operationsIdentify and preserve at-risk digital contentSupport development of tools, models, and methods for digital preservation

Network of Preservation PartnersVolume and complexity of digital content calls for a distributed approachLC is providing resources and leadership to construct a network of preservation partnersPrimary outcomes for partnerships:Identify and preserve significant contentLeverage resources, experience via collaborative networkPromote standards and best practices

Goals for the PartnershipsShare strategies for digital content selection/collectionProbe intellectual property issuesCollaborate in developing a technical architectureStudy economics and incentivesIdentify and share best practicesLearn how to build and sustain partnerships

Geospatial Data Focus of PartnersNC State University and NC CGIAUC Santa Barbara and StanfordU of Tennessee KnoxvilleSan Diego Supercomputer Center, Scripps Institute of Oceanography, and Woods Hole Oceanographic Institute

U of Michigan: Social science dataU of Illinois at Urbana-Champaign: State government publications, among other content

Why Geospatial Data?Congressional interestGrowing importance for all aspects of:GovernmentBusinessScience & technologyCultural expressionSocial software

Why Geospatial Data Networks?Leverage existing effortsModel cooperation & coordination at different scalesPublic/private partnershipsRecognized need for expertise in tempero-spatial data collection, analysis, & long-term management

NC Geospatial Data Archiving Project (NCGDAP)Partnership between NCSU Libraries and NCCGIA with Library of Congress under NDIIPPOne of 8 NDIIPP Digital Preservation Partners projectsFocus on state and local geospatial content in North Carolina (state demonstration)Tied to NC OneMap initiative objective: Historic and temporal data will be maintained and available.Objective: engage existing state/federal geospatial data infrastructures in preservation

Temporal Data Supports Decision MakingLand use change analysisReal Estate trend analysisSite selection (past uses?)Forecasting

Parcel Boundary Changes 2001-2004North Raleigh, NC

Digital Preservation Points of FailureData is not saved, or cant be found, or media is obsolete, or media is corrupt, or format is obsolete, or file is corrupt, or meaning is lost

Solutions:

MigrationEmulationEncapsulationXML

Different Ways to Approach PreservationTechnical solutions: How do we archive acquired content over the long term?

Cultural/Organizational solutions: How do we make the data more preservableand more prone to be archivedfrom point of production?

A temporally-impaired industry begins to discover time and the value of older data

Major vendors and consulting firms begin to see temporal data management and analysis as a customer problem

Project Surprises:Emerging Industry Interest in Data Longevity

The true counterpart to the old map is not the GIS dataset but rather the finished geographic product (map, chart, etc.)

More than dataalso classification, layering, symbolization, annotation, modeling, more

Project Surprises:Handling PDF as a Geospatial Format

County and city agencies beginning to digitize old maps and aerial imagery

NCGDAP-georectified maps made available for download and put in the National Geologic Map Database

Project Surprises:Resurrecting Old MapsSuperceded USGS Topo MapsGeologic Maps from Theses, Dissertations, and Reports

Project Surprises:Engaging Standards EffortsPartnered with EDINA (UK) and NARA to approach the Open Geospatial Consortium (OGC) in 2005-2006Working Group charter approved by OGC Technical Committee plenary Dec. 2006

High volume of state/federal requests for local data spurs rethinking of archive strategy for data acquisition

Leveraging more compelling business reasons to put the data in motion

Changes in the Domain:Emerging ContentExchange NetworksOrthophotosneakernetsystem

Started fall 2006Transportation data exchange system

Funded starting fall 2006Ongoing statewide data inventory

Started March 2006

Huge new audience for geospatial content

Massive crossover of mainstream IT to geospatial, spurring open source activities: e.g. WMS tiling and caching

Good enough approaches to data (formats, quality, standards)

Changes in the Domain:Mashups, Google Earth,Map APIs, and More

Mobile, LBS, and, social networking applications

Long-term cultural heritage value in non-overhead imagery: more descriptive of place and function

Changes in the Domain:More Place-based (versus spatial) DataOblique ImageryStreet View ImagesDOT VideologsTax Dept. Photos

NC Frequency of Capture SurveySurvey objective:Document current practices for obtaining archival snapshots of county/municipal geospatial vector data layersSeek guidance about frequency of captureSurvey topics:General questions about data archiving practiceSpecific questions about parcels, street centerlines, jurisdictional boundaries, and zoningSurvey subjects:All 100 counties and 25 municipalities58% response rateSurvey conducted September 2006

Survey Results: OverviewTwo-thirds of responding agencies create and retain periodic snapshotsLong-term retention more common in counties with larger populationsStorage environments vary, with servers and CD-ROMs most commonOffsite storage (or both onsite and offsite) is used by nearly half of the respondentsPopularity of historic images has resulted in scanning and geo-referencing of hardcopy aerial photos among one-third of the respondents

Whats Next?TechnicalAcquiring and ingesting data Refining ingest systemExploring new metadata approachesExploring Neogeography spaceEngagementOGC Data Preservation Working GroupCollaboration with State ArchivesMore site visits

Questions?Steve MorrisDigital Library InitiativesNCSU LibrariesSteven_Morris@ncsu.eduNCGDAP: http://www.lib.ncsu.edu/ncgdap/

NDIIPP: http://www.digitalpreservation.gov/Abby SmithNDIIPPasmith@abbysmith.net