Embed Size (px)
DESCRIPTION
BC Spatial is an initiative of the Integrated Cadastral Information Society (ICIS) to establish integration services for and one-stop access to the best available cadastral data for the province. This presentation will demonstrate ICIS’ use of FME Server as the key technology at the center of the society’s data sharing framework. Data contributors will learn how ICIS is using FME to streamline data delivery, standardize data handling and provide quality assurance feedback to its members. Data consumers will learn how FME is powering data integration from many disparate sources, capturing data quality characteristics and reporting them in real time. FME developers will learn about the software customizations and scripting standards that are being deployed to achieve the greatest maintainability and flexibility of design. All attendees will hear how ICIS uses FME to facilitate collaborative success among its members – from all levels of government, utilities and the private sector – for the sharing of geospatial data.
Citation preview
The BC Spatial Project – Integrating BC’s Cadastre with FME Server
Paul van Nieuwkuyk, CartoSmart GIS BC Spatial Project Coordinator, ICIS
ICIS
ICIS’ Vision BC Spatial
BC Spatial
ICIS’ Vision BC Spatial
Core Challenges
Challenges Constraints Solutions
Source Datasets
• No uniform data model
• Unknown / anecdotal data quality
Delivery / Sharing
• Ad hoc submission habits
• Modest operational resources
Integrated Products
• Slow refresh cycle
• Manually intensive process
Challenges Constraints Solutions
Operational Constraints
Source Datasets
• Existing models reflect business need
• ICIS does not alter member data*
Delivery / Sharing
• Minimize impacts to data providers in sharing their data
Integrated Products
• Automate integration
• Assess and report: so that producers can enhance and users can decide
Challenges Constraints Solutions
Solutions
• Transparent and repeatable data processes • Eliminate the manual
• Deploy where needed • Available to a distributed community… online
• Sensible standards • Data standards based on integration needs • Common processing pattern for all datasets
Use the best ETL Tool!
“Data Integration Engine” (DIE and Do!)
LG
GeoBC
BCA Utilities
Integrate Assess Report
QC Metadata
Integrated Data
Statistics
1. Extract/Publish 2. Subscribe/Integrate 3. Notify/Distribute
Data Provider: Extract & Publish
Delivering with Python
Publication Flavors Web folder FTP site
Python shutdown script Automate zip and
ship to ICIS
Scheduled Task Delivery routine
ICIS: Scheduled Acquisition
Python Startup Acquire FTP or
web Unzip Archive
Standard processes Load (not
integrate) Capture metadata Log change
ICIS: Real-time reporting
shutdown.py Calls FME service Refreshes statistics KML data footprints as
index to current statistics
On-the-fly thematic rendering Color spectrum from
newer to older vintage
Coordinating Product Refresh
Nightly scheduled service Analyzes ETL Log (populated in acquisition script) Determines which integrated layers to refresh Invokes layer refresh scripts
Layer Integration
Changed sources are refreshed: DELETE / INSERT
Localized integration QC performed Jurisdictional
overlaps Correlation
reports: LTSA BCA
Stats Updated ETL log QC metadata
The Program
Activities Pilot conducted 2009 CRD Beta Jan – June 2010
All municipalities and regional district
GeoBC BC Assessment Utilities
Infrastructure FME Workbench & Server
2009 ArcGIS Server 9.3 on
SQL Server
Thank You!
Questions?
For more information: [email protected] www.bcspatial.ca
• www.bcspatial.ca Portal
• Project Charter • Pilot Program Report Docs
• www.bcspatial.ca/blog Chat
