Upload
isaac-collins
View
217
Download
0
Tags:
Embed Size (px)
Citation preview
www.ordnancesurvey.co.uk
Overview
• PAI Impact on Pembrokeshire• Planning for the change• Data Quality• Migration• What’s next
www.ordnancesurvey.co.uk
Pembrokeshire County Council
• Rural unitary authority• Area 1614 Sq km• Pembroke Dock and Milford Haven on Urban tiles•Remainder rural•GIS development driven by a virtual team of 4 staff•MapInfo Professional users since v4
www.ordnancesurvey.co.uk
Impact on Pembrokeshire
• 3 areas supplied to date• Haverfordwest scheduled within next three months• More areas now in the 3-9 month window • Whole county scheduled for supply by end of ‘03-4
financial year• Perceived as a barrier to implementing OS MasterMap if
a means of implementing the data could not be found.
www.ordnancesurvey.co.uk
Understanding PAI
• Part of West Wales group of local councils• Took lead within group on
• OS MasterMap migration planning• PAI workflow development
• Learnt from others in the group on• Data audits• Use of OS MasterMap in PAI
• Established core team to coordinate programme across authority
www.ordnancesurvey.co.uk
Planning for Change (1) : Data Audit• Audit Questionnaire
• Based on Ordnance Survey draft used in Ceredigion
• Audit database• Basic MS Access database• All users required to complete records for data they managed on
behalf of authority
• Data analysis• Sift based on audit responses• Identified less than 20 tables of data that met criteria in matrix to
be worthwhile migrating
www.ordnancesurvey.co.uk
Planning for Change (2) : Data Quality Review•Visual inspection of data considered for migration•Results
• Data was a modern interpretation of an historic document
• Data capture techniques were generally poor
• Data generally followed rather than was snapped to Ordnance Survey lines
•Uses• Visual inspection only• Reference to original source if
ambiguity existed•Conclusions
• Existing data was fit for purpose
www.ordnancesurvey.co.uk
Conclusions of Audit Phase
• Data was fit for current purpose of visual inspection• BUT
• Overall quality was low due to a combination of• Limited quality control on capture
• Limited techniques available during capture of oldest data
• Data captured to meet specific purpose rather than for wider use
• THEREFORE • Data was not fit for future purposes
www.ordnancesurvey.co.uk
Migration : Aims
• Council data was perceived to have a low quality value• Resources needed to be focussed on quality
improvement not PAI migration• Speed and ease of migration were considered important• Cost minimisation was high priority• Ambiguity caused by PAI should be removed to ensure
data remained fit for use on current purposes• Quality improvement was a separate related process
www.ordnancesurvey.co.uk
Migration : Technical solutions
• Council purchased single licence of Shiftwiz from Dotted Eyes for PAI migration• Low cost (ca £2500 for licence)• Used existing technology• Provided means to migrate all data using same parameters• Batch process• Able to handle large volumes of PAI link files• Perceived fast processing of data
www.ordnancesurvey.co.uk
Migration : Workflow overview
Based on “Generic PAI data management workflow” joint document prepared with Ordnance Survey
•Stage 1 : Prepare feature level metadata•Stage 2 : Create PAI status table •Stage 3 : Freeze data and extract affected information•Stage 4 : PAI migration•Stage 5 : Reintegrate with original dataset
www.ordnancesurvey.co.uk
Migration : Workflow (1)•Stage 1 : Feature level metadata
• New column to denote status of data created
• All objects populated with appropriate current status flag
Tag Description
PR Object captured on rural tiles in an unimproved position in relation to PAI or quality improvement
PU Object captured on urban tiles prior to quality improvement
PM Object captured on Mountain & Moorland tiles prior to quality improvement
MR Object on a rural tile that has been migrated to a post PAI position but not yet improved in terms of data quality
QR Object on a rural tile that has been captured at high quality but will be subject to subsequent PAI revision
Q Data on any tile that has been captured at the defined quality level for the dataset or has been migrated and quality improved to the appropriate quality level
•Stage 2 : Create PAI status table• DMU based table showing the
internal PAI status of each map unit• All tiles colour coded for
• No PAI impact• PAI received• PAI implemented• PAI pending (with date if known)
www.ordnancesurvey.co.uk
Migration : Workflow (2)
• Stage 3 Freeze and extraction• Users notified of area undergoing processing• Data extracted from main file and stored in separate location
• Stage 4 PAI migration• Shiftwiz configured once to determine optimum settings in PAI block• Program used as “black box” solution for all data• Output files checked for anomalies
• Stage 5 : Reintegration• Status flag changed to post PAI unimproved• Data incorporated back into main file• Pre PAI data stored in archive• DMU status changed• Quality improvement commenced
www.ordnancesurvey.co.uk
What’s Next
• Data quality improvement process• Scheduled at a feature level using Dotted Eyes CaptureWiz• New processes in discussion for data capture with OS
MasterMap
• OS MasterMap deployment• Moved forward from Jan to end July to facilitate above program• Unlikely to be affected by PAI due to speed of process
• Link file triggers new download
• Migration undertaken while order is processed
• OS MasterMap and PAI data updated at same time
www.ordnancesurvey.co.uk
Testing the Theory (1)
• 2 day project in Pembrokeshire HQ• Test area northern PAI block on Ceredigion boundary.
• 466 link files
• Tested on NT4 workstation and Win 2000 laptop• 17 datasets • Approximately 1750 records not all in PAI area
www.ordnancesurvey.co.uk
Issues to watch out for : Software
• Including a $ sign in the file-path causes data load to fail• $ is a reserved character in SQL, Pembrokeshire use the symbol
to denote a server on the network but not visible
• Check your virtual memory settings• System uses large amounts of virtual memory to page
information in and out. Especially on NT the default settings will be too low.
• Virus checkers eat memory and should be disabled if there are limited amounts of physical memory
www.ordnancesurvey.co.uk
Results
•On the laptop •1.45 million links processed in 17 minutes to create both current matrix and stored data for future use•User data processed in 90 minutes•Visual inspection confirmed authority were happy that data could still be used for visual inspection
Pre
Post
www.ordnancesurvey.co.uk
Is the Model Generic?
• Applicable• Any dataset that has not been snapped to Land-Line features• Any data used for visual inspection were paper mapping is
regarded as definitive document• Data where a subsequent quality improvement is planned• Applications where maintaining the most current mapbase
overrides absolute replication of data
• Not Applicable• Data replicated from Land-Line where quality is vital to be
maintained• Data used in fully automated systems until full QA is completed
www.ordnancesurvey.co.uk
Conclusions
• Pembrokeshire have a model workflow they will use for PAI migration
• Pembrokeshire plan to produce a companion document for our generic workflow detaillign their experiences of working with PAI
• Their proof of concept provides us with a means of defusing some of the concerns about the time impact of PAI in the local authority sector.
www.ordnancesurvey.co.uk
And Finally
“Provided you do your areas as they come in it is quite easy and simplistic”
Contact for further information
Customer Contact CentreOrdnance Survey
Romsey RoadSOUTHAMPTONUnited Kingdom
SO16 4GU
Phone: 08456 05 05 05Fax: 023 8079 2615
Email: [email protected] site: www.ordnancesurvey.co.uk
www.ordnancesurvey.co.uk