14
Postal address clean-up Andrew Zolnai, Richard Mosley

Fmewt 2017 zolnai mosley

Embed Size (px)

Citation preview

Page 1: Fmewt 2017 zolnai mosley

Postal address

clean-up

Andrew Zolnai, Richard Mosley

Page 2: Fmewt 2017 zolnai mosley

Andrew Zolnai

Support Specialist, Lead

Development at Safe Software

helped with this.

Richard Mosley

Geologist turned to GIS for over 30 years, covered all aspects of geo-data, project and business management in petroleum and volunteered geography.

Page 3: Fmewt 2017 zolnai mosley

Two non-spatial Workbench uses:

1) postal address clean-up

2) & electoral list updates

Page 4: Fmewt 2017 zolnai mosley

Help French 2017 presidential / parliamentary election campaign

Using nationbuilder.com for a London UK based campaign

Liste ElectoraleConsulaire is structured but inconsistent

Ergo normalise 4 address columns into a schema

The premise

Page 5: Fmewt 2017 zolnai mosley

Regex on Steroids

1Spatial FME WT 2015:

scrape 4 years worth

of playlists and tracks

off the StrayFM

website

and categorise and

rank the most played

artists and tracks

Inspired usage here:

StringSearcher

search address

components

AttributeSplitter split

them into similar parts

AttributeManager re-

order into one schema

Page 6: Fmewt 2017 zolnai mosley

Address normalization

Page 7: Fmewt 2017 zolnai mosley

Clean

Rinse

Repeat

Get the first matches of address strings in the 4 address fields

If string is empty then assign the next address string to it

Country name is constant last string

Build normalised string sets backward from it

01

02

04

03

Page 8: Fmewt 2017 zolnai mosley

Address schema ready to upload

Page 9: Fmewt 2017 zolnai mosley

But that’s not all…

What about on-going updates to

Electoral Lists?

Page 10: Fmewt 2017 zolnai mosley

FeatureMerger non-spatial too!Initial load:

find rejected addresses

repeat the procedure if possible

Ongoing updates:

find the new entries as updated lists are received

repeat the procedure on the “delta” only

Page 11: Fmewt 2017 zolnai mosley

Refresh mega-data lists

Page 12: Fmewt 2017 zolnai mosley

Place your screenshot here

Electoral lists for campaigns

Page 13: Fmewt 2017 zolnai mosley

Metadata is king

Data never come clean

Create schemas is key

Automate? Web service?

Tame

mega

lists

Page 14: Fmewt 2017 zolnai mosley

Thank you!

www.zolnai.ca

[email protected]