DEALING GEOGRAPHIC DATA

David Dye

DEALING GEOGRAPHIC DATA

• Introduction

• Working With Addresses

• ETL

• Presenting

INTRODUCTION

• David Dye

• derekman1@msn.com

• HTTP://WWW.SQLSAFETY.COM

WORKING WITH ADDRESSES

BACKGROUND• Within Public Safety many reports are geographically based and visual reports

are helpful

• Pin maps became the de facto standard of geographic maps

• Looks kewl, but no so easy to automatw

• Crime clusters

• Burglaries within a xx radius

• Focused patrol areas

• Traffic crashes with injuries

• YES we target problem areas NOT people

• Traffic crashes without injuries

• Granularity based on the reason for the first harmful event

• Did He do that too???

• Locate similar crimes in the area where suspects are arrested

• Car hoppers!!!

• AKA job security!!

DEALING WITH LOCATION

• What location information do you deal with?

• Country

• State/Providence

• Zip code

• City

• Address

• Block range

• What do you need to report on

• Granularity!!!

• How does it have to be reported?

• Aggregations

• Non-map based

• Visual representation

DEFINE REPORT FORMAT

• Table, matrix, list

• Visual

PRESENTING TO POLICE

DEMONSTRATION

Tabular report (BasicAddressReport.rdl)

Data set review

SSRS report expression

Map control report (MapReport.rdl)

Map Control Wizard

HOW IS LOCATION RECEIVED/STORED

• How do you receive and store your date

• Fully parsed address

• Full address

• Lon and Lat

• SQL Spatial

• The higher the level of receipt/storage the more analysis and

cleansing

• TAKE THE TIME TO DO EXHAUSTIVE EVALUATION

• Look for the rules and then find the most common exceptions

• Generally the more granular the storage/receipt the easier it is to

validate and piece together

WHAT IS BEING REPORTED?

• Review how the geographic data is being reported

• Granularity

• Zip

• Block ranges

• Latitude and longitude/Point

• Strive to drill down to the lowest level

• Does it require visualization (SSRS Map Component)

• “14 Crashes in the 100 Block North Del Prado”

CREATE YOUR DIMENSION

• Based on the reporting needs will drive the schema of the

dimension table

CREATE TABLE [dbo].[DimGeography](

[geoid] [int] IDENTITY(1,1) NOT NULL,

[commitedkey] [int] NULL,

[inci_id] [varchar](14) NULL,

[block] [int] NULL…………

• Review DDL Example

DEMONSTRATION

DDL statement (LocationQueries.sql)

Discuss requirements

Crossrd(s)

Block range critical!!

geox

geoy

ETL

EVALUATE DATA SOURCES

• Where is geographic data being stored

• SQL

• Excel (SORRY BRO!!)

• ESRI

• CSV

• Fox Pro (REALLY!! Why not Aramaic?)

• Can separate sources be maintained

• Based on the ability to present as a whole and consistency and

accuracy

• Is location parsed?

• If not evaluate the need/ability to parse during the

transformation

• Eases presenting and cleansing

IMPORTING

• If possible parse the data to its most granular segments

• Street number

• Street

• Street prefix

• Street Suffix

• City

• Zip

• State

• Clean it up!!

• Find most common inconsistencies

• Str, S, or Street should all be “St”

• Del Prdo, D.P., Dele Prado should all be “Del Prado”

• REVIEW DEMO T-SQL CODE

TRANSFORMATIONS

• Based on the source data not all data sets may be present

• Block range

• VIEW DEMO CODE

• Longitude/Latitude

• Cross street(s)

• Address accuracy is critical at this point!!

• Accurate and consistent!

HOW DO I GET LAT/LON?

• You can manually enter each address

• http://itouchmap.com/latlong.html

• http://stevemorse.org/jcal/latlon.php

• C’Mon Man!! Really?!?!

• Automate!!

• Google Maps API

• https://developers.google.com/maps/documentation/geocoding/#geocoding

• Limits usage

• Free 2,500 within 24 hours

• Business 100,000 within 24 hours

• Will also begin to throttle based on requests submitted within a minute threshold

• MelissaData

• Native SSIS transformation

• Provides extensible transformation

• US and International

• DEMO

DEMONSTRATION

SQL Cleansing (LocationQueries.sql)

Discuss requirements

Crossrd(s)

Block range critical!!

geox

geoy

KEWL, BUT NOW WHAT?!?

• I got lat and lon!! Now what?!?!

• Power Pivot/ Power Map Demo

• Address only

• THIS IS WHY WE NEED LAT LON

• Lat lon in Power Map

• We can do better!!

• SQL Server spatial data type supports

• Native functions

• SSRS map control

• Provides dynamic visualization of geographic data

DEMONSTRATION

Power Map using address

OUCH!!

Power Map using Lat/Lon

Getting there!

SQL SPATIAL PRIMER

• Introduced in SQL 2008

• Based on CLR user defined type

• System provided

• Does not require enabling CLR

• Geography

• Supports methods

• Open Geospatial Consortium (OGC) standard

• Set of Microsoft extensions to that standard

• Geometry

• Supports methods

• Open Geospatial Consortium (OGC) standard

• Set of Microsoft extensions to that standard

DEMONSTRATION

Geometry (LocationQueries.sql)

Geography(LocationQueries.sql)

WHY USE SQL SPATIAL

• It’s KEWL

• Chicks dig it

WHY USE SPATIAL

• Provides the ability create SSRS reports using the Map Control

• Better than address and latitude and longitude

• More exact

• Spatial functions can be applied

• All points within the radius of a point

• Once longitude and latitude is obtained we can convert this to

geography data type

• Geography column can be set using an UPDATE statement

• DEMO

DEMONSTRATION

Update spatial column

Spatial methods

STDistance

SSRS MAP COMPONENT

• Wizards ROCK!!

• Report builder 3.0 Map Report

• SSDT

• More control!

DEMONSTRATION

Map Builder

Wizard

SSDT

Basic

Radius

Parameterized