NSF Industry/University Cooperative Research Center High Performance Database Research Center...

NSF Industry/University Cooperative Research Center

High Performance Database Research CenterNaphtali D. Rishe, Director, rishe@fiu.eduHPDRC.FIU.edu TerraFly.com

HPDRC Expertise:HPDRC Expertise:

• Data visualization

• Spatial databases

• Internet-distributed heterogeneous databases

• Database design methodologies

• Information analysis

• GIS

• Location Data

• Health Informatics

Director: Dr. Naphtali David Rishe Director: Dr. Naphtali David Rishe

The Inaugural Outstanding University Professor of FIU

Awards: $40M, Patents: 4, Books: 4, Papers: 250

HPDRC.FIU.edu TerraFly.com

GIS Solutions Based On Geo-Spatial Research GIS Solutions Based On Geo-Spatial Research TechnologyTechnology• Next generation GIS: Internet geo-visualization & spatial

cloud-computing platform and service

• Advanced geo-spatial computing engine

• Open architecture, API provided

• 40 TB database of aerial imagery and spatial data

• Rich datasets in a user-friendly environment

• Professionally customizable to domain requirements

• NASA, NSF, IBM and USGS funded technology

HPDRC.FIU.edu TerraFly.com

Web-BasedWeb-BasedWeb-BasedWeb-Based

Open Architecture APIOpen Architecture APIOpen Architecture APIOpen Architecture API

Geo-Spatial Mapping SolutionGeo-Spatial Mapping SolutionGeo-Spatial Mapping SolutionGeo-Spatial Mapping Solution

HPDRC.FIU.edu TerraFly.com

Address LocatorAddress LocatorAddress LocatorAddress Locator

Vicinity InformationVicinity InformationVicinity InformationVicinity Information

Demographic DataDemographic DataDemographic DataDemographic Data

Nationwide Layers: 2000 US Census, Businesses, Schools, Travel, Features (airports, heliports, public buildings, churches, hospitals, libraries, post offices, towers, tunnels, water, etc.), Property Lines and much more

Nationwide Layers: 2000 US Census, Businesses, Schools, Travel, Features (airports, heliports, public buildings, churches, hospitals, libraries, post offices, towers, tunnels, water, etc.), Property Lines and much more

HPDRC.FIU.edu TerraFly.com

HPDRC.FIU.edu TerraFly.com

Your System Your System Your DataYour Data

TerraFly TerraFly systemsystem

TerraFly TerraFly IntegrationIntegration

Solution Solution

HPDRC.FIU.edu TerraFly.com

Query Date Range

Time Series

Animation Player

Census Block

Groups are Selectable

Layer Control

HPDRC.FIU.edu TerraFly.com

MapReduce use casesMapReduce use cases● Spatial data indexing [SSDBM’09]● Geospatial query support [SSDBM’10]● Parallel spatial data processing

[GrC’10]● Parallel set-similarity spatial joins● Real Estate data analytics

HPDRC.FIU.edu TerraFly.com

● Objective: Provide computational analytics for estimating how an event influences property values

● Dataset: Miami-Dade county records (~ 20M)● Community Boundaries: U.S. Census Blocks and Tracts

(polygons)● Property transaction geo-database: Join of property

public records (Deeds, Tax Roll)

HPDRC.FIU.edu TerraFly.com

Parcel

Date Value Area Type Location

...

1001 03-10-2005 100K 840 Condo 1,2

1002 03-15-2005 130K 2000 Single-Family 2,3

1003 12-22-2005 80K 1085 Single-Family 2,2

… … … … … …

● MethodMethod• Virtual Community (VC): a set of homes with similar

characteristics within a geographical area• Consider unit price per VC, e.g. median of $/sq-foot• Compute VC’s unit price rate change to compare communities• Hadoop MapReduce: Temporal, in parallel, self-join of the

dataset to compute property value rate change

Parcel Date Value … Loc.

1001 03-10-2005 100K 1,2

1002 03-15-2005 130K 2,3

1003 05-22-2005 80K 3,2

1004 06-20-2005 80K 4,2

1005 07-10-2006 100K 5,2

1006 03-15-2005 130K 6,3

1007 12-22-2005 80K 7,2

1008 12-22-2005 80K 8,2

… … … …

9000 03-15-2005 130K 9,3

9001 12-22-2005 80K 9,2

9002 01-22-2006 80K 2,2

9003 02-10-2006 100K 1,2

9004 03-15-2005 130K 2,3

9005 12-22-2005 80K 2,2

9006 12-22-2005 80K 9,2

Reduce

Map

Map

(Partitions records by: Community, Type, Date)

(Computes community values and change

rate)

…

Reduce

…

Community Rate Value …

300 -10% 100

301 -60% 80

302 15% 55

303 2% 60

… … …

Input: dataset, tstart, tend

Output: Community value change rate

Hurricane Andrew, August 1992, Q2’92 vs. Q4’92

Drop

Raise

No change

HPDRC.FIU.edu TerraFly.com