Embed Size (px)
DESCRIPTION
Utah, the greatest SMOG on earth. Harvesting data for air quality prediction. Presentation walks through simple data sources, data sources that required javascript packet gathering and scraping. Finally data sources that require reverse map to data conversions.
Citation preview
Ben Taylor @bentaylordata
Greatest snow on earthSMOG
Finding data sources
HIREVUE | TALENT INTERACTION
Intel/Micron/Quant/HireVue
@BENTAYLORDATA
HIREVUE | TALENT INTERACTION @BENTAYLORDATA
? Take a look and get inspired
HIREVUE | TALENT INTERACTION
WeatherPhysicalCultural
TrafficSocialHealthIndustryPolitical
Other
@BENTAYLORDATA
Data Categories
HIREVUE | TALENT INTERACTION
Searching for datacan get messy
@BENTAYLORDATA
HIREVUE | TALENT INTERACTION
Vertical Sounding http://weather.uwyo.edu/
@BENTAYLORDATA
HIREVUE | TALENT INTERACTION
Cultural / Physicalhttp://www.naturalearthdata.com/downloads/
@BENTAYLORDATA
HIREVUE | TALENT INTERACTION
All roads
@BENTAYLORDATA
HIREVUE | TALENT INTERACTION
All roads
@BENTAYLORDATA
HIREVUE | TALENT INTERACTION
All roads
@BENTAYLORDATA
HIREVUE | TALENT INTERACTION
All roads* density
@BENTAYLORDATA
HIREVUE | TALENT INTERACTION
All roads* length
HIREVUE | TALENT INTERACTION
All roads* major
@BENTAYLORDATA
River Distance*Source: http://www.naturalearthdata.com/downloads/10m-cultural-vectors/
Calculate shortest distance
@BENTAYLORDATA
HIREVUE | TALENT INTERACTION @BENTAYLORDATA
HIREVUE | TALENT INTERACTION @BENTAYLORDATA
No API? No problem!
HIREVUE | TALENT INTERACTION
Weather station data (demo)http://mesowest.utah.edu/
@BENTAYLORDATA
HIREVUE | TALENT INTERACTION
Weather station data (demo)http://mesowest.utah.edu/
@BENTAYLORDATA
HIREVUE | TALENT INTERACTION
Weather station data (demo)http://mesowest.utah.edu/
@BENTAYLORDATA
HIREVUE | TALENT INTERACTION
Weather station data (demo)http://mesowest.utah.edu/
@BENTAYLORDATA
HIREVUE | TALENT INTERACTION
Weather station data (demo)http://mesowest.utah.edu/
@BENTAYLORDATA
http://mesowest.utah.edu/cgi-bin/droman/mm_get_monitor_data.pl?state=UT&rawsflag=290&type=&orderby=n&day1=&month1=&year1=&time1=&timeobs=1&unit=&tzoffset=360&lat=39.78321267821705&lon=-111.73095703125&radius=&product=&availoffset=3&checked=0&past=0&display=js
HIREVUE | TALENT INTERACTION
Weather station data (demo)http://mesowest.utah.edu/
@BENTAYLORDATA
HIREVUE | TALENT INTERACTION
Proper Interpolation methods Spatial interpolation (region “state”)
@BENTAYLORDATA
Nearest neighborBicubicBootstrap EWMAPhysical Model
HIREVUE | TALENT INTERACTION @BENTAYLORDATA
No API? No problem!
Unlocking closed source data
@BENTAYLORDATA
Annual wind data example
?
Coordinate system? Colors? Known lat/long + values
@BENTAYLORDATA
Import nearest colormatch
@BENTAYLORDATA
Removes state lines/text
@BENTAYLORDATA
@BENTAYLORDATA
Sanity Check
Original Map Transformed Map
@BENTAYLORDATA
HIREVUE | TALENT INTERACTION
Satellite Feedhttp://www.ssd.noaa.gov/http://www.firedetect.noaa.gov/viewer.htmhttp://www.ssd.noaa.gov/goes/west/wfo/slc.html
@BENTAYLORDATA
HIREVUE | TALENT INTERACTION
Feedback/connect
@BEN TAYLOR DATA
