Embed Size (px)
Citation preview
Geoapplications development
http://rgeo.wikience.org
Higher School of Economics, Moscow, www.cs.hse.ru
Agenda2
• What is geospatial data?
• How is it different from other types of data?
• Examples of geospatial data
• What is geoapplication?
• Course goals & organization
• Real-world examples: companies using geospatial data
• Tentative projects to participate in
• Main readings
• Quiz
• Practical lesson: 3D Globes
Geospatial data: 2 types (simplified)3
VECTOR
What is raster data?http://help.arcgis.com/EN/arcgisdesktop/10.0/help/index.html#//009t00000002000000
RASTER
matrix
The main difference from other data types (simplified)4
RASTER
matrix
A fundamental difference: the presence of spatial coordinates
each cell has (lat, lon, value)(lat_i, lon_i) for
each vertice
VECTOR
Is that all?5
is there something strange here??6
Hint: areas…
is Greenland larger than Africa??7
Let us consult an Encyclopedia or Wikipedia
2,175,600 km2
World size ranking: 1430,370,000 km2
World size ranking: 2
is Greenland larger than Africa??8
https://takeitscienceblog.wordpress.com/2016/08/31/how-big-is-your-country/http://geoffboeing.com/2015/08/map-projections-that-lie/
Distances9
Two airplanes are flying …
Plane A: (60ᵒN, 40ᵒE) → (60ᵒN, 30ᵒE)
Plane B: (40ᵒN, 40ᵒE) → (40ᵒN, 30ᵒE)
40ᵒ - 30ᵒ = 10ᵒ
Both traveled the same distance?
Everything is not so simple…10
To name a few:* distances between two points with the same longitudes are different for different latitudes
A small caveat: circular/periodic axes11
longitude range is [ –180 … +180 ]
but what if I want to move 10 degrees eastward from 179 degrees?179 + 10 = 189 …. actually -171
Lesson learned12
Areas, shapes, angles, distances depend on…
Examples of geospatial data: vector13
Geo prefix14
→
→
→
→
→
→
Examples of geospatial data: raster & vector15
More details are at:http://www.wikience.org/tropical-cyclones/wind-swaths/
Examples of geospatial data: satellite raster16
Visualization examples17
Much more examples are at: http://goworldwind.org/demos/
Video on terrainAnalytic Surface
AirspacesExtruded ShapesScankort Denmark Data
We can do more together during course work
NASA World Wind engine
18
Satellites,
smartphones, GPS
(1): basic theory: geographic coordinate systems, geocoding, vector and raster data types and operations (3): I/O with vector (SHP, KML, GeoJSON, WKT) and raster (GeoTIFF, NetCDF, HDF) formats (4): network protocols for geodata exchange: WCS, WFS, WMS, OPeNDAP(4): frameworks: Mina, Netty that is used by Twitter(3,4,5): spatial databases and servers: PostGIS, SciDB, GeoServer, TDS(5): software libraries for geodata processing: vector Java Topology Suite and raster ImageMagic, GDAL(6): visualizing geodata with NASA WorldWind, Mapbox, Leaflet, CartoDB
Course goals & organization
Topics cover technologies for a complete geospatial data management stack:
19Course assessment
Code Description Contribution, %
LP Lecture presence 20
PP Work during practical lessons 5
CW Control work 10
HA1 Home assignment 1 30
HA2 Home assignment 2 15
EX Exam 20
TT Total contribution
Geospatial technology stack (not exhaustive)20
Storage
Transmission
Processing
Visualization
DBMS
Files
Almost all DBMS are geo-enabled
SHP, KML, GeoJSON, GeoTIFF, NetCDF, HDF
network protocols WCS, WFS, WMS, OPeNDAP
Libraries in Java, Python, R, JavaScript, …
Geospatial theory stack (not exhaustive)21
Notations
Topology
Processing
GPS, WiFi, geohash, …
Projections
ISO/IEC standard
The technologies and theory are building blocks for solving real-world tasks
22
The technologies and theory are building blocks for solving real-world tasks
23
Example task:Because of dramatic traffic increase, the town planners want to widen the main road and add a second lane. The lane width is 20 meters. Find properties that must be demolished.
main road
new lane(its central line)
house
tree
The solution (DBMS, WKT, Buffer, Topology, SQL, …)24
LINESTRING (290 230, 330 250, 390 260, 440 260, 495 252, 535 238, 560 220, 580 190, 590 150)
main road
house
tree
new lane(its central line)
DBMSSELECT a_geom, address, ownerFROM city_databaseWHERE ST_Intersects (
ST_Buffer(ST_GeomFromText("LINESTRING (290 230,…) ",4326),10),
a_geom)
POLYGON ((325.5278640450004 258.94427190999915, 326.90755828109235, ….))
BUFFER (ST_Buffer)
WKT: ISO/IEC standard
Consumers of geospatial data
25
Who needs geospatial data?How do they process them?What kinds of geospatial data exist?
Consumers of geospatial data: motivating examples26
http://www.spatial.cs.umn.edu/Book/sdb-chap1.pdf
Consumers of geospatial data: motivating examples27
http://www.spatial.cs.umn.edu/Book/sdb-chap1.pdf
Consumers of geospatial data: motivating examples28
http://www.spatial.cs.umn.edu/Book/sdb-chap1.pdf
Consumers of geospatial data: motivating examples29
http://www.spatial.cs.umn.edu/Book/sdb-chap1.pdf
Consumers of geospatial data: motivating examples30
http://www.spatial.cs.umn.edu/Book/sdb-chap1.pdf
Geoapps: Marine Traffic31Real-time ship tracking
http://www.marinetraffic.com
http://shipfinder.co/https://www.vesselfinder.com/
EigenRisk33
Hurricane swaths, flood footprints, assets
Spatial operations
Visualization
Hurricane
Katrina, 2005
Zillow34
Selling real
estates
Filter, nearby,
etc.
Examples of geoapplications – 2GIS35
Places
Positioning
Routes
….
Yelp36
Seek restaurants, hairdressing saloons, etc. + add other
UrtheCast37
Live video streaming from International Space Station
A platform for imagery processing
Sputnix, Dauria – Skolkovo residents38
Native satellite manufacturers –
ensure Russian national interests
Д.К. Мозговой и О.В. Кравец «Использование данных MODIS для экологического мониторинга и контроля чрезвычайных ситуаций»
Blooming in Каховском водохранилище
Precision agriculture
UAV: Unmanned Aerial Vehicles
http://www.youtube.com/watch?v=4ErEBkj_3PY
Consumer:40
Optimize wind farms locations and operation
Data: wind speed and direction reanalysis and forecasts – raster data
For calendar year 2014, the electricity produced from wind power in the United States amounted to 181.79 terawatt-hours, or 4.44% of all generated electrical energy. https://en.wikipedia.org/wiki/Wind_power_in_the_United_States
Wind speed data41
Energy sector, hazard monitoring, etc.
https://earth.nullschool.net
http://wikience.org
Thermal power stations42
Optimize load depending on climate/weather forecast
-> reduce penalties
Air temperature is
one of the crucial
factors upon
which the station
load depends
Railroads43
Qingyun (Jeffrey) Xie, High Performance Raster Database Manipulation and Data Processing with Oracle Spatial and Graph
More examples44
Our Projects45
ChronosDB – a new DBMS
Michael Stonebraker
PostgreSQL, Vertica, Streambase, Illustra, VoltDB,
IEEE John von Neumann Medal (2005)ACM Turing Award (2014)
ChronosDB outperforms SciDB by up to 75× on average
R.A. Rodriges Zalipynis. ChronosDB: Distributed, File Based, Geospatial Array DBMS
Rio de Janeiro, Brazil27 – 31 August 2018
VLDB (Very Large Data Bases) –a top conference on
Big Data, Distributed Systems, Databases
2018: the only talk from Russia10 years ago: the previous talk
from Russia
SciDB
University of California, Berkeley,
University of Michigan,
Massachusetts Institute of Technology
https://db-engines.com/en/ranking
Moscow & Rio 46
Corcovado47
The photo is not mine
www.climate.wikience.org
Tentative course work topics
JavaScript: Web GIS (Geographical Information Service)Data compression, processing, visualization, user interaction
Performance evaluation
Tentative course work topics
performance evaluation
We will develop distributed & multithreaded application to simulate a large load
DASKGeoTrellisRasDaManSciDB….
Tentative course work topics: machine learninghttps://habrahabr.ru/company/avito/blog/325632/https://habrahabr.ru/company/ods/blog/325096/
