Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
GeoCouch:
A spatial index for CouchDB
Volker Mische
Free and Open Source Software for GeospatialSeptember 6–9th 2010
Barcelona
Volker Mische FOSS4G 2010 Barcelona 1 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
What is GeoCouch?
A spatial index for CouchDB
Volker Mische FOSS4G 2010 Barcelona 2 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
?Volker Mische FOSS4G 2010 Barcelona 3 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
NoSQL
Volker Mische FOSS4G 2010 Barcelona 4 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
Documents
{"_id": "Station-2942_001","_rev": "3-77f17a55f6ab11f7f6668e63a75f2281",
"name": "Station-2942","date": "2010-09-09","loc": [2.183333, 41.383333],"country": "Spain","temperature": 25,"rainfall": 3
}
Volker Mische FOSS4G 2010 Barcelona 5 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
RESTful HTTP API
The four basic functions of a database (CRUD):
Create HTTP PUT /db/Station-2942_001
Read HTTP GET /db/Station-2942_001
Update HTTP PUT /db/Station-2942_001
Delete HTTP DELETE /db/Station-2942_001
Volker Mische FOSS4G 2010 Barcelona 6 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
Fancy Features
Volker Mische FOSS4G 2010 Barcelona 7 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
Schema free
Volker Mische FOSS4G 2010 Barcelona 8 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
Arbitrary input/output
JSON + JavaScript=
anything
Volker Mische FOSS4G 2010 Barcelona 9 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
Concurrency/Robustness
No lock on reads (MVCC)
Append-only
Durability (crash-only design)
Volker Mische FOSS4G 2010 Barcelona 10 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
Replication
Volker Mische FOSS4G 2010 Barcelona 11 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
Replication
Volker Mische FOSS4G 2010 Barcelona 11 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
Replication
Volker Mische FOSS4G 2010 Barcelona 11 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
Replication
Volker Mische FOSS4G 2010 Barcelona 11 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
Replication
Volker Mische FOSS4G 2010 Barcelona 11 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
Replication
Volker Mische FOSS4G 2010 Barcelona 11 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
Replication
Volker Mische FOSS4G 2010 Barcelona 11 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
GeoCouch
Volker Mische FOSS4G 2010 Barcelona 12 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
Previous version
CouchDB Python
SpatiaLite
HTTP Request stdin/out
HTTP Response
GeoCouch
Volker Mische FOSS4G 2010 Barcelona 13 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
Current version
CouchDBHTTP Request
HTTP Response GeoCouch
Volker Mische FOSS4G 2010 Barcelona 14 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
GeoCouch Features
Volker Mische FOSS4G 2010 Barcelona 15 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
GeometryCollection
Polygon
MultiPolygonMultiLineString
MultiPoint
Point
LineString
XXX OpenSeaarchVolker Mische FOSS4G 2010 Barcelona 16 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
Barcelona
La Barceloneta
Palau Reial
Pius XII
Maria Cristina
Avinguda de Xile
Zona Universitària
Ernest Lluch
Prat de la Riba
Ildefons Cerdà
Europa-Fira
Fira
Parc Logistic
Zona Franca Port
Foc Cisell
Foneria
Bellvitge
Can Rigal
Campus Nord
Campus Sud
Manel Girona
Provençana
Torrassa
Gornal
Collblanc
Marina
Zona Franca Litoral
Motors
Plaça del Centre
Plaça de Sants
L'Illa
Mercat Nou
Les Corts
Numancia
Santa Eulàlia
Badal
Bellvitge
Sant Josep
Pubilla Cases
Florida
Rocafort
Espanya
Hospital Clínic
Entença
Tarragona
Francesc Macià
Passeig de Gràcia
Sant Antoni
Universitat
Barcelona - Passeig de Gràcia
Poble Sec
Urgell
Magòria-La Campana
Sants Estació
Tetuan
Drassanes
Liceu
Barcelona - Plaça Catalunya
Paral·lel
Girona
Barcelona - Arc de Triomf
Ciutadella - Vila Olímpica
Bogatell
Wellington
Arc de Triomf
Hostafrancs
Barceloneta
Urquinaona
Jaume I Estació de França
senalataC stroC sel ed aiV narG
airàcI 'd adinevA
Passeig de Gràcia
anat
eiaL
a
iV
lanogaiD .va
àirraS ed adugnivA
B-10
Carrer d
e Ber
lin
Gran Via
de les C
orts
Cat
alan
es
Pass
eig
de C
olom
B-10
Carrer N
umancia
amoR ed adugnivA
giM l
ed
ad
no
R
Ronda d
el M
ig
Trav
esse
ra de le
s Cor
ts
Avingu
da de Madrid
giM led adnoR
Carrer de Sants
alaz
irA'
d rer
raC
stroC sel ed aressevarT
tlaD ed adno
R
B-20
B-20
ógarA
B-20
redaugiA rotcoD led rerraC
lanogaiD .va
av. Diagonal
lanogaiD adugnivA
Avinguda Diagonal
Avinguda del Paral·lellel·laraP led adugnivA
L'Hospitaletde Llobregat
Antiguocircuito
de Montjuic
TERCAT
Cementiride Montjuïc
Parcdel migdia
Parcde La
Ciutadella
JardíBotànic
de BarcelonaVolker Mische FOSS4G 2010 Barcelona 17 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
Barcelona
La Barceloneta
Palau Reial
Pius XII
Maria Cristina
Avinguda de Xile
Zona Universitària
Ernest Lluch
Prat de la Riba
Ildefons Cerdà
Europa-Fira
Fira
Parc Logistic
Zona Franca Port
Foc Cisell
Foneria
Bellvitge
Can Rigal
Campus Nord
Campus Sud
Manel Girona
Provençana
Torrassa
Gornal
Collblanc
Marina
Zona Franca Litoral
Motors
Plaça del Centre
Plaça de Sants
L'Illa
Mercat Nou
Les Corts
Numancia
Santa Eulàlia
Badal
Bellvitge
Sant Josep
Pubilla Cases
Florida
Rocafort
Espanya
Hospital Clínic
Entença
Tarragona
Francesc Macià
Passeig de Gràcia
Sant Antoni
Universitat
Barcelona - Passeig de Gràcia
Poble Sec
Urgell
Magòria-La Campana
Sants Estació
Tetuan
Drassanes
Liceu
Barcelona - Plaça Catalunya
Paral·lel
Girona
Barcelona - Arc de Triomf
Ciutadella - Vila Olímpica
Bogatell
Wellington
Arc de Triomf
Hostafrancs
Barceloneta
Urquinaona
Jaume I Estació de França
senalataC stroC sel ed aiV narG
airàcI 'd adinevA
Passeig de Gràcia
anat
eiaL
a
iV
lanogaiD .va
àirraS ed adugnivA
B-10
Carrer d
e Ber
lin
Gran Via
de les C
orts
Cat
alan
es
Pass
eig
de C
olom
B-10
Carrer N
umancia
amoR ed adugnivA
giM l
ed
ad
no
R
Ronda d
el M
ig
Trav
esse
ra de le
s Cor
ts
Avingu
da de Madrid
giM led adnoR
Carrer de Sants
alaz
irA'
d rer
raC
stroC sel ed aressevarT
tlaD ed adno
R
B-20
B-20
ógarA
B-20
redaugiA rotcoD led rerraC
lanogaiD .va
av. Diagonal
lanogaiD adugnivA
Avinguda Diagonal
Avinguda del Paral·lellel·laraP led adugnivA
L'Hospitaletde Llobregat
Antiguocircuito
de Montjuic
TERCAT
Cementiride Montjuïc
Parcdel migdia
Parcde La
Ciutadella
JardíBotànic
de BarcelonaVolker Mische FOSS4G 2010 Barcelona 17 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
Count
Volker Mische FOSS4G 2010 Barcelona 18 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
List function(arbitrary output)
Volker Mische FOSS4G 2010 Barcelona 19 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
R-Tree vs. Geohash
R-Tree Geohash
Data Structure R-Tree B-Tree
Approximation bounding box point
BBox precision exact approximate
Good for all geometries points
Used by PostGIS,SpatiaLite, Oracle
Spatial
MongoDB
Volker Mische FOSS4G 2010 Barcelona 20 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
Benchmarks
Volker Mische FOSS4G 2010 Barcelona 21 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
Benchmarks: Competitors
PostGIS 1.4.0
SpatiaLite 2.4.0RC2
GeoCouch tag: diplomathesis
Volker Mische FOSS4G 2010 Barcelona 22 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
Benchmarks: Datasets
100 000 points
USGS GNIS
350 000 polygons
TIGER
Volker Mische FOSS4G 2010 Barcelona 23 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
Benchmarks: Test 1
50%Volker Mische FOSS4G 2010 Barcelona 24 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
50 000 75 000 100 000 125 000 150 000 175 000Number of returned polygons
0
500
1000
1500
Tim
e in
mill
isec
onds
Bounding box query returning approx. 50% of the polygons
PostGIS 1.4.0SpatiaLite 2.4.0GeoCouch tag: diplomathesis
Volker Mische FOSS4G 2010 Barcelona 25 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
20 000 30 000 40 000 50 000Number of returned points
0
100
200
300
400
500
Tim
e in
mill
isec
onds
Bounding box query returning approx. 50% of the points
PostGIS 1.4.0SpatiaLite 2.4.0GeoCouch tag: diplomathesis
Volker Mische FOSS4G 2010 Barcelona 25 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
Benchmarks: Test 2
25%20%
15%10%
5%
Volker Mische FOSS4G 2010 Barcelona 26 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
17 500 35 000 52 500 70 000 87 500Number of returned polygons
0
500
1000
1500
2000Ti
me
in m
illis
econ
ds
Query with changing bounding box sizes(350 000 polygons)
PostGIS 1.4.0SpatiaLite 2.4.0GeoCouch tag: diplomathesis
Volker Mische FOSS4G 2010 Barcelona 27 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
5 000 10 000 15 000 20 000 25 000Number of returned points
0
50
100
150
200
250
300
350
400
Tiem
in m
illis
econ
ds
Query with changing bounding box sizes(100 000 points)
PostGIS 1.4.0SpatiaLite 2.4.0GeoCouch tag: diplomathesis
Volker Mische FOSS4G 2010 Barcelona 27 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
Applications
Volker Mische FOSS4G 2010 Barcelona 28 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
http://pdxapi.com/
Volker Mische FOSS4G 2010 Barcelona 29 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
Volker Mische FOSS4G 2010 Barcelona 30 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
Next steps
Polygon search (→ Erlang geometry library)
Sharding
Layer on top (e.g. for clustering, reprojection)
Volker Mische FOSS4G 2010 Barcelona 31 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
Thanks!
Volker Mische FOSS4G 2010 Barcelona 32 / 33
Intro CouchDB GeoCouch GeoCouch:A spatial index for CouchDB
Get in touch with me
Webiste: http://vmx.cx/
IRC: vmx @ freenode
Email: [email protected]
Jabber: [email protected]
Volker Mische FOSS4G 2010 Barcelona 33 / 33