GeoServer, GeoTools and Friends 2014

FOSS4G Tech Talk

About your Presenters Contributors, Guests, and Cameos

Andrea Aime (GeoSolutions)Martin Davis (JTS)Justin Deoliveira (Boundless)Jody Garnett (Boundless)Kevin Smith (GeoWebCache)

Outline● Going to go fast - and have time for Q&A● Intro

o GeoServer 2014o GeoTools 2014o JTS 2014 (Martin Davis)o GWC 2014o GeoScript 2014o Friends

● Features● Roadmap Wishlist● Q&A

$Money - Here is how this works

● Thanks to customers/employers/sponsors we can share a preview of some work scheduled for 2014o We thank customers early and ofteno it is how open source software gets made

♡● If you have an idea or can help out

volunteers are always neededo Contact details provided for active developmento Some wishes are ten years old

● Do not assumeo Anyone will pay for a visual SLD editor (sorry users)o A volunteer is testing Java 8 (sorry developers)

Love - here is how this works

GeoServer 2014

GeoServer

Designed for interoperability, it publishes data from any major spatial data source using open standards.

GeoServer: History & Organisations● Quick History …

o OpenPlans: WFS 2002o Refractions: UI 2003o Axios: WMS 2003

● Open Source Geospatial Foundationo Incubation 2009o Graduated 2013o New! © OpenPlans → © OSGeo 2014

GeoServer Highlights for 2014● GeoServer 2.4: released September 2014

o Served as stable branch for six monthso Extended for six months as maintenance branch

● GeoServer 2.5: released April 2014o Current stable brancho Scheduled six months of maintenance updates

● Governanceo Transfer of codebase from OpenPlans to OSGeoo New “community” modules → Graduated extensions

GeoServer: Health Checkup● 28 members of team geoserver

o 779 geoserver-devel!o Core group of very active contributorso New contributors being added

● How about team user-list?o 2290 geoserver-userso 169 geoserver italiao ??? GeoServer en Espanolo ??? OpenGeo.cn (GeoServer and more)o Some help with 2.5-beta … but many wait for 2.5.2

● Active well managed codebaseo 200 pull requests this yearo New! CONTRIBUTING.md file for GitHub

GeoServer: 2.5 Series

GeoServer: Downloads this Week

GeoServer moves to OSGeo● Open plans is stepping

back from software development● Strong relationship with OSGeo

o Vendor Neutral custodian to encourage participation from additional parties

o Lazy: Participants had often already signedOSGeo Contributors License for GeoTools

o See: GSIP-118

GeoTools 2014

GeoTools

GeoTools is an open source Java library that provides tools for geospatial data.

GeoTools: History and Organisation● GeoTools 1: 1996-2001

o Leeds University● GeoTools 2: 2002-Present

o OpenPlans, Refractions, GeoSolutions, etc ...● Governance

o OSGeo Incubation 2006o Graduated 2008o Copyright OSGeo as

a vendor neutral organisation

GeoTools Highlights for 2014● GeoTools 10: released September 2014

o Served as stable branch for six monthso Extended for six months as maintenance branch

● GeoTools 11.5: released April 2014o Current stable brancho Scheduled six months of maintenance updates

● Governanceo New “Unsupported” plugins → Graduated pluginso New Committers

GeoTools Health Checkup● 27 members of team geotools

o Active in bursts near releaseo New contributors being added

● Active well managed codebaseo 203 pull requests so far this yearo New! CONTRIBUTING.md file for GitHub

GeoTools 11 Series Downloads

JTS Topology Suite 2014Putting the Geometry in “Geo” since 2001

JTS Topology Suite 2014● Geometry

● SFSQL Implementation● News

● LocationTech proposal● LGPL → BSD / EPL● Version 1.13 (2012)● Version 1.14 (2014 Q4)

Martin DavisSenior Technical Architect

Vivid Solutions Inc.

GeoWebCache 2014

GeoWebCache 2014● Tile Cache

o WMS-Co TMSo WMTS

● Newso OpenPlans →

OSGeoo Version 1.5 (2013 Q3)

o Version 1.6 (2014 Q3)

Kevin SmithJunior Software Developer

Boundless Spatial Inc.

GWC: Mass Truncation via RestExternal management of GeoWebCache!

POST http://localhost:8080/geowebcache/rest/masstruncate<truncateLayer> <layerName>trekConventionDensity</layerName></truncateLayer>

1.5.0Kevin SmithBoundless

● GeoWebCache 1.5.0: August 2014o Extensible Parameter Filterso First release synchronized with GeoServer

● Fixes/Improvements added to 1.5.3o WMS Metadata Passthrougho WMS GetLegend Passthrough

GWC Features / Fixes

1.5.1Andrea AimeGeoSolutions

1.5.1Eric DahlVolunteer

GeoWebCache Roadmap (1.6.0)● Read ArcGIS Compact Caches

o Generate with ArcGIS and Serve with GWC

● Improvements to Tile Fuser

Nicola 1.6.0GeoSolutions

Bjoern Saxe 1.6.0Independent

GeoScript

Spatial capabilities for dynamic scripting languages● Groovy● Scala● JavaScript● Python

Justin Deolivera

GeoScript

● Jared working hardo Curved geometries, GeoPackage, MBTiles

● Working on a common versioning scheme● Standardizing on GeoTools version● Early stages of incubation● TODO: Logo :)

GeoScript: Update

GeoScript for GeoServer

Friends 2014

ImageIO-Ext (more formats)

Extensions, fixes andfor Java ImageIO:● Support for GDAL● JPEG2000 via

Kakadu● Improved TIFF● Now on GitHub!

Simone Giannecchini

JAI-Ext (more image processing)● more features to

existing operations● improved

performances● developing new

operations

Nicola Lagomarsini

JAI Tools

Image Processing● Tools for analysis● Zonal-Stats● Contours, vectorize

Michael Bedward

Jiffle

Raster Calculator● Domain specific

language for image processing

● C-Syntax for that curly brace comfort

Michael Bedward

The magic of Open Source

Features and Fixes

Customer ReleaseImplementor

A WCS version for human beings

WCS 2.0

DLRAndrea Aime

Simone GiannecchiniDaniele Romagnoli

2.6.0

WCS protocol support● New GeoTools classes to parse/encode

the XML documents in the WCS 2.0 protocol

Full GeoServer implementation● Implementing the full protocol● Including the GeoTiff

Extensions, controlling details of the generated GeoTiff file

● NetCDF output (see later)

Did I mention the protocol looks sane?

http://localhost:8080/geoserver/wcs?request=GetCoverage&service=WCS&version=2.0.1&coverageId=geosolutions__NO2&Format=NetCDF&subset=http://www.opengis.net/def/axis/OGC/0/Long(5,20)&subset=http://www.opengis.net/def/axis/OGC/0/Lat(40,50)&subset=http://www.opengis.net/def/axis/OGC/0/elevation(300,1250)&subset=http://www.opengis.net/def/axis/OGC/0/time ("2013-03-01T10:00:00.000Z","2013-03-01T22:00:00.000Z")

Natively multidimensional

NetCDF/Grib

DLR/EumetsatDaniele

Romagnoli12.0/2.6.0

Many coverages per reader● A GeoTools reader can read and

advertise metadata for multiple coverages now

● GeoServer updated to follow suit

StructuredCoverageGridReader● Drill down in the various dimensions● Interact with slices, list, filter, add, update,

delete (GranuleSource/GranuleStore)● Harvest

entire new files into the sources (mosaic only)

GeoServer support● Publish NetCDF/Grib as any other format● Configure WMS-T dimensions● Return a multidimensional array via WCS

2.0 NetCDF output format

http://localhost:8080/geoserver/wcs?request=GetCoverage&service=WCS&version=2.0.1&coverageId=geosolutions__NO2&Format=NetCDF&subset=http://www.opengis.net/def/axis/OGC/0/Long(5,20)&subset=http://www.opengis.net/def/axis/OGC/0/Lat(40,50)&subset=http://www.opengis.net/def/axis/OGC/0/elevation(300,1250)&subset=http://www.opengis.net/def/axis/OGC/0/time ("2013-03-01T10:00:00.000Z","2013-03-01T22:00:00.000Z")

Handle difficult projections

Coverage reprojection improvements

GeoSolutions Andrea Aime 12.0/2.6.0

Chilling beauty of polarstereographics

BeforeAfter

Crossing the dateline with rasters

Speeding up your PNG and JPEGs

Faster output encoding

GeoSolutions Andrea Aime 2.5.0

GeoServer faster PNG encoder● PNG encoder

based on the PNGJ open source library

● Pure java, yet quite a bit faster

Put a turbo in that JPEG● Based on the libJPEGTurbo encoder● Significant speedup compared to JDK own

encoder

Because we care about your icons

Pixel precise GetFeatureInfo

Eskilstuna Andrea Aime 2.5.0

GetFeatureInfo issues● Most software does a small square bbox

query● Symbol sizes took into account to an extent

(issues with dynamically sized symbols)● No care for symbol shape ● Solution → actually draw stuff and check

pixels

Dynamic sizes● Size proportional to

attribute● Before, either miss

the big circle, or report the small one as being hit

● Now right match

Shapes and offsets● The new code

won’t report a hit if you did not click the symbol

● Takes care of odd symbol shapes, and offsets

Everybody loves curves

Curves support

City of Vienna Andrea Aime 12.0/2.6.0

Represent curves in memory● Circular arc by 3 points● CircularString: sequence of the above● CompoundString: mix straight and circular

arcs● Use in polygons as well

Smart linearization● When rendering, linearize

according to the rendering resolution

● Otherwise, if necessary, do so with a given tolerance (e.g., when using JTS topological operations)

● Selected code paths just keep the curves

GML encoding

Reading Curves● Only from Oracle spatial

$Reading Curves

● Contributions/funding to read from other data sources welcomed!

Restructure your rasters in GeoServer

Coverage views

GeoSolutionsDaniele

Romagnoli2.6.0

Creating coverage views

Band selection

Coming soon: computed bands

Performance and scalability on all platforms

Marlin

VolunteerLaurent Bourges

2.6.0

Java rasterizer scalability issues● Rasterizing: turning vector data into

antialiased images (maps)● Basically two choices

o Use Oracle JDK (Ductus), binary, closed source, fast rasterization, but scales up poorly

o Use OpenJDK (Pisces), pure java, open source, slower rasterization, but scales up well

Marlin rasterizer● Fork of OpenJDK Pisces● As fast as Ductus

o as scalable as Pisceso Testing a complex

multiscale mapwith 10 concurrentclients at differentzoom levels

Marlin Easy Install

For Windows/Linux, Oracle or OpenJDK:1. Download one jar from

https://github.com/bourgesl/marlin-renderer2. Add these options to the

JVM-Xbootclasspath/a:/path/to/marlin-X.Y.jar -

Dsun.java2d.renderer=org.marlin.pisces.PiscesRenderingEngine

3. Done!

Marlin in Java 9?● The author, Laurent Bourges,

is trying to get Marlin merged in Java 9● Lack of response from Oracle● Please Oracle, we care about

performance… don’t you?

Filter 1.0 Classes

Boundless Jody Garnett 12.0

Technical Debt / Cleanup

Filter Interfaces - Technical Debt

GeoTools 2.3 introduced Filter 1.1 Interfaces● Budget only covered new implementation

o GeoTools tests still used classes directlyo GeoServer and other apps also

● Two options for Filter confuses developerso Class or Interface/Factory?o What do I import to make this code work?

OpenGeo Justin Deolivera 2.3

Filter Cleanup - Cleanup

GeoTools 12.0 Cleanup● Cleaned up with / careful refactoring!● Test for XML Parsers / WFS were hardestGeoServer 2.6.0 Cleanup● Cleanup went smoothly● BBOX Filters caused trouble

Boundless Jody Garnett 12.0

wfs-ng

Boundless Niels Charlier 12.0

WFS “Next Generation” client

Ground up implementation of wfs-client● Using GTXML parser (yay!)● Based on ContentDataStore

(implementation is much easier to follow)● WFS-T 1.1 ← exciting!

WFS Cascade● GeoServer surfaces the wfs-ng as the ability

to “cascade” content from remote (or internal) WFS Servers

● Performing 2.6 preflight testing wfs-ng was a marked improvement over the origionalo We have cut over “cold turkey”o We asked the geoserver-users list for help testing all

those tricky INSPIRE services● wfs-ng includes configuration overrides

o You can account for several common mistakes(such as servers that get confused about axis order)

gt-render-wkt

geotools-users Group Effort! 12.0

gt-render-wkt merged to gt-render● Formally an optional unsupported plugin

o This was a hidden “power tool” for great mapso Create symbols using WKT directly inside your SLD

<WellKnownName> wkt://LINESTRING(0.0 0.25, 0.25 0.25, 0.5 0.75, 0.75 0.25, 1.00 0.25)</WellKnownName>

● Merge with gt-rendero Available by default for everyoneo Go out and make maps!

windbarbs

GeoSolutions Simone 12.0

New mark factory for windbarbs● Use WMO compliant wind barbs● Well known name template:windbarbs://$

(value)[m/s]?emisphere=(n/s)

● E.g.: windbarbs://25[m/s]?emisphere=n

ResourceStoreWeaning GeoServer off Disk

Boundless Jody Garnett 2.6.0

ResourceStore (Internal)● GeoServer DataDirectory

o Default is “in the web apps folder”o Copy to separate location when upgradingo Can be split up several directories ← why?

● Optional to Store in JDBC Databaseo “Catalog” configuration stored in Databaseo “Data Directory” still used for SLD, Icons, Fonts

● ResourceStore is “Step 1” of fixing thiso Use “Resource” rather than “File”

Direct access to JDBC BLOB “check out” file from the Database (if needed)

Boundless Jody Garnett 2.6.0

New GeoServer WebsiteOne stop shop for everything GeoServer

BoundlessJustin Deolivera

Jody Garnett2.6.0

GeoServer: Home

BoundlessJustin Deolivera

Jody Garnett2.6.0

WebSite: Downloads

BoundlessJustin Deolivera

Jody Garnett2.6.0

WebSite: GitHub Wiki

BoundlessJustin Deolivera

Jody Garnett2.6.0

DataStore TutorialThanks FOSS4G 2014

Boundless Jody Garnett 12.0-RC1

ContentDataStore

Boundless Jody Garnett 12.0-RC1

RoadmapScheduled Features for 2014

$SOLR data store

● Connect, display, filter data stored in your SOLR server

● Expose all of them to the web via WMS/WFS/WPS

GeoSolutions 2.7.x/2.6.xNSW-GS-AU

$Clustering asynch WPS

● The only OGC operation maintaining state● Improve GS so that the state gets shared

across cluster nodes

GeoSolutions 2.7.xGeoSolutions

WPS1 WPS2 WPS3

Shared async state

Load balancer

$Clustering GS config

● Instantly propagate config changes among GS nodes

GeoSolutionsBoundless

2.7.xGeoSolutions

Boundless

GS1 GS2 GS3

Load balancer

Message passing bus

$GeoFence community module

● Advanced security rules in GeoServer● Separate web app for config, GS plugin to

drive security● Fully open source, already in use by some

GS based projects (GeoOrchestra)

GeoSolutions 2.6.xGeoSolutions

GeoServer GeoFence plugin

GeoFence UI and REST

♡CSS module for GeoTools

● Port of the GeoScript Scala version to Java

● Allow more people to contribute

● Faster translation to SLD● Adding LessCSS features

down the road

Andrea Aime 13.xAndrea Aime

/* @title Population < 2M */[PERSONS

< 2000000] {

fill: #4DFF4D;

fill-opacity: 0.7;

}/* @title 2M < Population < 4M

*/[PERSONS > 2000000] [PERSONS

< 4000000] {

fill: #FF4D4D;

fill-opacity: 0.7;

}/* @title Population > 4M

*/[PERSONS > 4000000] {

fill: #4D4DFF;

fill-opacity: 0.7;

}

WishlistGet involved - your support is needed!

♡Upgrade Wicket User Interface

● User interface is slowly falling aparto No longer works with latest browser

(Current advice - use Firefox!)o Need to upgrade to newer version of “Wicket”

● Why?o User interface is not fun enough to attract volunteerso Popularity of REST API

♡Update CITE Tests

● CITE Tests are the OGC conformance tests● Pass them and you get a cool sticker for

your websiteo Well only if you payo Idea: Ask OSGeo Board to talk to OGC?

● Tests keep us all safe!o We would like some time to upgrade / fixo Help out at the Code Sprint this weekend

♡Money/Love needed for:● WPS 2.0 Operations

o Improved Job Control (stop process?)● WPS has CITE tests!

GeoServer: WPS 2.0

♡High DPI Mapping

● OL3 and GeoServer can already show wonderful images on High DPI displays

● To roll this concept out across the stacko GWC to Tile based on “dpi” format optiono GeoTools rendering external graphics

♡GWC Cloud Storage

● Add options for cloud storage:o Amazon S3, Acumulo, Azure, Google, OpenShift

● Mad ideas:o Integrate S3 with AWS’s CDN

Kevin Smith 1.7.xBoundless

♡GWC Storage Filter

● Processes tiles before storing them● Specific implementations can:

o Image Processing: color balance tileso Re-encode: Changed PNG z-compression levelo Change colour depth: PNG24 → PNG8o Run external tools: PNGCrush etc ...

Kevin SmithBoundless 1.7.x

♡Geometry + Geometry + Geography

JTS Project has limited scope:● GeoTools extends Geometry for Curves● ISO 19107 Point/Curve/Surface● Spatial4J looking at Geography● GML Geometry wrappers around JTSMoney/Love needed: Geometry needed for Open Source GIS industry

Questions?Thank you - for another amazing year!

Q: Software Downloads?http://geoserver.org/download/ http://www.geotools.org http://tsusiatsoftware.net/jts/main.htmlhttp://geowebcache.orghttp://jaitools.orghttps://github.com/geosolutions-it/imageio-ext/

Q&A● Q: When will be Java port of

CSS Style be available? Interested in using it from MapFish.A: Andrea - It is a background activityneed several more months on it.

● Q: Coudl not use as our State Plane defaults to meters and we measure in feet? I reported a bug last monthA: Feet? That is a terrible idea! Still hard to get answer this close to foss4g - will you be at the code sprint? Thanks for bug report.

Q&A● Q: Would anyone here be interested in a

GeoTools Cookbook?A: Please contact Ian Turton