State of GeoTools

1

About your PresenterJody Garnett

ActivitiesGeoTools PMCGeoServer PSCuDig PSCOSGeo Charter MemberOSGeo Incubation ChairLocationTech

LISAsoft

LISAsoftA great system integration company helping our customers make effective use of open source spatial.

2

Thanks to Previous PresentersJustin Deolivera(PMC)

OpenGeoBringing the best open source geospatial software to organisations around the world.

Andrea Aime(PMC)

GeoSolutionsGeoServer/GeoTools core developers, raster data management, map rendering, spatial data processing

3

GeoToolsThe Java GIS Toolkit

4

GeoTools - the Java GIS Toolkit

5

About GeoTools

6

Open SourceLGPLOpen DevelopmentPublic ProcessAnyone can Play

1994 - PresentReally old for JavaStarted in Leads UniversityOSGeo Project in 2006Active / Diverse committers

History1994 GeoTools (and GeoTool “Lite”)

Version 1 had more users then documentationSolution was to fire the users ...

2002 GeoTools 2We learned - GeoTools 2 leans on the OGC standardsOGC provides public standardsThey are documentation we do not have to write

Benefit: Development was faster without inventing names

2010 GeoToolsBack to simply “GeoTools”

7

Traditional Diagram of Boxes

8

Why Mapping

9

10

Why Mapping

Map for Your Mother!

(or Dad!)

Do your Parents Know what you Do?

11

UML bad / Map good

12

Text GeoServer Gallery by Gérald Estadieu

State of GeoTools

13

2012

Community 261 Andrea Aime (GeoServer) - GeoSolutions 261 Jody Garnett (uDig) - Lisasoft 167 Simone(GeoServer) - GeoSolutions 123 Justin Deoliveira (GeoServer) - OpenGeo 114 Michael Bedward (JAITools) 66 Danielei (GeoServer) GeoSolutions 54 Ben Caradoc Davies - CSIRO 51 Niels Charlier - CSIRO 32 moovida (JGrass) - Hydrologis 28 ang05a - CSIRO 25 groldan - (GeoServer) - OpenGeo 17 mpazos - Axios 15 victortey - CSIRO 9 kengu - Norwegian Uni of Tech & Science 8 ischneider - OpenGeo 7 ianturton - University of Leeds 7 jesseeichar - CampToCamp 7 lmoran - Consultant 6 afabiani - GeoSolutions 2 alfonx (AtlasStyler) - WikiSquare

2010 Stats tracking commits

30 committers10 organisations

14

geotools-devel:    414 members total

geotools-gt2-users:    969 members total

15

Summary• Ohloh summary

Q: Jump in Size?A: Move to SVN then

move to GIT

Issue Tracker Activity

Created vs resolved tickets (click image for update )

16

Real Lock Down thisyear for 8.0 Release!

Name ChangeDropping the “2”

17

GeoTools 2 now called GeoTools• What is in a Name?

• sed/GeoTools 2.8.0/GeoTools 8.0/

• What does this mean?• Not very much - business as usual

• What does that really mean?• It means there is no plans for a “GeoTools 3” rewrite :-)• Confidence in our community• Continue our commitment to stability• Evolution handled with continuous small changes

18

DocumentationIts a feature - really

19

User Guide - Before

20

User Guide - After

21

Code Examples - Before (Wiki)

Compile?What Version?When?

All very good questions with no clear answer.

22

private synchronized static FeatureSourcebuildMemoryFeatureSource(FeatureCollection coll, String typename,FeatureType currentFeatureType, AttributeType newTypes[]){

FeatureType ftState=null; MemoryDataStore MStore=null; try { MStore = new MemoryDataStore(); //---- current attributes AttributeType currentTypes[] = currentFeatureType.getAttributeTypes(); //---- new array of attributes = current array+ new attributes AttributeType typesNew[] = newAttributeType[currentTypes.length+newTypes.length]; for(int i=0;i<currentTypes.length;i++){ typesNew[i] = currentTypes[i]; } for(int i=0;i<newTypes.length;i++){ typesNew[currentTypes.length+i] = newTypes[i]; } ftState = FeatureTypeFactory.newFeatureType(typesNew, typename); MStore.createSchema(ftState); Iterator iterator = coll.iterator(); FeatureCollection newColl = FeatureCollections.newCollection(); Feature feature, newFeature; Object[] objs; try { for( int count=0; iterator.hasNext(); count++) { feature = (Feature) iterator.next();

private synchronized static FeatureSourcebuildMemoryFeatureSource(FeatureCollection coll, String typename,FeatureType currentFeatureType, AttributeType newTypes[]){

FeatureType ftState=null; MemoryDataStore MStore=null; try { MStore = new MemoryDataStore(); //---- current attributes AttributeType currentTypes[] = currentFeatureType.getAttributeTypes(); //---- new array of attributes = current array+ new attributes AttributeType typesNew[] = newAttributeType[currentTypes.length+newTypes.length]; for(int i=0;i<currentTypes.length;i++){ typesNew[i] = currentTypes[i]; } for(int i=0;i<newTypes.length;i++){ typesNew[currentTypes.length+i] = newTypes[i]; } ftState = FeatureTypeFactory.newFeatureType(typesNew, typename); MStore.createSchema(ftState); Iterator iterator = coll.iterator(); FeatureCollection newColl = FeatureCollections.newCollection(); Feature feature, newFeature; Object[] objs; try { for( int count=0; iterator.hasNext(); count++) { feature = (Feature) iterator.next();

SimpleFeatureSource  alter(SimpleFeatureCollection  collection,  String  typename,                SimpleFeatureType  featureType,  final  List<AttributeDescriptor>  newTypes)  {                try  {                                //  Create  target  schema                SimpleFeatureTypeBuilder  buildType  =  new  SimpleFeatureTypeBuilder();                buildType.init(featureType);                buildType.setName(typename);                buildType.addAll(newTypes);                                final  SimpleFeatureType  schema  =  buildType.buildFeatureType();                //  Configure  memory  datastore                final  MemoryDataStore  memory  =  new  MemoryDataStore();                memory.createSchema(schema);                                collection.accepts(new  FeatureVisitor()  {                        public  void  visit(Feature  feature)  {                                SimpleFeatureBuilder  builder  =  new  SimpleFeatureBuilder(schema);                                                                builder.init((SimpleFeature)  feature);                                for  (AttributeDescriptor  descriptor  :  newTypes)  {                                        builder.add(DataUtilities.defaultValue(descriptor));                                }                                                                SimpleFeature  newFeature  =  builder.buildFeature(feature.getIdentifier().getID());                                memory.addFeature(newFeature);                        }                },  null);                                return  memory.getFeatureSource(typename);

Code Examples - After (Sphinx)

Pulled Live from Source!

GeoTools Version at the Top of the Page!

23

Code Examples - Example

24

Used for both Java code examples and also configuration files such as maven pom.xml.

Diagrams - Before (Visio)

25

Diagrams - After (Inkscape)

26

Class Diagrams - Before

27

Class Diagrams - After (Object Aid)

28

Tutorials - Before

29

Tutorials - After

30

Advanced TutorialsFunction Finally a nice quick guide to how to create a function!

AbstractDataStore Tutorial - Property DataStore The classic updated for GeoTools 8.

ContentDataStore Tutorial - CSVDataStore New kid on the block; leaner; meaner and ready.

Welcome Introduction for different developer communities.

Update Have not used GeoTools in a while? Guide to updating.

31

Welcome Geomajas

32

Welcome GeoServer

33

Welcome uDig

34

Documentation HighlightsDetails- 291 “Pages”- longest page is 36 pages printed out on paper- Over 100,000 words- Written from Feb through to May- Accounts for every page of the wiki

New features- Extensive FAQ- whole project javadocs- module by module reference- plugins organised by module- build instructions

35

Referencing(do you know where your data is?)

36

Referencing• Ellipsoids, datum, map projections

• Embedded EPSG database, over 5000 supported codes (gt-epsg-hsql module)

• Reprojection utilities

37

Geometry g = ...;CoordinateReferenceSystem wgs84 = CRS.decode("EPSG:4326");                        CoordinateReferenceSystem utm32n = CRS.decode("EPSG:32632");Geometry gtx = JTS.transform(g);...SimpleFetureCollection fc = featureSource.getFeature();SimpleFeatureCollection fcTx =                 new ReprojectingFeatureCollection(fc, utm32n);

Projections

Mollweide

Eckert IV

Winkel Tripel

Robinson

Equidistant conic

Old projections

AitoffAlbers Equal A.CassiniSoldnerCylindrical Equidist.KrovakLambert Conf.MercatorOblique MercatorOrthogonalPolyconicStereographicTransverse Mercator 38

New

Vector PluginsBoth direction and magnitude!

39

Vector Data Sources• Application schema*• ArcSDE• Aggregating *• CouchDb *• CSV *• DXF *• Excel• Edigeo• PreGeneralized• SimpleFeatureService *• Shapefile (directory of)• WFS

• JDBC• DB2• H2 spatial• MySQL• Postgis• Oracle • SQL Server• Spatialite *• Teradata *• Ingress

name* = new entry name = “unsupported”

40

JDBC DataStore "NG"• JDBC stores:

• common base• over 200 ready to use tests, just prepare the test data and

run• Parametric SQL views: define a new data source as a raw

query

• Extensions for native joins

select ST_Union(the_geom) as geom from countrieswhere REGION = %region%

41

Application schema• Complex feature support• Recent development by CSIRO• Feature chaining, polymorphism• Better performance (native DBMS joins), memory use• Usable as vector data source, GML encoding, rendering

42

Raster Formatspixel power

43

Raster Data Sources• ArcGrid• ArcSDE • GeoTiff• GRASS *• GTopo30• Image Mosaic• Image Mosaic JDBC *• Image Pyramid• Image + World file• JP2 direct Kakadu• Matlab• Image collection *

• imageio-ext based• AIG• DTED• ECW• EnviHdr• ErdasImg• EsriHdrJP2• MrSID• NITF*• RPFTOC

name * = new entryname = unsupported land

44

Non geo-referenced data

45

Image mosaic

46

RenderingGotta see it to believe it

47

Rendering

48

Layers

49

• FeatureLayer:any vector + Style(shapefile, spatial db, WFS, ...)

• GridCoverageLayer:in memory raster grid

• GridReaderLayer:dynamic raster read from disk(leverage overviews and tiling)

• WMSLayer:get maps from a WMS server

• DirectLayer: (new feature!)roll your own custom paint method

• Style API is inspired by SLD 1.0 and SE 1.1: • a style is a complex, deep object graph

• StyleBuilder provides a set of hierarchical builders• Build a style easily, just pick the portion you need

StyleBuilder

50

Style style = new StrokeBuilder().color(Color.BLUE)              .width(3).dashArray(5, 2)              .buildStyle();

StyleBuilder

51

• StyleBuilder allows you to freely mix CQL expressions• Provides “literate” methods allowing you to define your style

using your IDE autocomplete

fts = new FeatureTypeStyleBuilder();fts.rule().filter("pop < 200000")   .polygon().fill().colorHex("#66FF66");fts.rule().filter("pop between 200000 and 500000")   .polygon().fill().colorHex("#33CC33");fts.rule().filter("pop > 500000")   .polygon().fill().colorHex("#009900");

Showing the Map Part 1To start we get the data and create a StyleIn this case we open a file and generate a style from the schema

Similar to SE 1.1 transformations, but open-ended,

52

FileDataStore store = FileDataStoreFinder.getDataStore(file);SimpleFeatureSource featureSource = store.getFeatureSource();SimpleFeatureType schema = featureSource.getSchema();

Style style = SLD.createSimpleStyle( schema );

Showing the Map Part 2Create Map and use JMapFrame to display

Similar to SE 1.1 transformations, but open-ended,

MapContent map = new MapContent();map.setTitle("Quickstart");Layer layer = new FeatureLayer(featureSource, style);map.addLayer(layer);

JMapFrame.showMap(map);

53

Pluggable Mark Factory TTF

<Mark> <WellKnownName>ttf://Wingdings#0xF054</WellKnownName>  <Fill> <CssParameter name="fill">#000088</CssParameter> </Fill></Mark>

54

Pluggable Mark Factory Slash

<Fill>  <GraphicFill>  <Graphic>    <Mark>      <WellKnownName>shape://slash</WellKnownName>      <Stroke>        <CssParameter name="stroke">0xAAAAAA</CssParameter>      </Stroke>    </Mark>    <Size>16</Size>  </Graphic>

55

Pluggable Mark Factories Interface

public interface MarkFactory {    public java.awt.Shape getShape(            Graphics2D graphics,             Expression symbolUrl,             Feature feature) throws Exception;}

org.geotools.renderer.style.WellKnownMarkFactoryorg.geotools.renderer.style.TTFMarkFactoryorg.geotools.renderer.style.ShapeMarkFactory

56

Geometry transform Arrow Head

<PointSymbolizer>  <Geometry>     <ogc:Function name="endPoint">                <ogc:PropertyName>the_geom</ogc:PropertyName>     </ogc:Function>  </Geometry>  <Graphic>    <Mark>      <WellKnownName>shape://arrow</WellKnownName>      <Fill/> <Stroke/>    </Mark>    <Rotation>      <ogc:Function name="endAngle">         <ogc:PropertyName>the_geom</ogc:PropertyName>      </ogc:Function>    </Rotation>  </Graphic></PointSymbolizer>

Similar to SE 1.1 transformations, but open-ended,

57

Geometry transform Drop Shadow

<Geometry>  <ogc:Function name="offset">       <ogc:PropertyName>the_geom</ogc:PropertyName>    <ogc:Literal>0.00004</ogc:Literal>     <ogc:Literal>-0.00004</ogc:Literal>  </ogc:Function> <Geometry>

58

Based on functions, a pluggable extension point

Geometry transform Implementation

public class FilterFunction_offset extends FunctionExpressionImpl implements GeometryTransformation {...public Object evaluate(Object feature) {        Geometry geom = getExpression(0).evaluate(feature, Geometry.class);        Double offsetX = getExpression(1).evaluate(feature, Double.class);        if(offsetX == null) offsetX = 0d;        Double offsetY = getExpression(2).evaluate(feature, Double.class);        if(offsetY == null) offsetY = 0d;

        if (geom != null) {            Geometry offseted = (Geometry) geom.clone();            offseted.apply(               new OffsetOrdinateFilter(offsetX, offsetY));            return offseted;        } else {            return null;        }

59

Transform data before rendering. Contour extraction, wind maps, geometry clustering. Pluggable, add your function!

Rendering Transform Wind Map

<FeatureTypeStyle>    <Transformation>     <ogc:Function name="gs:RasterAsPointCollection">       <ogc:Function name="parameter">             <ogc:Literal>data</ogc:Literal> 

60

u v

Rendering Transform Contour Extract

<FeatureTypeStyle>   <Transformation>     <ogc:Function name="gs:Contour">       <ogc:Function name="parameter">         <ogc:Literal>data</ogc:Literal>       </ogc:Function>       <ogc:Function name="parameter">         <ogc:Literal>levels</ogc:Literal>         <ogc:Literal>1100</ogc:Literal>         <ogc:Literal>1200</ogc:Literal> ....        <ogc:Literal>1700</ogc:Literal>         <ogc:Literal>1800</ogc:Literal>       </ogc:Function>     </ogc:Function>  </Transformation>

61

XML(what could be as fun as rendering? not xml)

62

XML

Technology Parse Encode Supports

GeoTools SAX SAX,DOM Filter, GML, SLD

GeoTools DOM DOM Filter, GML, SLD

GeoTools Transform XML Filter, GML, SLD

JAXB SAX,DOM n/a

XDO(Schema Assisted

Generation 1)SAX,DOM XML Filter, GML, SLD, WMS, WFS1.0,

XSD

GTXML(Schema Assisted

Generation 2)SAX,DOM XML Filter, GML, SLD, WMS, WFS1.0,

WFS 1.1, WPS, XSD, and more...

63

Schema Assisted Parsing

64

GeoTools Specific Technology; makes use of the schema information to minimise the code you need to write.You teach GeoTools how to “bind” a object once; and it will work in any schema that uses those objects.

Schema Assisted Encoding

65

The same approach is used when encoding objects; basically using the schema as a cheat sheet to sort out what to emit next.

Extensionshigh value functionality built on top of GeoTools

66

BrewerWraps up the ColorBrewer for Java Developers; generating a normal Style you can use in GeoTools; or export to other applications.FeatureTypeStyle style = StyleGenerator.createFeatureTypeStyle( groups, ff.propertName(“population”), colors, "Generated Style", schema.getGeometryDescriptor(), StyleGenerator.ELSEMODE_IGNORE, 0.95, stroke);

67

Because getting there is half the fun.

Graph

68

BasicLineGraphGenerator graphGen = newBasicLineGraphGenerator();for ( MultiLineString line : lines ) { graphGen.add( line );}Graph graph = graphGen.getGraph();

EdgeWeigter weighter = new EdgeWeighter() { public double getWeight(Edge e) { SimpleFeature feature = (SimpleFeature) e.getObject(); Geometry geometry = (Geometry) feature.getDefaultGeometry(); return gometry.getLength(); }};DijkstraShortestPathFinder pf =new DijkstraShortestPathFinder( graph, start, weighter );

pf.calculate();Path path = pf.getPath( destination );

ProcessStill in Progress

69

ProcessAnnotation Based No more filling in massive data structures; GeoTools will generate based on annotations against a static method.

Bridge Functions and Process Used for rendering transformations; allowing you much greater control of geometry prior to display.

Massive influx of High Quality Processes The full range of processes defined by GeoServer have been back ported resulting in some of the best examples of how to use GeoTools.

WPS Client Still unsupported ... nothing to see here 70

Tutorial Coming soon!

Process Annotation Example

71

@DescribeProcess(title="Buffer", description="Buffers a geometry using distance")@DescribeResult(description="The buffered geometry")static public Geometry buffer( @DescribeParameter(name="geom", description="The geometry to be buffered") Geometry geom, @DescribeParameter(name="distance", description="The distance") double distance) { return geom.buffer(distance);}

Application SchemaGraduated this year!

72

Application SchemaAuScope has completed their work on Application Schema.This is an interesting module that is able to take information from several DataStores and "map" the values into the correct shape for publication against an application schema.

This is used when the data format used cannot be generated and must exactly match that provided by a standards body or expert community.

Thanks to Ben and the AuScope team for their dedication on this very challenging problem.

73

Teradata DataStoreAnother spatial database into the mix!

74

Terradata DataStoreJesse Eichar (Camptocamp) took this module from development, a quick unsupported beta period and to supported status in record time.

75

EFeatureMapping between Eclipse Modelling Framework

and GeoTools Feature Model.

76

EFeature• Kenneth Gulbrandsoy has been an amazingly active

addition to the GeoTools community. Kenneth is a PHD student from Norwegian University of Technology and Science.

• EFeature datastore allows teams to leverage their Java objects into the GeoTools dynamic feature system

• Adding Spatial capabilities to EMF models• 98% finished• Goal is to make it a supported plugin in GeoTools• Docs are comming soon

77

Grass Raster readerCreated and Graduated this year!

78

Grass Raster ReaderAndrea Antonello has pulled together support for the GRASS raster format and donated it to the GeoTools library (going through the effort to match our quality assurance and documentation standards).

79

TemporalMerged into gt-main as core feature

80

TemporalThe temporal module has provided an enhanced range of time and time range operations.Justin has gathered this up into the gt-main module as one of the cornerstones for WFS 2.0 Filter support.

81

after = ff.after( ff.property("date"), ff.literal(temporalInstant));

within = ff.toverlaps( ff.property("constructed_date"), ff.literal(period));

Query ImprovementsJustin takes on WFS 2.0

82

Query ImprovementsJoin

Justin has sorted out how Joins will be represented - finally!Function

Between Jody and Justin Functions now have excellent runtime documentation. Arguments are now listed by name; with specific type information.

Filter Evaluation over a CollectionNiels (AuScope) has sorted out this one allowing you to control how many matches are required (one, any, all) when comparing groups of values.

TemporalAs mentioned temporal filter support has been added; seriously increasing the number of operations available.

83

Swing and SWT(Getting your GUI on)

84

Swing The much loved gt-swing module has been used for years to make the GeoTools tutorials visual. At long last, it is looking to graduate to supported status.

Michael Bedward has been a star contributor this yearcleaning up the code base,making it testable, all thewhile answering questionson the email list.

85

Swing Major clean up of the internals!JMapPane rewritten

Ported to Layer, FeatureLayer, RasterLayer etc...DirectLayer support for Animation / map decorationsNice clean background rendering

Ability to add new InfoTool classes via pluginSupport for InternationalisationSupported Status?

Docs and Test Coverage improving by the day

Coming Soon: Concurrent rendering

86

SWTA new addition to the project is a port of the gt-swing code to an easy to use component for SWT.

Andrea Antonello (HydroloGIS) was responsible for this work and it looks like gt-swing and gt-swt will be able to share the same data models as time progresses.

87

What is Past?Reflections on departed modules

88

Recently DroppedJDBCDataStore and PostGIS-Versioned

The initial proof of concept JDBC datastore created in 2003 has finally been replaced. We had a good 2 year deprecation cycle to allow migration.

GeoAPIThe "opengis" interfaces have been folded back into the library allowing us to take responsibility, improve javadocs and dive into filter improvements for WFS 2.0 work.

89

What is Next?Ideas on the future of GeoTools

90

Future PlansProcess, Process, Process

Web Processing Service is finally attracting funding, with it comes a lot of new process ideas, implementations and directions. Hold on it is going to be a wild ride!

Prep for Java 7 try-with-resourceUpdate our API to mark which items are “Closable”

FeatureCollection as a Result SetFor Java 5 we needed to prevent FeatureCollection extending java.util.Collections - so that iterators could be closed.We are completing this work by removing the deprecated method names (add, remove, etc...)This will allow FeatureCollection to be a simple result set.

91

Future RnD IdeasOSGi Bundles

Branch started to explore this topic with contributors from several projects.

AndroidJody had a chance to look at what it would take to port GeoTools to Android. If you are interested in doing the work talk to us at the Code Sprint.

3DA number of teams hook GeoTools to 3D engines each year. Anyone want to do it as part of the community?

Geometry Curves and 3D Tisham Dhar (CSIRO) evaulated several alternatives. GeoTools has components of ISO 19107 provided by SYS Technologies and the of University Fachhochschule Koeln.MIL2525B

Another classic we revisit each year - any takers? 92

Open DevelopmentIs so much more than open source

93

Get Involved - Open DevelopmentGeoTools provides clear “time boxed” procedures allowing you to plan your community involvement.

Contribute Clean patch + test + description = contribution

Request for Change Open RFC procedure, time boxed allowing commit three days after submitting (we have to meet deadlines as well).

Release Public release process - make a release to match your projects schedule and deadlines.

Procedures are in place just add developers (or money).94

Get Involved - Unsupported AreaUnsupported is a scratch pad used to work on new ideas.

Easy Commit Access Send email and read the developers guide; a PMC member will help you with the paper work.

Unsupported does not mean unloved You get commit access to your module, and your work deployed to maven nightly, and published each release.

Formal review only when Ready Ask for a QA / Docs check at any time and graduate to a formal supported part of the library.

95

Predictable Release Cycle• The “wait” for GeoTools 8 was far too long.

• GeoTools is switching to a 6 month release cycle:

• Stable• monthly releases• New features that do not change the API or Stability

• Master• 1-4: open for change proposals• 5: release candidate• 6: final release!

96

State of GeoToolsAny Questions? Any time?

97

Upstream ProjectsThanks! We don't Do this alone

98

Upstream ProjectsJTS Topology Suite (JTS) Provides the excellent Geometry support we enjoy in GeoTools. Thanks Martin!imageio-ext Provides geospatial raster formats to the Java Advanced Imaging project we use for coverages. Thanks Simone and GeoSolutions.jaitools Another extension to Java Advanced Imaging this time thanks to Michael Bedward. Really fun for processing and generating rasters.

99