Esri UC 2014 | Technical Workshop | Data Interoperability: An Introduction Bruce [email protected] Dale LutzSafe Software

Esri UC 2014 | Technical Workshop |

Data Interoperability: An IntroductionBruce Harold [email protected]

Dale Lutz Safe Software

Esri UC 2014 | Technical Workshop |Esri UC 2014 | Technical Workshop |

Why Data Interoperability?

Background

ArcGIS + FME


What is the Data Interoperability Extension?

• Connect to Formats- 200+ added file formats

- Feeds & Web sources

- Custom formats

• Quick Translation- Any format to File GDB

- Any format to another Format

• Transformation- Schema transformation

- Geometry transformation

Connects your World of Apps

File, Photo, Parcel, BIM, MS Office, DBMS, CSV, Network, Military, Survey, Sensor, GPS…


ArcGIS Data Interop Common Tasks

• Integration

• Migration

• Cleanup

• Validation

• Distribution

• Merging Changes

Connected workflows

Spatial ETL


Data Interoperability DifferentiatorsFor Data Triathletes

• Feature by Feature Processing

• Inline SQL across all your data

• Data Inspection

• Deep format & standards support

• XML Documents as Features or Attributes

• JSON Documents as Features or Attributes

• Protocol Support (FTP, HTTP, RSS…)

• URL and Compressed Inputs

• Flow Control & Parallel Processing

• No Coding


What you get just by installing

Basic OGC Support


WFS and GML

• GML- Simple Features Profile

• WFS in Data Interoperability- Cache lives 24 hours by default

- Numeric Identifier forces live access

• You must install the extension

Supported without Licensing

OGC GML & WFS suitable for Geoprocessing

Type Presentation Title Here


Data Interoperability as an OGC client

OGC GML & WFS

Demo

Data Interoperability - An Introduction


Other OGC Stuff

• KML- Schema object preserved

- KML Properties

- KML Styling

• CityGML- CityEngine Tools coming

- To & From File GDB

But wait there’s more

OGC Formats in Data Interoperability



Ways to read any format data

Using Formats


Direct Read/Write From Disk

• Catalog browse tree- No need to add file type

- Feature Dataset icon

- Available feature types shown

- Format defaults used

• As geoprocessing input- Tools

- Models

- Scripts (enable the extension!)

File Based – recognized by extension

Recognized formats ‘just work’



Interoperability Connections

• When default format connection is:- Infeasible, e.g. authentication required

- Undesirable, e.g. optional parameter wanted

• Authored in Catalog tree

• Off-disk feeds & web sources

• May view source data prior to connecting

• Not cached, read when used

• Coordinate system may be selected

• May be shared (.fdl file)

Manually set up and persistent

Yes, Excel can be read as Point data!



Selecting Formats

• Browse

• Search

• Customize

Or, ‘What was that “FME Reader Gallery” thing in the previous slide?’

Don’t see what you’re looking for? Ask us!



Using any format in ArcGIS for Desktop

Direct Read & Connections

Demo



Quick Import & Quick Export GP Tools

Quick Translation


Quick Import & Quick Export

• Quick Export- To Any Format

• Quick Import- To File Geodatabase

Geoprocessing tools in Data Interoperability

Rapid format conversion using format defaults



Automate Quick Translations

• Easy way to ship multiple formats

• Retain your GIS schema

Use the Quick tools in ModelBuilder

Supply popular formats with GIS-friendly properties



Automated Format Supply

Quick Tools

Demo



Custom Formats

Leveraging Formats


What if you need your own format?

• Based on a known format

• Build format definition workflow once

• Remains available in the gallery

Save time by “canning” a custom format workflow

You can share a custom format with other Data Interoperability users

Based on JSONBased on JSON


JSON read from the web

Custom Formats

Demo



Spatial ETL

The Power of Transformation


The Full Power of Spatial ETL

• Format reading and writing is only a beginning- Information has no ‘format’

- Formats are only languages; many have dialects

- Formats merely facilitate information (or wreck it)

• Information is often hidden or requires business logic

• Transformation reveals and derives information- Calculate stuff, mess with words, ignore junk

- Use relational, dictionary, list & network structures

- Way, way more…

• Stop. Writing. Code. (Unless you find it fun)

Connect. Transform. Automate.

Transformation is the real power of Data Interoperability


Workbench Application

• Create via context menu

• Follow the wizard

Graphically Design Data Flows

The Workbench application authors Spatial ETL tools


Workbench Authoring

• Read

• Transform- Attributes

- Geometry

- Style

• Write

• Run standalone

• Run as tool

Design & Document

Workbench has a canvas and supporting windows



Transformers

• Eponymous- Named after their function

• Categorized

• Browse

• Search

• Popup

Hundreds of functions at your fingertips

Drag or click transformers onto the canvas or use the popup functionality



Connect. Transform. Automate.

ETL

Demo



A few more observations…

Wrapping Up


Data Interoperability in ArcGIS for Server

• Map services can use supported formats- 64 bit only

• Spatial ETL tools must be in Models- Publish service as usual from Result

• Good Zip & Ship solution- Generic Writer into ZIP file

- Output format made parametric

Considerations

Data Interoperability is available in ArcGIS for Server



Coming in 10.3

• FME 2014 SP 2 Engine

• Write images & hyperlinks to Excel (maps!)

• Data Interoperability for ArcGIS Pro- 64 bit

• More…

And it’s service packs…

Data Interoperability will be released for ArcGIS Pro



Learn More

• At the User Conference- Esri Showcase Tuesday 9am until Thursday 1:30pm

- After this presentation

• 60 Day Software Evaluation- http://www.esri.com/software/arcgis/extensions/datainteroperability/evaluate

• Training- http://training.esri.com/gateway/index.cfm?

fa=search.results&searchterm=data+interoperability

Data Interoperability

Talk to us now!



Thank You! Now its Question Time!

• Please fill out the session survey:

First Offering ID: 1221

Second Offering ID: 1402

Online – www.esri.com/ucsessionsurveys

Paper – pick up and put in drop box

Data Interoperability – An Introduction

Have a great conference!


Esri UC 2014 | Technical Workshop | Type Presentation Title Here