25
BIG DATA MANAGEMENT WITH APOLLO TONY KENT, IMGS

IMGS Geospatial User Group 2014 - Big data management with Apollo

  • Upload
    imgs

  • View
    65

  • Download
    1

Embed Size (px)

Citation preview

Page 1: IMGS Geospatial User Group 2014 - Big data management with Apollo

BIG DATA MANAGEMENT WITH APOLLOTONY KENT, IMGS

Page 2: IMGS Geospatial User Group 2014 - Big data management with Apollo

Data Deluge

Geospatial data volumes

are growing at an ever

increasing rate…

Faster than processing and

storage costs are falling

Page 3: IMGS Geospatial User Group 2014 - Big data management with Apollo

Data Deluge

Greater Resolution, Greater Coverage, Greater Frequency

Page 4: IMGS Geospatial User Group 2014 - Big data management with Apollo

Data Deluge

New Data Sources

Page 5: IMGS Geospatial User Group 2014 - Big data management with Apollo

Data Deluge

New Data Sources

Page 6: IMGS Geospatial User Group 2014 - Big data management with Apollo

Rising User Demand

Every day, new users are

demanding imagery, across

all applications, on a variety

of devices

Page 7: IMGS Geospatial User Group 2014 - Big data management with Apollo

Summary of Market Drivers

Increasing need to leverage huge amounts of

geospatial data

Dynamic discovery of dispersed information

Need for quicker access

Multi-purpose data

Page 8: IMGS Geospatial User Group 2014 - Big data management with Apollo

Increasing pressures

Data managers

Infrastructure

Procurement

Users

Performance

Page 9: IMGS Geospatial User Group 2014 - Big data management with Apollo

Need for data management

Page 10: IMGS Geospatial User Group 2014 - Big data management with Apollo

ERDAS APOLLO

A comprehensive data management, analysis and delivery system

enabling an organization to describe, catalog, search, discover,

process and securely disseminate massive volumes of data.

Page 11: IMGS Geospatial User Group 2014 - Big data management with Apollo

Solving the “Bounding Box”

problemGiven a bounding box in space, and a given time period, discover and access

all available, relevant & authorized information within that area.

Geospatial Data Types

Maps, Imagery, Features, Terrain, Place

Names, Buildings, Infrastructure, Roads,

Political Boundaries, Hydrographic,

Geodetic, etc.Location References in

Structured Data

Relational Databases, Travel Itineraries,

Financial Transactions, Corporate Data,

Personnel Records, Statistical Data, etc.

Sensor Data

EO, Spectral, Radar,

LiDAR, Infrared, FMV, in

situ, GPS, etc.

Access from Any Device

Desktop, Laptop, PDA, Wireless,

Smartphone

Location References in

Unstructured Data

News Reports, Publications, Manifests,

Internet, World Wide Web, Audio, Video, etc.

Page 12: IMGS Geospatial User Group 2014 - Big data management with Apollo

Wizard-driven Workflow to

Catalog and Publish

Search any file system visible

to the server for data.

Set schedulers to repeat the

search on a given schedule

as a one-time event or on a

recurring basis.

Process and service-enable

the datasets.

Set security to control access.

Restrict by user, resolution

and location.

Page 13: IMGS Geospatial User Group 2014 - Big data management with Apollo

ECW IMAGE COMPRESSION

Page 14: IMGS Geospatial User Group 2014 - Big data management with Apollo

ECW Technology

ENCODING

SPEED

DECODING

SPEED

IMAGE

QUALITY

FILESIZE

TIME

SAVED

REDUCED

STORAGE

COSTS

ENHANCED

USER

PERFORMANCE

REDUCED

DATA

MANAGEMENT

The Power of ECW

Page 15: IMGS Geospatial User Group 2014 - Big data management with Apollo

Case Study: Big Data Made Small

The World’s Largest

Geospatial Image?

A single aerial image covering

Germany @ 20cm GSD

3,210,000 px by 4,340,000px

Big Data Made Small

38,000gb Uncompressed

50,000gb with image pyramids

370,000 source files

1 ECW file

875gb ECW Compressed

Page 16: IMGS Geospatial User Group 2014 - Big data management with Apollo

Enhanced Compression Wavelet (ECW)

Compresses bulky imagery files

into manageable sizes while preserving

their visual quality

Page 17: IMGS Geospatial User Group 2014 - Big data management with Apollo

ECW Protocol (ECWP)

world’s fastest streamingof ECW and JPEG2000 imagery

Delivers the

Page 18: IMGS Geospatial User Group 2014 - Big data management with Apollo

ERDAS APOLLO: LiDAR Workflow

Requirement to manage large numbers of LAS files.

Organization needs to:

Catalog

End users need to:

Discover

View/preview

Analyze

Download

Subset & Mosaic

Page 19: IMGS Geospatial User Group 2014 - Big data management with Apollo

Why treat LAS like a Raster

(Conversion to grid)Why treat LAS like raster?

Easy viewing

Easy distribution

Because LiDAR is so dense and

semi-regular it can be gridded quickly

and automatically

Grid automatically

Available via

WMS, WCS and WPS

Download or CZS

Raster

LiDAR

Page 20: IMGS Geospatial User Group 2014 - Big data management with Apollo

Data Manager – Catalog LAS

Page 21: IMGS Geospatial User Group 2014 - Big data management with Apollo

Data Manager – Edit Metadata

Page 22: IMGS Geospatial User Group 2014 - Big data management with Apollo

Geospatial Portal –

Discover LAS data

Page 23: IMGS Geospatial User Group 2014 - Big data management with Apollo

Clip, Zip and Ship LAS

CZS LAS-formatted point cloud data

Select clip area

Select classifications

Filter by return value

Output to LAS, IMG or TIFF

Clipped output LAS

Region

to subset

Page 24: IMGS Geospatial User Group 2014 - Big data management with Apollo

LAS Management in

Apollo 2014

Page 25: IMGS Geospatial User Group 2014 - Big data management with Apollo

2015 Geoprocessing –

Delivered from the Cloud