Apr 18, 2013 ISTeC Big Data Forum, Colorado State University 1

“Big Data” at theNatural Resource Ecology Laboratory

Randall B. Boone

Research Scientist, Natural Resource Ecology Laboratory andAssociate Professor, Department of Ecosystem Science and Sustainability

ISTeC Big Data ForumColorado State University

April 18, 2013

Department of EcosystemScience and Sustainability

Apr 18, 2013 ISTeC Big Data Forum, Colorado State University 2

Integrating across scales through

Top-down

and

Bottom-up

approaches

D. Ojima

Apr 18, 2013 ISTeC Big Data Forum, Colorado State University 3

Agent-based Modeling and Big Data

Apr 18, 2013 ISTeC Big Data Forum, Colorado State University 4

G-Range, A Global Rangeland Model

Apr 18, 2013 ISTeC Big Data Forum, Colorado State University 5

T. Hilinski

Apr 18, 2013 ISTeC Big Data Forum, Colorado State University 6

T. Hilinski

Potential Vegetation: Mean Annual NPP (gC/m2) 1961-2006 Mean

DayCent and Century Simulations

Apr 18, 2013 ISTeC Big Data Forum, Colorado State University 7

Managing Big Data at NREL

Our data are big, but well described

Unlike some industrial applications, the system must be exceedingly flexible

Must be responsive to a variety of users (e.g., more diverse uses than the ISTeC Cray)

Calculation-intensive uses

Users are unlikely to have the ability to parallelize tools

Apr 18, 2013 ISTeC Big Data Forum, Colorado State University 8

Rubel Cluster – The Backbone of NREL’s “Mid-Performance Computing”

Distributed memory computer cluster

256 processors

500 Gflops

463 GB memory

2 TB immediate storage

50-60 TB extended storage, plus 120 TB coming online

Private 1 Gbps Ethernet interconnect, with 10 Gbps access to storage

Apr 18, 2013 ISTeC Big Data Forum, Colorado State University 9

Rubel Cluster

Provides a testing platform for the CRAY for some

Others use alternatives, such as R, which can make use of multiple processors

Good, but not high performance computing

A mix of desktop and cluster analyses yield “mid-performance computing”

But it is effective!

Apr 18, 2013 ISTeC Big Data Forum, Colorado State University 10

Tom Hilinski

Apr 18, 2013 ISTeC Big Data Forum, Colorado State University 11

Rubel – Past, Present, and Future