A Collaborative Approach A Collaborative Approach to Component-Based to Component-Based

Community Models and Community Models and ToolsTools

George Leavesley and Olaf George Leavesley and Olaf David Colorado State David Colorado State

University, USDA, Fort Collins, University, USDA, Fort Collins, COCO

2

Overview

• Definitions

• Technical and Social (Cultural, Institutional, ..) Issues

• The Object Modeling System (OMS)– Component development

– Model assembly

– Collaborative Component and Model Development (Colab)

– Component and model sharing (the Repository)

– Support components (visualization, calibration, sensitivity analysis, …)

– Selected applications

• Closing Remarks

3

Definitions

• Component-based modeling - based on the concept of

composing or assembling models out of independent but

pluggable parts– Model components support -

• REUSE: can be reused in other models

• CONSISTENCY: provide consistent results

• QUALITY: can be rigorously tested

• FLEXIBILITY: appropriate code for problem objectives, data

constraints, and space-time scales of application

• MULTI-DISCIPLINARY APPROACH: experts focus on their

discipline and interfaces among disciplines

• Community – Hydrologic/environmental science and all

related disciplines? Research and operational users?

4

Technical Issues

• Multiple system architectures (portability)

• Numerous modeling frameworks

• Multiple programming languages

• Legacy models and legacy code (wrap, compile, refactor …)

• Open source

• Continuing advances in software and hardware

• Need highly experienced and skilled computer scientists as

a part of the community

• …

5

Social Issues

• Acceptance and awareness of the advantages of community modeling frameworks

• Agreement on appropriate and desirable framework technologies

• Concurrence about terminology across disciplines

• Organizational scope

• Team skills

• Framework/language learning curve

• Rewarding reuse, benefit it will have others

• Crediting papers vs. code

• Technology diffusion – how innovations are accepted by successive subgroups

6

Object Modeling System

• Component based IDE

for

– Developing model

components

– Integrating

components into

models

– Running models

– analyzing model

results

– Managing modeling

resources

7

Object Modeling System (OMS)Modeling Projects

Component Library

Assembled Model

Parameter Editor

ComponentEditor Output

Analysis

IDE based on NetBeans

8

Generic Modeling System Schematic

Data IO

GUI

Time stepcomponent

Spatial unitcomponent

DataParameterHandling

time step iteration

spatial unit iteration

ETP

Inter-ception

Snow

Soil-water

Ground-water

Inter-flow

Base-flow

SurfaceRO

Irrigation

Erosion

Surfacewater use

Groundwater use

Plantgrowth

Stre

am

RO

Generic SystemComponents

ModelSetup

SensitivityAnalysis

Optimization

Process modulelibrary

ETP

Hydr.

GW

…

…

…

WQWQ

Irrig.

…

…

[Krause 2004]

Component Development

“Creating the model building

blocks”

10

What is a Component ?

• Encapsulates operations to realize one specific purpose

• Typically a ‘physical’ process.

• Component attributes provide the communication among

components

Model

ComponentComponent

Attribute

Snow

Snowmelt

Soil

11

Component Implementation

• Java Class that implements

– Executable,

– Stateful …

• Defines Attribute references

– Attribute.Double

– Attribute.Integer

– ..

• Defined Method structure

– initialization

– component logic

– cleaning up resources

– additional methods

init()

run()

cleanup()

Component Implementation

Attribute objects

12

Component API

13

Component Meta Data

• Component Name

• Annotations

– Description

– Keywords

– Version Info

– Source

– Author

– (Status)

14

Component Attributes

• Attribute Objects

– Type

– Name

• Attribute Annotations

– Description

– Access

– Role

– Unit

– Constraint

• Packages– org.oms.model.data

– org.oms.model.data.annotations

15

Native Components

• Can be implemented in FORTRAN,C, C++

• FORTRAN: automatic encapsulation of

subroutines/functions into a component.

• Embedded JNative library is able to call dynamic link

libraries.

Model Building

- Integrate Components into a

model

- Connect Component Attributes

17

Adding a Component

• Drag and

Drop a

component

into a model

• Where to

instantiate

the

Attributes?

• Scoping

18

Create a Model

• A model is a

hierarchical

assembly of

components

• Sequential

Execution order

• Model Wizard

19

Component Attribute Connectivity• Introduced by components

when they are added to the

model or explicitly declared

• Attributes get shared among

components

• Global data element in a model

• Primitive and aggregated data

structures

• Connections changeable

• Component Attribute

references point to Model

Attribute Objects

20

Component and Attribute Connectivity

21

Component Palette

• Controls execution of

components

• Control Components

– Sequential execution

– Parallel Execution

– Conditional Exec

– Iteration

– Temporal Execution

– Spatial Execution

• Repository Components

22

Time – Space – Process Configurations

Space

Time

ProcessTime

Process

SpaceTime

Space

Process

23

Time – Space – Process Configurations

TimeSpace

Space

Processes

Processes

24

Modeling Projects

• Workspace that contains all resources for developing and running a model– Documentation– Input Data– Output data– Model Components

• Component = behavior– Model Assemblies

• Model = components + attributes– Simulations

• Simulation = model + input data + execution scheme + selected output analyses

• Project level Actions

25

Collaborative Model SupportCollaborative Model Support

• 1300+ users

• 20+ orgs

• 350+ projects

• since 2005

• USDA/USGS/

EPA/ARS/CSU/..

26

Modeling Project Support in CoLab

• Issue Tracking for

Bugs & Tasks

• Document

Management

• Wiki Spaces

• Version Control

(Subversion)

• QA metrics

• Build Support

• Discussion Forum,

Email Integration

• Search and Indexing

27

USDA Collaborative Modeling Infrastructure

Model DevelopmentModel Development- Construct- Run- Analyze- Test- Verify

Model ProjectModel Project ManagementManagement - Tracker- Forums- Documents- CMM Reports- Access Control

ModelModel ResourcesResources Change ManagementChange Management- Version Control- Change Management- Repository- Concurrent Access

OMSOMSOMSOMS

SubversionSubversionSubversionSubversion

CodebeamerCodebeamerCodebeamerCodebeamer

Component Repository

“Towards the establishment of a standard library of interoperable, reusable science

and auxiliary modeling components”

OMS Component Repository

OMS Model

Component Repository

Library of reusable components each described by:- Set of attributes

- Meta data on key details to enable retrieval

Represent / Publish (push)

Search/Retrieve (pull)

30

Access CoLab Components directly from OMS

Connect

ExplorePalette

Use

31

Model Support and Analysis Components

32

Map OMS model input/output data into NASA World Wind

33

Interactive Model Interface Tools

ProjectBrowser

SimulationNavigator

Data Table

EquationBox

MapWindow

HRU 15Selected

Data Table Filters

TimeSlider

34

Analysis Tools

• Canned analysis configurations

• Analysis of output data

– Graphs

• TimeSeries Plot

• XY Error

• Flow duration

• Residuals

– Statistics

• Table IO/API

35

ESP Trace Analysis

• Ensemble

Streamflow

Prediction (ESP) Tool

• Statistical analysis

• Report generation

36

Hay and Umemoto, 2006, Multiple-Objective Step-Wise Calibration using Luca: U.S. Geological Survey OFR 2006-1323.

Multiple-Objective Step-Wise Calibration Tool Using SCE

LUCA -

37

Selected Current Applications

38

Initially:Initially: Water Quality Water Quality Water ConservationWater Conservation Soil QualitySoil Quality

Later:Later: Air QualityAir Quality Wildlife HabitatWildlife Habitat

Initially:Initially: Cropland (Rainfed)Cropland (Rainfed)

Later:Later: Cropland (Irrigated)Cropland (Irrigated) GrazinglandsGrazinglands Wetlands Wetlands Agro-forestry LandsAgro-forestry Lands

Resource Concerns Land Use CategoriesResource Concerns Land Use Categories

USDA Conservation Effects Assessment Project (CEAP)

A Cooperative Effort to Assess Environmental Effects and A Cooperative Effort to Assess Environmental Effects and Benefits from Conservation Programs at National and Benefits from Conservation Programs at National and

Watershed ScalesWatershed Scales

39

SoilClimate

Plant/Crop …

NRCS Data Warehouse

OMS Model Interface

PS1

Nutrients

Hydrology 1

Crop Growth 1

PS2PS1

Erosion WindLeaching

1

Hydrology 2

Crop Growth 2

Hydrology 2

Crop Growth 1

OMS Model A OMS Model B OMS Model C

ARS Science Module WarehouseOMS Model Engine

OMS Toolbox

Hydrology

ErosionCrop

Growth

Modules..

ARS/CEAP Goal regionalized models, databases, and scenarios for site-specific watershed

assessment studies and national assessments

40

Agricultural Water Supply Forecasting

(USDA – NRCS National Water and Climate Center)

41

Moving from a regression-based seasonal supply

forecast to a process-model based daily/weekly/monthly

forecast approach using OMS

Agricultural Water Supply Forecasting

Research Elements:

•Evaluate alternative climate scenarios in ESP

•Evaluate alternative precip distribution methods

•Develop an ensemble of models for forecasting

42

Next Steps

• Continue to add components to the shared library from

research, support, and applications users

• OMS 3.0 will enable scalability from 1CPU to multi-core to

cluster to cloud computing

• OMS runtime will be part of USDA SOA Business Enterprise

architecture

43

Closing Remarks

• OMS is one of several frameworks focused on “community” model development

• Regarding technical issues OMS –– Supports multiple platform applications, open source

– Supports multiple programming languages using industry standard language support tools (minimizing use of design specific languages, minimizing learning curve)

– Supports scalability across single to multiprocessor computing approaches

• Is there a universal framework concept acceptable to the “community” or a need to consider multiple framework concepts?

• Support from the Computer Science community is critical in community model framework development

44

Closing Remarks

• Regarding Social Issues

– Concensus on the CUAHSI community vision needed

– Workshops are a great start

– Need to address Technology Diffusion in the process of Community

Model standards adoption (See Dibner and Arctur, IEMSS 2008)

• Innovators, Early Adopters, Early Majority, Late Majority, Laggards

• Theory says chasm is between Early Adopters and Early Majority

– Include all the anticipated community disciplines in the design

workshops

– While research is a focus, don’t exclude the applications community