Good modelling practice NGHS symposium 2014

Imagine the result

| 4 November 2014 | © ARCADIS 2014 Slide 2

| 4 November 2014 | © ARCADIS 2014 Slide 3

| 4 November 2014 | © ARCADIS 2014 Slide 4

| 18 December 2013 | © ARCADIS 2013 Dia 5

Motivation GMP

• Client requirement

• Certification requirement (Delft3D, ISO)

• Efficiency and cooperation

• Knowledge management

• Training of new employees

| 4 November 2014 | © ARCADIS 2014 Slide 6

Content

• Introduction

• Quality and quality improvement

• Good Modelling Practice (7 steps; model journal)

• Modelling Guidelines

• Version Control

• Discussion

| 4 November 2014 | © ARCADIS 2014 Slide 7

Introduction

• Models are standard tools

• Might be routinely applied but subjective

• Depends on team and skills

• Often referred to as “the art of modelling”

• But unscientific and ambiguous aspects

• Risks:

• Many choices

• Complex

• Incomplete knowledge

• Result: many uncertainties

| 4 November 2014 | © ARCADIS 2014 Slide 8

Quality

• Models are used habitually

• Easiness to use enlarges risk of injudicious

use, e.g.

• Careless handling input data

• Insufficient calibration and validation

• Use a model outside its scope

• No predefined requirements

• Result

• Unrepeatable

• Trustworthiness unknown

• Not matching predefined prerequisites

Quality control

Quality control:

“Something

adheres to

some criteria”

Without a properly defined

something there is no basis

whatsoever for quality control.

• date x • batch no. x • ingredients x • calories x • ℮ gram x

• date √ • batch no. √ • ingredients √ • calories √ • ℮ gram √

| 18 December 2013 | © ARCADIS 2013 Dia 10

Quality control flavours

• Documents (Imandra, VF)

• Project management (APM)

• Health & safety (risk form)

• Tools (SVN subversion)

• Software (idem)

• Models (…?)

| 18 December 2013 | © ARCADIS 2013 Dia 11

Need?

• Is there a need to check and trace model

quality?

• If there is:

To which extent check and trace?

| 18 December 2013 | © ARCADIS 2013 Dia 12

How to check and trace?

Check:

• Based on set-up?

• Based on results?

• Based on report?

• Check against what?

Trace:

• Personal communication?

• Verification form/Model Journal?

• Version Control?

| 4 November 2014 | © ARCADIS 2014 Slide 13

Quality management

• Quality management common in software

engineering

• However, modelling quality often restricted

to verification and validation

• Five maturity stages (Scholten et al., 2007):

• Ad hoc

• Repeatable

• Defined

• Managed

• Optimized

| 4 November 2014 | © ARCADIS 2014 Slide 14

Ontology of modelling • “An explicit specification of a conceptualization”

Gruber (1993, 1995)

• Defines modelling

• Represents shared knowledge which can be

reused

• How:

• Dividing in activities to build/test

intermediate products.

• Based on general agreement and

negotiation (Scholten, 2007)

| 4 November 2014 | © ARCADIS 2014 Slide 15

1. Start model journal

2. Describe problem Define objective Specify requirements Make plan

3. Define/build model Make conceptual model Choose software Choose discretization Implement model Verify model

4. Test/check model Check mass balances Test sensitivity

Calibrate Validate Determine uncertainty

5. Run model Make application plan Run model Check results

6. Describe results Discuss results Describe conclusions Check objective Analyse consequences for problem

7. Report and archive Check model journal and templates

Good Modelling Practice flow chart defines modelling process Use Modelling Guidelines as reference

| 4 November 2014 | © ARCADIS 2014 Slide 16

Modelling Guidelines • Introduction

• Data

• Bathymetry, Topography, Coastlines, Tidal

constituents, Discharges, Water levels, Wind,

Waves, Tsunamis, Sediment, Temperature and

salinity

• Model set-up

• Define model area, resolution and grid layout,

check grid quality, convert bathy samples,

interpolate bathy, schematize sediment, boundary

conditions

• Model calibration and validation

• (Technical) report

• See above

• + criteria for figures and tables

• + MATLAB tips and tricks

Quality control

Quality control:

“Something

adheres to

some criteria”

Without a properly defined

something there is no basis

whatsoever for quality control.

GMP flow chart (model journal)

and Modelling Guidelines

should be accompanied by

Version Control

• date x • batch no. x • ingredients x • calories x • ℮ gram x

• date √ • batch no. √ • ingredients √ • calories √ • ℮ gram √

| 18 December 2013 | © ARCADIS 2013 Dia 18

Windows Explorer

| 18 December 2013 | © ARCADIS 2013 Dia 19

Matlab

Description and version info

| 18 December 2013 | © ARCADIS 2013 Dia 20

Report

| 4 November 2014 | © ARCADIS 2014 Slide 21

Good Modelling Practice

Why:

• Client requirement

• Certification requirement (Delft3D, ISO)

• Efficiency and cooperation

• Knowledge management

• Training of new employees

How:

• Make Model Journal using GMP flow chart

• Use Modelling Guidelines as reference

• Use Version Control for models and scripts

Delft3D Flexible Mesh facilitates parts of the Model Journal and Version Control

| 4 November 2014 | © ARCADIS 2014 Slide 22

References

• Roelvink & Reniers (2012). A guide to modelling coastal morphology.

World Scientific.

• Scholten et al. (2007). A methodology to support multidisciplinary

model-based water management, Environmental Modelling &

Software 22 (2007) 743-759

• Refsgaard et al (2005). Quality assurance in model based water

management – review of existing practice and outline of new

approaches. Environmental Modelling & Software 20 (2005) 1201–

1215

• Scholten et al. (2000). Good Modelling Practice in water

management. HydroInformatics 2000, 23-27 July 2000, Cedar

Rapids, IA, USA

• Van Waveren et al. (1999). Good Modelling Practice Handbook.

STOWA report 99-05.

Imagine the result

Slide 23 | 4 November 2014 | © ARCADIS 2014