Syracuse University Syracuse University

SURFACE SURFACE

Theses - ALL

May 2019

Doing a Lot with a Little: A Diagnostic Analysis of SWMM to Doing a Lot with a Little: A Diagnostic Analysis of SWMM to

Simulate Hydrologic Behavior within LID Systems Simulate Hydrologic Behavior within LID Systems

Lucie Lynn Worthen Syracuse University

Follow this and additional works at: https://surface.syr.edu/thesis

Part of the Engineering Commons

Recommended Citation Recommended Citation Worthen, Lucie Lynn, "Doing a Lot with a Little: A Diagnostic Analysis of SWMM to Simulate Hydrologic Behavior within LID Systems" (2019). Theses - ALL. 321. https://surface.syr.edu/thesis/321

This Thesis is brought to you for free and open access by SURFACE. It has been accepted for inclusion in Theses - ALL by an authorized administrator of SURFACE. For more information, please contact surface@syr.edu.

Abstract

Low Impact Development (LID) aims to mitigate the hydrological impacts of urbanization by

promoting evapotranspiration, storing and slowing the flow of water in formerly impervious

areas. Green roofs, a form of LID often utilized in highly urbanized watersheds, are widely

simulated using the Storm Water Management Model (SWMM). However, methods to improve

diagnostic analysis of SWMM have lagged compared to other environmental disciplines. In this

study, I utilize frugal diagnostic analyses to investigate potential sources of non-linearity,

uncertainty, and equifinality within SWMM applied to a particular case study, the OnCenter

green roof in Syracuse, New York. My findings highlight the major sources of uncertainty in

SWMM – model inputs, parameters, structural equations, and reconciling differences between

simulated outputs versus observed variables – and demonstrate that more complex diagnostic

analysis is necessary to fully understand the fundamental drivers of, and interactions amongst,

uncertainty in the SWMM LID bioretention module. As SWMM contains many parameters and

therefore multiple degrees of freedom, sensitivity analyses performed using one-at-a-time tests

highlight that these analyses are only local estimates within a neighborhood of the selected

parameter set. Though we could achieve strong agreements between simulated and observed

runoff, SWMM was not able to replicate observed storage timeseries during simulation,

suggesting that common approaches to calibrate only to periods of precipitation may

misrepresent key hydrologic storages and fluxes within the model. While information gained

from frugal analyses can aid in SWMM calibration, the approaches we’ve used oversimplify

complex hydrological processes in an extremely non-linear model, limiting their effectiveness as

diagnostic tools. The development of a more flexible model structure that allows for complex

diagnostic analysis is necessary to fully understand the fundamental drivers of uncertainty in the

SWMM LID bioretention module. Encouraging the co-production of knowledge through

mutually beneficial dialog between researchers and practitioners presents an opportunity to

accelerate SWMM model improvement.

Doing a Lot with a Little: A Diagnostic Analysis of SWMM to Simulate

Hydrologic Behavior within LID Systems

By

Lucie Worthen

B.S. University of New Hampshire | Durham, NH, 2013

Thesis

Submitted in partial fulfillment of the requirements for the degree of

Master of Science in Environmental Engineering

Syracuse University

May 2019

Copyright © Lucie Worthen, April 26, 2019

All rights Reserved

iv

Table of Contents

1 Introduction ............................................................................................................................... 1

2 Case Study ................................................................................................................................. 5

2.1 Study Site ............................................................................................................................ 5

2.2 Experimental Data Collection ............................................................................................. 6

3 Methods...................................................................................................................................... 6

3.1 SWMM Model .................................................................................................................... 6

3.2 Model Setup ........................................................................................................................ 9

3.3 Model Calibration ............................................................................................................. 10

3.4 Diagnostic Analysis of SWMM ........................................................................................ 12

3.4.1 Initial Conditions ....................................................................................................... 13

3.4.2 Parameter Equifinality ............................................................................................... 13

3.4.3 Parameter Sensitivity ................................................................................................. 14

4 Results ...................................................................................................................................... 16

4.1 Calibration and Assessment of Model Performance ......................................................... 16

4.2 Diagnostic Analysis of SWMM ........................................................................................ 16

4.2.1 Equifinality ................................................................................................................ 16

4.2.2 Initial and Best Fit Parameter Set Sensitivity Analyses............................................. 17

4.2.3 Initial Soil Moisture Analysis .................................................................................... 17

4.2.4 Considering Multiple Error Metrics........................................................................... 18

v

5 Discussion ................................................................................................................................ 19

5.1 Assessment of SWMM Bioretention Module ................................................................... 19

5.2 How can and why should we aim to prove model realism within applications of SWMM? . 20

5.3 What can we learn from and how can we improve SWMM sensitivity analyses? ........... 22

5.4 How can we and why do we need to improve SWMM calibration? ................................ 24

5.5 How do we best incorporate new sources of information into the model evaluation process? .... 26

6 Conclusions .............................................................................................................................. 28

7 Tables ....................................................................................................................................... 31

8 Figures...................................................................................................................................... 33

9 References ................................................................................................................................ 39

10 CV ........................................................................................................................................... 46

vi

List of Tables

Table 1. Characteristics of rainfall events......................................................................................31

Table 2. SWMM Parameters: Initial and best fit values for simulations .......................................31

Table 3. Parameter interactions between saturated hydraulic conductivity and porosity ..............32

vii

List of Figures

Figure 1. Conceptual diagrams of study site and SWMM model processes .................................33

Figure 2. Flowchart of frugal sensitivity analysis approach ..........................................................33

Figure 3. Timeseries from best fit parameter set for October, May, and June events ...................34

Figure 4. Parallel coordinate plot and runoff timeseries for parameter interactions .....................35

Figure 5. Radar plots for OAT sensitivity analyses .......................................................................36

Figure 6. Timeseries for initial conditions cases ...........................................................................37

Figure 7. Time series for porosity-based soil moisture content comparisons................................38

1

1 Introduction

Urbanization strongly modifies land cover, which subsequently alters various processes within

the hydrological cycle (Arnold & Gibbons, 1996). In comparison to forested watersheds, urban

watersheds typically exhibit larger surface runoff volumes, higher peak flows, reduced

infiltration, decreased transpiration and more severe pollutant loads (Bannerman, Owens,

Dodds, & Hornewer, 1993; Booth & Jackson, 1997; Haase, 2009; Jefferson et al., 2017; Lee &

Heaney, 2003; Ruth, 2003). One alternative to mitigate the environmental impacts of urban land

use is to utilize Low Impact Development (LID) systems. LIDS replicate predevelopment

hydrological functions (Stovin, Vesuviano, & Kasmin, 2012; US EPA, 2000), and provide many

additional ecosystem services, including removing air and runoff pollutants (Bianchini &

Hewage, 2012; J. Li et al., 2010), mitigating urban heat islands (Blanusa et al., 2013; Susca,

Gaffin, & Dell’Osso, 2011), and increasing urban biodiversity (Cook-Patton & Bauerle, 2012;

Francis & Lorimer, 2011). Green roofs, a form of LID, manage stormwater directly at the source,

by retaining (i.e., reducing volume) and detaining (ie., shifting peak intensity) stormwater runoff

(Y. Li & Babcock, 2014; Anna Palla et al., 2010; Stovin, Poë, De-Ville, & Berretta, 2015).

Green roofs are a particularly attractive stormwater management option in dense urban areas,

where rooftops in developed nations account for roughly 40 to 50% of urban impervious surfaces

(Dunnett and Kingsbury 2008).

To date, numerous monitoring studies have been conducted to understand hydrologic

processes within green roofs and to predict how much water they may retain under extreme

scenarios (Y. Li & Babcock, 2014; Nawaz, McDonald, & Postoyko, 2015; Stovin et al., 2012).

The latter can indicate the impact of green roofs on sewershed or city-wide stormwater

management, and permits predictive extrapolation beyond observations and beyond a specific

2

period of record (Cipolla, Maglionico, & Stojkov, 2016). Modeling green roof behavior (i.e.,

fluxes and storages) and performance (i.e., runoff volume and peak intensity) is especially

important given that green roof configuration and climate can lead to very different magnitudes

of rainfall capture, and therefore may alter the timing and magnitude of green roof runoff

(Carson, Marasco, Culligan, & Mcgillis, 2013; Mentens, Raes, & Hermy, 2006; Spolek, 2008).

Among commercial models, the Storm Water Management Model (SWMM) is the most

commonly used hydrologic and hydraulic model by researchers and practitioners alike. SWMM

has historically been applied to simulate the quality and quantity of runoff from urban

subcatchments. As the need for understanding and predicting performance within individual LID

systems has grown, SWMM has been modified to include more specific modular frameworks for

different types of LID forced by both long-term and single-event simulations (Burszta-Adamiak

& Mrowiec, 2013; Rossman, 2008). SWMM (version 5.1.012) (Rossman, 2017) currently

contains a green roof module (Y. Li & Babcock, 2014) and a bioretention module (Burszta-

Adamiak & Mrowiec, 2013) capable of simulating green roof hydrologic behavior, with

validation studies across the literature (Cipolla et al., 2016; Krebs, Kuoppamäki, Kokkonen, &

Koivusalo, 2016; Anna Palla & Gnecco, 2015; Peng & Stovin, 2017). Beyond SWMM, the

simulation of hydrologic behavior within green roofs and other forms of LID has been performed

with many different types of models, including characteristic runoff equations, HYDRUS-1D,

PARFLOW, HELP, and others (Carson, Keeley, Marasco, McGillis, & Culligan, 2017; Fassman-

Beck, Voyde, Simcock, & Hong, 2013; Hilten, Lawrence, & Tollner, 2008; Lim & Welty, 2017;

A. Palla, Gnecco, & Lanza, 2012; She & Pang, 2010; Voter & Loheide, 2018). Though there are

limitations to its use, SWMM remains one of the most popular tools for simulating green roofs

among engineering practitioners, likely due to its historical legacy in urban hydraulic and

3

hydrologic applications. Though popular, SWMM LID modules are relatively new compared to

the lifespan of the SWMM model, and have primarily been used in a ‘calibration-validation’

framework, with no formalized recommendations for best practices. Altogether, this has resulted

in many studies that focus on demonstrating that SWMM LID modules can realistically simulate

a green roof in a given place, but few critiques of the strengths and limitations of the SWMM

model. This means that research related to SWMM LID modules does not move beyond this

approach.

Like any model, SWMM is subject to numerous uncertainties, including uncertainty in

model inputs, parameters, and structural equations, in addition to the uncertainty introduced by

abstractions between observed and simulated variables. In this vein, studies that have applied

SWMM to simulate green roof performance and behavior have recognized several important

limitations. First, several studies have noted that SWMM parameters are conceptual rather than

physically-based, meaning that field-based measurements of parameters may fail to depict

observations when used as direct parameter values in the modeling framework (Alfredo,

Montalto, & Goldstein, 2010; Burszta-Adamiak & Mrowiec, 2013). This is especially important,

because as-built drawings with LID technical specifications and field measurements are often

used as initial parameter inputs of SWMM models. If these parameters are not calibrated, the

resulting simulations are likely to be unrepresentative of observations. Secondarily, because

SWMM is often calibrated manually, researchers and practitioners rarely perform sensitivity

and/or uncertainty analysis (Burszta-Adamiak & Mrowiec, 2013; Carson et al., 2017; Cipolla et

al., 2016). We do note this is changing, but slowly. To our knowledge, only a few recent studies

have attempted to explore the SWMM LID parameter space and model structure (Krebs et al.,

2016; Peng & Stovin, 2017).

4

Diagnostic modeling approaches have long been applied to improve understanding of

conceptual hydrologic and water quality models (Herman, Reed, & Wagener, 2013; Kelleher,

Mcglynn, & Wagener, 2017; Tang, Reed, Wagener, & Van Werkhoven, 2007; Wagener,

McIntyre, Lees, Wheater, & Gupta, 2003). Model diagnostic analyses are rooted in information

theory and test theoretically relevant relationships among model performance, parameters, and

initial conditions, including the testing of hypotheses about system structure (Gupta, Wagener, &

Liu, 2008). These methods allow the modeler to explore the relations between different types of

data and the processes simulated in the model, with the goal of improving the modeling process

and representation of hydrologic behavior through computationally frugal methods (Foglia, Hill,

Mehl, & Burlando, 2009; Hill et al., 2016). These approaches are critical to developing useful

models of complex hydrologic systems for which important characteristics cannot be measured

accurately or completely enough to define model input values prior to calibration (Gooseff,

Bencala, Scott, Runkel, & McKnight, 2005; Scott, Gooseff, Bencala, & Runkel, 2003). We aim

to bring diagnostic analyses to the SWMM LID literature with the goal of improving calibration-

validation approaches. We also aim to show how sensitivity and uncertainty analysis can

complement this approach, to provide a wealth of information that will facilitate more direct

comparison of results across studies and contextualize the information obtained during

calibration-validation.

In this manuscript, we assess a SWMM LID bioretention model using the current

SWMM model framework and diagnostic analysis that have been explored when using

parsimonious watershed models to deliver conclusions around model uncertainty and sensitivity.

We test three potential sources of sensitivity and uncertainty related to model calibration applied

to a particular case study, the 1.5-acre OnCenter extensive green roof in Syracuse, New York.

5

First, we analyze the role of antecedent moisture in relation to calibration. Second, we utilize two

sensitivity analyses with the goal to reduce overparameterization and improve understanding of

controlling system characteristics (represented as parameters), as well as to assess how

sensitivity may vary at different points in the SWMM parameter space. Third, we consider

calibrating to complex hydrological processes (i.e. runoff vs soil moisture). From these analyses

we provide recommendations to standardize current approaches to SWMM with an eye to frugal

approaches to sensitivity and uncertainty analysis.

2 Case Study

2.1 Study Site

The study site is an extensive green roof located on the Nicholas J. Pirro Convention Center

(OnCenter) in Syracuse, New York (43.04368N, 76.14824W). Syracuse climate, as relevant to

this work, is detailed in Squier and Davidson (2016). The green roof was retrofit onto the

existing structure in 2011. The roof is rectangular, with a geometry of 111 m North-South and 50

m East-West. As is typical, the roof has a peak running North-South midway between the East

and West walls of the building, to help direct runoff. The roof is sloped at 1% in both directions

from its peak. Vegetation, established by sprayed cuttings, includes species of Sedum and

Phedimus.

The roof’s structure includes drain conduits and a drainage mat designed to convey

excess water from the substrate. There are 12 roof drains along both the east and west sides of

the building. Perforated triangular drain conduits, 5.1 cm tall, begin 5.1 meters from the

centerline of the roof and run diagonally to each roof drain, as illustrated by Figure 1A. The

mineral-based substrate was sprayed onto the roof with an average 7.6 cm depth. The substrate

6

and drainage layer are underlain by a geosynthetic fiber drainage mat, a single-ply waterproofing

membrane, and traditional roofing structure. More detail on roof layers is given in Squier and

Davidson (2016).

2.2 Experimental Data Collection

A CR1000 datalogger (Campbell Scientific, Logan, UT) and 2 AM 16/32B multiplexers were

used to collect data from hydrologic, meteorological, and thermal instrumentation at 5-minute

intervals. Rainfall was measured by a tipping bucket (TE 525, Campbell Scientific, Logan, UT)

secured to a tripod approximately 29 meters from the southern end of the roof. Runoff from a

1708 m2 region of the roof was collected from the eight drains in the southeastern portion of the

roof and measured using an electromagnetic flowmeter (M2000, Badger Meter, Milwaukee, WI).

Soil moisture sensors (CS616, Campbell Scientific, Logan, UT) were installed along a transect to

measure the change in moisture content of the substrate. Meteorological instruments located on

tripods south of the center of the roof, were sourced through Campbell Scientific. Data were

collected between 4/1/15 and 10/31/18, but this study focuses on data from 4/1/17 and 10/31/17.

3 Methods

3.1 SWMM Model

The EPA Storm Water Management Model (SWMM version 5.1.012) is a dynamic hydrology,

hydraulic, and water quality simulation model that can be used for both single-event or

continuous simulation (Rossman, 2015). The LID controls in SWMM are designed to

specifically model LIDs, such as green roofs. The LID framework tracks moisture balances in

one-dimension between three different vertically-stacked layers (surface, substrate, and drainage

layer), as illustrated by Figure 1B, that are defined by parameters in a graphical user interface

7

(GUI). For green roofs with conduit drainage systems, it is recommended to use the SWMM LID

bioretention module instead of the SWMM LID green roof module and restrict infiltration

through the bottom of the storage layer, therefore creating an impermeable barrier like a green

roof system. This approach was used in this study (Burszta-Adamiak & Mrowiec, 2013).

For SWMM LID bioretention modeling, six equations are used to describe the processes

across the three model layers. Bottom exfiltration is not considered in this study. While input

parameters and variables, and output variables are reported in SI metric units, within the model

computer code all calculations are carried out using feet as the unit of length and seconds as the

unit of time. Therefore, the units for model process equations are discussed according to model

methods. A routing equation [Eq. (1)] is used to quantify surface runoff (q1, [ft/sec]) according to

�� = max ���∆ , 0� (1)

where it is assumed that any ponded surface water in excess of D1 becomes immediate overflow,

and d1 represents depth of water stored on the surface [ft] all over a model timestep ∆� [sec]. The

Green-Ampt infiltration model [Eq. (2) and Eq. (3)] is adopted to calculate how much water

infiltrates into the subsurface, following

� = ��� �1 + ����������� � (2)

and

= ��� + �!" − $��%� + &�'( �1 + ������������ (3)

8

where the infiltration rate (f, ft/sec) and the cumulative infiltration volume per unit area over a

storm event (F, ft) only apply after a saturated condition develops at the top of the soil zone.

Prior to this, all inflow from rainfall and runoff from impervious areas infiltrates. The initial

value of moisture content ($, volume of water/total volume of soil) for a dry soil would be its

residual moisture content or its wilting point (θWP, volume of wilting point water/total volume of

soil). Ksat represents the saturated hydraulic conductivity of the soil [ft/sec], Ψ represents the

suction head at the infiltration wetting front formed in the soil [ft], and !" represents the porosity

of the soil [void volume/total volume of soil] or the saturated moisture content. A variation of

Darcy’s Law [Eq. (4)] is used to model the rate of percolation of water through the soil layer into

the storage layer, according to

�) = *��� exp-−�!" − $�./, $ > $�1 0, $ ≤ $�1

(4)

where, when θ drops below the field capacity (θFC, volume of field capacity water/total volume

of soil), percolation rate (�), ft/sec) becomes zero. S represents the conductivity slope [unitless]

or decay constant derived from the moisture retention curve data that describes how conductivity

decreases with decreasing moisture content. A simple empirical power law [Eq. (5)] is used to

simulate storage layer underdrain flow (q3, ft/sec), following

�4 = 5�ℎ4�7 (5)

where the hydraulic head seen by the underdrain (h3, ft) varies with the height of water above the

underdrain, the underdrain discharge exponent (b, unitless) is set to 0.5 to set the drain flow

formula equivalent to the standard orifice equation, and the underdrain discharge coefficient (C,

ft-(b-1)/sec) incorporates both the normal orifice discharge coefficient and available flow area.

9

There is no underdrain flow until the depth of water in the storage layer reaches the drain offset

height (Hd, ft). Potential evapotranspiration (PETH, ft/sec) is simulated following the Hargreaves

equation [Eq. (6)] following

89:; = 0.0023 ?@AB C :D

��:� + 17.8� (6)

where Tr represents the average daily temperature range for a period of five days [deg C], Ta

represents the average daily temperature for a period of five days [deg C], Ra represents the water

equivalent of incoming extraterrestrial radiation [MJm-2d-1], and λ represents the latent heat of

vaporization [MJkg-1] or 2.50-0.002361Ta. The calculation of PETH proceeds from the surface

layer downwards during dry periods. Any unused PETH is made available to the next lower layer.

Daily minimum and maximum ambient air temperature gathered on the OnCenter green roof and

the geospatial location of the roof were supplied as climate data to calculate daily PETH.

3.2 Model Setup

The green roof was modeled in SWMM as a subcatchment that was 100% occupied by the green

roof with a single outlet. The dimensions of the subcatchment were 111 m width by 50 m length;

this matched the actual OnCenter green roof dimensions and flow path, as the modeled water

flow path is perpendicular to the width. The SWMM bioretention module was used to generate

runoff in response to three rainfall events. The internal simulation time step was set to 1 second

with a reporting time step of 5 minutes to match observed data. This method followed standard

green roof hydrology evaluation SWMM model setup (Carson et al., 2017; Cipolla et al., 2016;

Peng & Stovin, 2017).

10

3.3 Model Calibration

The goal of calibration is to determine if SWMM can represent in situ observations. As is typical

with SWMM simulations of green infrastructure, we set initial green roof parameters to realistic

values representative of (or equal to) in situ estimates from this particular system. Many initial

green roof parameter values were estimated from field measurements by Squier and Davidson

(2016), Yang and Davidson (2017), and laboratory soil measurements from the Penn State

Agricultural Analytical Services Lab (AASL). Additional parameters were based on initial

modeling studies at this site performed by Jacobs Engineering, with inputs selected by Jacobs

Engineering based on OnCenter green roof design specifications and as-built drawings. The

values and sources for each parameter utilized in the SWMM LID bioretention module are

presented in Table 2. As SWMM is often treated as a physically-based model, where real-world

estimates can be translated into model estimates, our initial goal was to test whether these

parameter estimates yielded accurate simulations of runoff, or whether calibration was required.

To identify storms for calibration, we identified 105 rain events from 4/1/17 to 10/31/18

and selected three events to incorporate in our model assessment. For each event, the percent

initially saturated was determined from the average of the three soil moisture sensors and

converted based on the SWMM definition of soil saturation. Characteristics of these events are

summarized in Table 1. These three rainfall events selected intentionally varied in antecedent

dry weather period, time of year occurrence, duration, intensity, and return period, to test the

calibration under different scenarios. First, we calibrated the SWMM model to an event

beginning on October 28, 2017, which has the greatest yearly return period. To assess whether

the calibrated parameter set was capable of reproducing conditions for other storm events, we

11

also used this parameter set to simulate two events, one beginning on May 1 and one beginning

on June 5, 2017.

Model performance was assessed between observed and simulated green roof runoff

using the Nash-Sutcliffe model efficiency (NSE) coefficient (Nash & Sutcliffe, 1970). While

NSE is a common error metric used in the hydrologic literature (Carson et al., 2017; Peng &

Stovin, 2017; Razavi et al., 2010), it is biased towards identifying a best fit parameter set that fits

peak flows due to its use of squared residuals (Legates & McCabe, 1999). Though use of NSE is

debated for this reason (Foglia et al., 2009; Gupta, Kling, Yilmaz, Martinez, & Kling, 2009;

Wagener et al., 2003), this analysis follows common SWMM calibration methods that use this

error metric to assess model performance (Carson et al., 2017; Peng & Stovin, 2017). NSE for

each event was calculated according to

G.9 = 1 − H IJ KLMLNO�

IJ �LMLPM��Q (7)

where N represents the number of samples, Qm is runoff observed, QP is modeled runoff, and

QAm is the mean observed runoff. An NSE value equal to 1 indicates perfect agreement between

simulated and observed values, while an NSE value greater than 0.5 is an indication of

acceptable model performance (Peng & Stovin, 2017; Rosa, Clausen, & Dietz, 2015).

In most SWMM exercises, calibration from initial parameter estimates is necessary to

match observed runoff. Standard SWMM calibration techniques follow a formulaic strategy

(Carson et al., 2017; Cipolla et al., 2016; Peng & Stovin, 2017). Model predictions were refined

through a manual calibration process, which is typical of SWMM applications. Manual

calibration is performed through a trial-and-error procedure, often using a number of different

measures of performance and visual inspection of the hydrograph (Gupta, Sorooshian, & Yapo,

12

1998; Wagener et al., 2003). In our exercise, parameters were adjusted one at a time until the

difference between measured and simulated values was minimized. Parameters describing

substrate depth, area, and width were held constant, given these properties describe static

properties of the green roof. All other surface layer, soil layer, and storage layer parameters were

allowed to vary, given these properties can change over time or are cannot be directly measured.

The model was calibrated manually by first changing the (most sensitive) parameters,

determined from an initial parameter set sensitivity analysis, to find good NSE model fit and then

fine-tuning model fit with less sensitive model parameters. The resulting parameter set with the

highest NSE across our manual search of the parameter space is hereafter referred to as the ‘best

fit parameter set’.

3.4 Diagnostic Analysis of SWMM

Regardless of their structural equations and purpose, environmental models like SWMM are all

simplified mathematical representations of physical systems (Kelleher et al., 2013; Wagener &

Gupta, 2005). In this way, many SWMM parameters are either empirical, conceptual, or

typically lumped in practice such that there is no singularly correct SWMM parametrization (or

structure) for a given LID system. As many of the parameter values in SWMM are often

conceptualized physical processes or system properties that cannot be directly measured, they

must be estimated via calibration. When it comes to calibration, SWMM LID modules are still

often treated and explored with simplistic methods (manual calibration and perhaps one

assessment of parameter sensitivity). In this study we perform a diagnostic assessment to test

three potential sources of sensitivity and uncertainty, with the goal of gaining insight to make

recommendations about frugal approaches to sensitivity and uncertainty analysis within SWMM.

Below, we outline different tests we have applied to assess sensitivity and uncertainty within

13

SWMM, with the goal of improving understanding of our system and making recommendations

to other SWMM users.

3.4.1 Initial Conditions

Within SWMM, initial soil moisture represents a potential source of uncertainty, especially given

that many green roofs may not be outfitted with sensors to observe this variable. As the soil

water content in the beginning of a precipitation event depends on the antecedent conditions and

varies from event to event, it represents a large source of uncertainty in the simulation of green

roof runoff (Burszta-Adamiak & Mrowiec, 2013; Palla & Gnecco, 2015). To demonstrate the

impact of this initial condition on simulations, we tested the impact of setting initial soil moisture

at zero and at an average observed soil moisture for the best fit parameter set for the October 28,

2017 and June 5, 2017 events. We report NSE for both events and scenarios.

3.4.2 Parameter Equifinality

Ideally, the best fit parameter set can be treated as a unique combination of parameter values that

best reproduce the observed data (Cobelli & DiStefano, 1980; Sorooshian & Gupta, 1983).

However, it is also possible that many parameter combinations will reproduce the observations

equally well with respect to a single error metric (Cobelli & Distefano, 1980; Wagener et al.,

2003). This condition is often referred to as equifinality (Beven, 1989; Beven & Binley, 1992;

Kelleher et al., 2013; Wagener et al., 2003). Parameter equifinality is especially problematic for

complex models with many parameters (Beven & Binley, 1992; Wagener et al., 2003), and is

generated by either insensitive parameters (e.g., parameters that can be varied but have little

effect on simulated runoff) or interactive parameters (e.g., parameters values that effect

simulated runoff based on dependence to other parameter values).

14

To test equifinality as it relates to SWMM, we explored the parameter space to identify

combinations of parameter values that generated hydrographs yielding similar error metric

values. This was, in part, an exploratory exercise, to assess whether equifinality is a concern

within SWMM. Though there are recommended ranges for SWMM LID parameters (Rossman,

2015), our assessment included values that exceeded these reasonable ranges. Analyzing how far

the model could be “pushed” with respect to parameter equifinality was critical to demonstrate

potential uncertainty in model calibration.

3.4.3 Parameter Sensitivity

Sensitivity and uncertainty analysis encourages greater understanding of system dynamics. It

may help to identify parameters that interact and therefore produce equifinal simulations, and

assists in obtaining a model that reproduces observed quantities with a minimum amount of

effort during the calibration process (Beck, 1987; Beven, 2008; Kelleher et al., 2013; Sorooshian

& Gupta, 1983). In SWMM, sensitivity analyses are often used to determine the parameters that

would most effectively minimize the difference between observed and simulated results, with the

goal of improving model fit (Krebs et al., 2016; Peng & Stovin, 2017). However, sensitivity

analyses can also provide information about the potential sources of equifinality in a model, as

equifinality can be the result of parameters that are insensitive (Johnston & Pilgrim, 1976) and/or

interactive (Ibbitt and O’Donnell 1971). Due to the current SWMM setup, most SWMM users

perform what is termed one-at-a-time (OAT) sensitivity analysis. This is a simplistic analysis in

which a single parameter value is increased or decreased by some equal amount holding all other

values constant; it may be performed from an initial parameter set, or from a calibrated

parameter set, though which sensitivity analysis is reported varies throughout the literature.

15

Assessing whether a parameter is insensitive, influences model output through

interactions, or has a direct impact on model output can be performed through a variance-based

sensitivity analysis (Kelleher et al., 2013; Pianosi et al., 2016; Song et al., 2015), though this

requires running a given model thousands of times. In the case of graphical user interface

(GUI)-based models like SWMM where there is no flexibility to directly implement variance-

based sensitivity analysis without additional programming, parameter influence on model output

can be assessed through two tests which can be seen in Figure 2. If a parameter is important by

itself or through interactions, with an equal increase or decrease of the parameter value by some

fraction of the parameter range, the error metric (i.e. NSE) between observed and simulated

runoff increases or decreases (Figure 2, Test 1). From this result, no distinction can be made

between individual or interactive importance. However, if the error metric does not increase or

decrease, then a second test needs to be performed (Figure 2, Test 2). If a parameter is sensitive

but interactive, the error metric should vary if this parameter value is set to zero. In contrast, if

the error metric improves, this suggests this parameter is sensitive because the conceptual

property or process it represents is not critical to reproducing the system behavior of interest

(e.g., green roof runoff). However, if the error metric still does not change when the parameter

value is set to zero then the parameter is insensitive.

The portion of the parameter space being sampled can also influence the interpretation of

identifiability and sensitivity (Saltelli, Ratto, Tarantola, & Campolongo, 2006). Therefore,

parameter sensitivity also may depend on “location” in the parameter space. To assess this, we

performed a sensitivity analysis with respect to initial parameter values (Table 2) as well as

calibrated values (October 28, 2017 event) and reported changes in NSE per parameter.

16

4 Results

4.1 Calibration and Assessment of Model Performance

The results of individual event simulations using initial parameter values (Table 2) showed

relatively poor agreement between measured and simulated runoff from the OnCenter green roof,

with NSE values below 0.5 for all three events (May 1, 2017; June 5, 2017; October 28, 2017).

All simulated runoff peak runoffs were underestimated. Simulated runoff also failed to capture

the steepness of the falling limb. All these phenomena indicate that the green roof detention

processes were not well represented within the initial parameter set SWMM model.

With manual calibration, model performance improved substantially. NSE increased to

0.9 for the calibration period (October 28, 2017 event; Figure 3A) and exceeded 0.9 for two

additional storms not included in calibration (May 1, 2017, Figure 3B; June 5, 2017; Figure 3C).

Table 2 lists parameter values corresponding to the manually obtained best fit parameter set. All

parameter values fell within literature ranges.

4.2 Diagnostic Analysis of SWMM

4.2.1 Equifinality

To explore the potential for equifinality in SWMM, we were capable of identifying several

parameter sets spanning both reasonable and unreasonable ranges that yielded nearly equivalent

values of NSE. These parameter combinations, visualized in Figure 4A, all resulted in NSE

values above 0.9 and nearly equivalent simulations of runoff that closely match observations

(Figure 4B). Table 3 further illustrates how parameter interactions between saturated hydraulic

conductivity (Ksat) and soil porosity (Φ2) can yield equifinality across multiple storm events.

17

Even unreasonable values of Ksat were capable of producing acceptable error metric values when

varied in tandem with Φ2.

4.2.2 Initial and Best Fit Parameter Set Sensitivity Analyses

Results of the initial parameter set OAT sensitivity analysis are presented in Figure 5A. The

OAT sensitivity analysis indicated that NSE was most sensitive to one (Φ2) of the thirteen

defined parameters (change in NSE>0.100), moderate for four (ΘFC, ΘWP, C, b) parameters

(0.099>NSE>0.001), and insensitive for eight (n, S1, S, Ψ, Hd, Ksat, D3, Φ3) parameters (change

in NSE = 0). In contrast, the best fit parameter set sensitivity analysis indicated that model

performance was most sensitive to four (Φ2, ΘFC, Hd, D3) of the thirteen defined parameters,

moderate for five (ΘWP, C, b, Ksat, S) parameters, and insensitive for four (n, S1, Ψ, Φ3)

parameters (Figure 5B).

As an additional test on parameter insensitivity, we determined NSE when all insensitive

parameters were set to zero. It was not possible to set Ksat, D3, and Φ3 to zero, so these

parameters were excluded from this analysis. For both the initial and best fit parameters, we

observed no difference in NSE when setting each of these values to zero, confirming these

parameters were insensitive local to initial and best fit parameter sets.

4.2.3 Initial Soil Moisture Analysis

Soil moisture can be critical to predicting runoff peaks for events with short antecedent dry

weather periods, such as successive storms, and storm events with high intensity precipitation

peaks in the beginning half of an event. To test the impact of having no information (set to a

value of zero) versus observed information (set to average of observed) to inform initial soil

moisture conditions on event simulations, we compared NSE for two events (June 5, 2017;

18

October 28, 2017) in Figure 6. Figure 6A shows the event beginning on October 28, 2017 which

had an antecedent dry weather period of 42.17 hours and reached both peak precipitation

intensity of 1.7 mm 5-min-1 and peak runoff at 30.25 hours into the storm event. Increasing the

initial soil moisture from zero to the observed initial measured soil moisture value of 0.11 only

slightly decreased the model error metric from an NSE of 0.939 to 0.973. Figure 6B shows the

event beginning on June 5, 2017 which had an antecedent dry weather period of 13.5 hours and

reached both peak runoff and peak precipitation intensity of 3.8 mm 5-min-1 at 10 hours into the

storm event. Increasing the soil moisture from zero to the initial measured soil moisture value of

0.35 improved NSE from 0.304 to 0.942.

4.2.4 Considering Multiple Error Metrics

In stormwater management, the most common output to calibrate to is runoff. Reduction in peak

runoff intensity and shift in peak are two of the foremost desired outcomes from low impact

development systems. However, other observations beyond runoff may provide unique

information to constrain parameter uncertainty. Figure 7 shows the range of soil moisture values

that were gathered from three soil moisture sensors on the OnCenter green roof compared to soil

moisture content simulated based on a best fit porosity and a measured porosity. These two

simulated time-series were quite different, and produced NSE values of 0.476 (best fit) and -

0.356 (measured) when compared to the observed soil moisture averaged across these three

sensors. This exercise illustrates that it can be difficult to use physically measured values as

inputs to, or constraints on, the SWMM model. The modeler exhausted all efforts, even letting

parameters vary widely, and could not achieve any strong agreement between modeled soil

moisture and an envelope of observed soil moisture, illustrating that while SWMM simulations

19

are capable of replicating the observed hydrograph, representative simulations of soil moisture

were more difficult to attain.

5 Discussion

5.1 Assessment of SWMM Bioretention Module

As shown in Figure 3, model simulations approximated observed OnCenter green roof runoff for

three events. Notably, our error metrics for all storms were comparable to or exceeded error

metrics documented in other literature (Krebs et al., 2016; Palla & Gnecco, 2015; Peng & Stovin,

2017). Like many SWMM studies, we used a ‘calibration-validation’ approach to demonstrate

that the parameter set yielding high NSE for several storms was capable of simulating observed

runoff for additional events. We used a total of three events (Table 1), with the goal of

incorporating multiple return periods, event sizes, and initial conditions (e.g., wet vs. dry soil

substrate). Still, we caution that parameter estimates (Table 2) were biased by the period and

conditions for which they were assessed (Peng & Stovin, 2017), meaning that extrapolation to

larger events or for periods beyond spring, summer, and fall may not be possible. We therefore

recommend that any modeling analysis of SWMM LID behavior incorporate events from all

seasons, multiple events exceeding a 1-year return period, and events that display varying sizes

and initial conditions.

Earlier studies of modelling green roof runoff have produced results with varying

success. Alfredo et al. (2010) achieved reasonable results using SWMM after significant

calibration efforts with both the ‘storage node’ and ‘curve number’ approaches, two alternative

methods to model green roofs in SWMM prior to the creation of SWMM LID modules. Burszta‐

Adamiak and Mrowiec (2013) applied the SWMM LID bioretention module to simulate runoff

20

monitored from different green roof test beds and concluded that the model had limited

capabilities in simulating green roof runoff. They reported mostly negative model efficiencies

resulting from excessive simulated runoff peaks, while the model achieved a good replication of

the monitored event runoff volume. More recently, Cipolla et al. (2016), Krebs et al. (2016), and

Peng and Stovin (2017) reported reasonable results using the SWMM LID green roof module to

model full-scale green roofs and green roof test beds but thorough comparison to their study is

difficult since their roof systems did not include drainage conduit systems (Figure 1), only

drainage mats. These studies underscore the importance that further testing with SWMM applied

to LID systems such as green roofs is needed.

5.2 How can and why should we aim to prove model realism within applications of

SWMM?

All models are assumed to be imperfect representations of the system they are designed to

simulate. In this sense, deviations between simulations and observations are the result of

uncertainty introduced throughout the modeling process. Within SWMM, model realism,

ensuring the model is a reasonably accurate representation of reality, is tied to two different

aspects of the model: initial conditions and model structural equations. The source of

uncertainty that SWMM modelers have the least control over is uncertainty in model structure,

introduced when translating real-world processes into SWMM structural equations. There are

several model assumptions incorporated into the SWMM LID bioretention module that may

differ from real-world green roof structure and function. In particular, substrate water detention

is modeled by Eq. (4) assuming the matric potential varies linearly with water content and

porosity, and the wetting front advances at the same rate with depth. However, as experimental

tests have shown, the soil moisture curve (i.e., water content vs. matric potential) of green roof

21

substrate may deviate from this model assumption (Berretta, Poë, & Stovin, 2014; Cipolla et al.,

2016). While the SWMM LID bioretention module assumes a drainage system within a gravel

drainage layer, many green roofs are not designed in this manner. The OnCenter green roof has a

waterproof membrane, thin drainage mat, and drain conduits in the soil substrate layer on top of

the drainage mat. To account for this, the gravel drainage layer was minimized, and the drain

offset height was set to allow the drain system to be within the soil layer, though we caution that

this configuration has the potential to introduce errors in the estimation of the overall

bioretention water balance. The best fit parameters for the underdrain should therefore not be

assumed representative of other green roofs, and should be treated as specific to the OnCenter

green roof used in this study. Values such as porosity and saturated hydraulic conductivity are

assumed to be constant over time. However, it is known that porosity changes during the aging

of media and saturated hydraulic conductivity varies based on depth and saturation of substrate.

Broadly, this demonstrates that flexible model structures within SWMM, a concept widely

adopted when it comes to rainfall-runoff modeling (Clark et al., 2008, 2015), may be needed so

that users can ensure that model structure closely approximates green roof design.

Opportunistically, SWMM modelers do have control over setting initial subsurface

saturation conditions. As we and others have shown (Burszta-Adamiak & Mrowiec, 2013; Palla

& Gnecco, 2015), initial conditions shape the agreement between simulations and observations

of runoff intensity and volume, depending on the time of peak and the intensity of the event

(Figure 6). Collecting data on antecedent substrate moisture can be used as a simple and strategic

method to reduce model uncertainty. This underscores the importance of instrumenting green

roofs to gain soil moisture observations, which can be used to inform this key model input.

22

5.3 What can we learn from and how can we improve SWMM sensitivity analyses?

Sensitivity analysis can assist a modeler in the process of identifying a parameter set that

reproduces an observed variable of interest. In this way, sensitive parameters can be treated as

those pieces of the model that contain the most “information” for ensuring that model

simulations match observations. In this work, we incorporated sensitivity analysis within our

calibration framework, with the goal to shorten what can be a lengthy calibration process.

Currently, OAT sensitivity analysis is the only possible approach for assessing parameter

sensitivity in SWMM. Moreover, most studies only report an initial parameter set sensitivity

analysis (Krebs et al., 2016; Peng & Stovin, 2017) to aid in determining which parameters to

estimate, those that will have the most influence on minimizing a chosen error metric. As we

show, OAT sensitivity results vary based on which baseline parameter set is used (Figure 5). In

our analysis, we assessed sensitivity with respect to an initial parameter set, informed by physical

measurements and expert information (e.g., Jacobs Engineering), and with respect to a manually

calibrated, best fit parameter set that achieved very high NSE across three storms. We found that

parameter sensitivity varied widely across these two analyses, with several parameters (S, Hd,

Ksat, D3) that were insensitive with respect to the initial parameter set (e.g., no change in NSE)

found to be sensitive for the ‘best fit’ parameter set. The differences shown between initial and

best fit parameter set sensitivity analyses occur due to non-linear relationships between

parameters and simulated runoff.

As we show, executing additional sensitivity analyses may identify new sensitive

parameters, as sensitivity varies with “location” in the parameter space. As manual calibration

requires awareness and information on the sensitivity of parameters, running initial and best fit

parameter set sensitivity analyses can reveal information that can improve the calibration

23

process. Since most studies report sensitivity analysis results to highlight the most important

“controlling” model parameters, it is vital to report the sensitivity analysis around the best

parameter set estimate. Sensitive parameters, as identified from the initial parameter set, are

often sensitive because their estimated value does not approximate observations (e.g., negative

NSE). In contrast, sensitive parameters identified from the ‘best fit’ parameter set can be treated

as containing more information about the system being simulated. In this sense, it is especially

important that SWMM modelers incorporate both of these analyses into their own diagnostic

analyses.

When equifinality occurs (Table 3; Figure 4), it is usually the result of parameters that are

insensitive (Johnston & Pilgrim, 1976) and/or interactive (Ibbitt and O’Donnell 1971). The

power in identifying insensitive parameters is that they have no effect on the model output.

These identified insensitive parameters tell what make the system distinct, and that their

processes or components are insignificant. This information can provide a “check” on model

realism, to ensure the model is providing a realistic representation of the system being simulated.

For instance, Φ3 was insensitive, signifying that this property of our storage layer did not impact

model fit; given our green roof system does not have a storage layer, this analysis suggests that

our simplification to set this value to zero (Table 2) was realistic, and would not impact our

simulations. At the same time, insensitivity may also be a reflection of certain limitations. For

instance, insensitivity for parameters S1 and n signified that ponding was not an important

control on runoff in our system. However, we simulated events where surface ponding is not

expected to occur; we would expect that these parameters would be more sensitive when

simulating larger magnitude or high intensity events.

24

Though local OAT sensitivity analyses have limitations, a local sensitivity analyses

should be the first step in evaluating if computationally demanding analysis is necessary (Beven

& Binley, 1992; Hill et al., 2016). Regardless, determining sensitive parameters, a key step to

improving calibration as well as our fundamental understanding of the SWMM model, requires

more computationally demanding methods (Kelleher et al., 2013; Pianosi et al., 2016; Song et

al., 2015). To date, few studies have applied global sensitivity analysis to SWMM, though the

use of these computationally demanding methods is increasing (Krebs et al., 2016). The use of

these more complex methods is necessary to improve the application of SWMM LID modules to

LID systems.

5.4 How can we and why do we need to improve SWMM calibration?

As shown by other researchers, parameters within SWMM may be conceptual and cannot always

be taken as physically-based, ultimately meaning that adjustment through calibration is needed.

In this vein, we unsurprisingly found differences between a best model estimate of porosity (Φ2 =

0.47) and a single field validation of porosity (0.57) (Figure 7). Importantly, increasing Φ2 to the

measured porosity while maintaining a high NSE required adjustment of field capacity (ΘFC) and

wilting point (ΘWP) beyond realistic values outside of literature ranges. This highlighted the

important role of Φ2, which is consistent with the results of the OAT sensitivity analyses. Given

that porosity is a heterogeneous and dynamic property, changing through time due to the aging of

media and shifts in ratio of macropores and micropores, modeling may help document changes

to this dynamic property that are difficult to obtain via field validation.

Many analyses of SWMM LID modules rely on manual calibration to search the model

parameter space for a combination of model parameters that achieves an optimal error metric

value (Carson et al., 2017; Cipolla et al., 2016; Peng & Stovin, 2017). However, as SWMM is

25

highly nonlinear, manual calibration is not an adequate solution for searching the parameter

space. Currently, to manually calibrate SWMM, modelers perform numerous model iterations to

explore the parameter space “by hand”. The time required to manually search the parameter

space therefore requires that model run times be short. This often limits SWMM calibration to a

shorter period that incorporates fewer events (Broekhuizen, Leonhardt, Marsalek, & Viklander,

2019), though this is balanced by demonstrating that the ‘best fit’ parameter set can reliably

simulate additional storms. Furthermore, SWMM simulations are prone to equifinality (Figure

4). The fact that there are multiple disparate model structures and parameterizations which may

be equally good at simulating a particular period of monitored record is the concept of

equifinality (Beven, 2006), which has shaped a considerable body of environmental modeling

research in recent decades. Current methods of manual calibration are time consuming,

monetarily expensive, and require in-depth knowledge of the model system.

The alternative to manual calibration is auto-calibration, where an algorithm is used to

search the parameter space for a parameter set that minimizes single or multiple error metrics.

The pros and cons of manual versus auto-calibration techniques have been discussed in the

hydrological literature for decades (e.g. Boyle, Gupta, & Sorooshian, 2000; Gupta et al., 2009).

Due to the complexity and skill required, there are few papers that analyze SWMM LID modules

through external software packages (Krebs et al., 2016). Auto-calibration certainly has many

advantages over manual techniques, but may also incorporate subjective elements, such as the

selection of parameter ranges and the choice of an auto-calibration algorithm (Jackson-Blake &

Starrfelt, 2015; Vrugt, Braak, Gupta, & Robinson, 2009). At the same time, manual calibration

can be as expensive and time consuming as some analysis methods, requiring 1000s of trial and

error model runs to provide a thorough explanation for calibration results. This can be

26

impractical, where forward SWMM model runs sometimes take hours or days to complete, even

based on computational costs alone (Hill et al., 2013, 2016; Razavi et al., 2010).

One path forward may be to combine auto-calibration and manual techniques. As shown

by Boyle et al. (2000), the time and effort required to estimate the parameter range representing

the trade-offs in the performance of the model can be dramatically reduced by use of a

computerized global search procedure algorithm. Then, the attention of the hydrologist can be

redirected from the tedious effort of manually searching for the “good” regions. While SWMM

LID literature already provides expected parameter ranges, automatic calibration can certainly

further aid the search process. The inclusion of an automatic calibration technique would

decrease calibration time and allow for a potential starting location for further manual

calibration. While we argue that the wide use of SWMM necessitates the incorporation of auto-

calibration strategies, this addition should ultimately consider the wide literature that exists on

this topic, to best balance flexibility and user consideration.

5.5 How do we best incorporate new sources of information into the model evaluation

process?

Typical evaluations of SWMM calibration select runoff for error metric evaluation and have

concluded that SWMM LID modules are able to simulate measured runoff (Carson et al., 2017;

Cipolla et al., 2016; Krebs et al., 2016; Peng & Stovin, 2017). While matching to runoff is key,

there are multiple signals in a green infrastructure system that can be considered in model

evaluation. Calibrating to more complex hydrological variables can elucidate potential error or

misrepresentation within the model. In our own diagnostic analysis, capturing the falling and

rising limb of the soil moisture graph was not achievable (Figure 7). This suggested that soil

moisture is potentially sensitive to different parameters than runoff. Ultimately, this highlights

27

that SWMM structural equations may not accurately represent hydrological processes as a green

roof wets up and dries down.

Evapotranspiration (ET) between precipitation events is ultimately going to determine

and be determined by available soil moisture. In SWMM LID modules, ET is represented by the

Hargreaves equation on a daily timestep, which is temperature based and disaggregated to the

model timestep. Though there are many who argue that ET is poorly represented in SWMM

(Carson et al., 2017; Cipolla et al., 2016; Peng & Stovin, 2017)., model calibration nearly always

only considers fitting to runoff. The key to understanding model representation of processes

such as changes in soil moisture could be the periods in between these events. To have the

processes within green roofs represented more accurately, it may be critical to consider dry

periods during parameter estimation and model evaluation. Our soil moisture simulations (Figure

7) highlight that there is room for improvement.

Overall, the types of observations included in both calibration/validation and diagnostic

analyses exert important influence over what can be learned from model simulations. While we

focused on fitting SWMM with respect to a single error metric, other researchers have shown

that multi-response error metrics offer the potential for improved simulation of model outputs

(e.g., runoff) as well as other state variables, such as ET and soil moisture (Kuczera &

Mroczkowski, 1998). Including observations of internal processes, such as concentrations and

fluxes, in diagnostic analysis can help to ensure that the right results are being obtained for the

right reasons. Incorporating such additional ‘soft’ data into auto-calibration is not common

practice at present, but it is feasible and indeed highly desirable (e.g. Rankinen, Karvonen, &

Butterfield, 2006; Seibert & McDonnell, 2002).

28

6 Conclusions

While there are existing studies demonstrating that SWMM is capable of simulating green roof

behavior, we are currently missing an understanding of the fundamental drivers of uncertainty

within SWMM and how these can best be addressed in the current generation of SWMM models.

In this study, we performed diagnostic analyses of the SWMM LID bioretention module through

a modeling study of the OnCenter green roof in Syracuse, New York. Utilizing frugal sensitivity

and analysis methods, we evaluated calibration processes to examine potential sources of

parameter and model structure uncertainty. As discussed here and by others, many of the

parameters are either empirical, conceptual, or typically lumped in practice such that there is no

singularly correct SWMM parametrization (or structure) for a given LID system. Thus, while

EPA SWMM is inherently a deterministic model predicting a single set of model outputs from a

single set of parameters and inputs, there are typically many different parametrizations and

structures which could be used to represent a system of interest, potentially yielding 10s of 100s

of different “good” simulations. In the case of SWMM LID modules, simply relying on

modeler’s best judgement point estimates for uncertain parameters where the feasible parameter

space can generate a large range of outputs (i.e., ignoring equifinality) can limit the use and

acceptability of SWMM LID module results for management purposes.

SWMM, even for what may be considered a simple case – simulating green roof runoff –

is prone to nonlinearity, uncertainty, and equifinality. Frugal diagnostic analyses elucidated

multiple feedbacks between the model and observations, including that runoff simulations for

SWMM were highly sensitive to initial soil moisture conditions. As shown with initial and best

fit parameter set sensitivity analyses, any OAT sensitivity analyses are only local estimates

within a neighborhood of the selected parameter set. SWMM, while successful at simulating

29

runoff, was not capable of simulating other hydrological processes (i.e. substrate moisture).

These results indicate that the prediction of SWMM LID runoff is likely due to complex

interactions between the numerous model parameters used to describe LID characteristics,

further highlighting the extreme nonlinearity of SWMM LID modules.

While we show there is still information that can be gleaned from these frugal analyses to

test and improve understanding of responses, they fundamentally oversimplify hydrological

behavior within green roofs and limit the research that can be done with this tool. Partially

influenced by structural analysis limitations, many SWMM applications focus on single events

and wet periods. However, our research highlights that this strategy may miss important

information and misrepresent other hydrological processes. Utilizing different sources of

information to “calibrate” SWMM (i.e. measurements of ET, soil moisture, etc.) can identify

these misrepresentations. We also found that calibration strategies are often not explicitly stated

or consistent, making comparisons between reporting for model uncertainty problematic. We

recommend that applications of SWMM be elevated to a level of consistency that moves beyond

frugal analysis methods if the research goal is to improve representation of processes beyond

runoff simulation. Model structure flexibility combined with more robust diagnostic analyses are

necessary to fully understand the fundamental drivers of uncertainty in SWMM, allowing for

better predictions.

At the same time, SWMM represents a real opportunity through its popularity amongst

researchers and frequent application by practitioners. Encouraging interactions between these

groups can create symbiotic conversations that will only improve SWMM LID modules. The

strategy builds off the co-production of knowledge theory, which aims to build dialog between

parties that possess mutually beneficial knowledge. In the case of SWMM, practitioners convey

30

the practical applications of the model, which provides information to researchers around how to

formalize diagnostic analyses to be most informative. As the end users in many cases,

practitioners likely have developed key insights with respect to challenges applying SWMM and

suggestions for improvement; researchers possess the opportunity to explore these suggestions

and produce helpful tools, ultimately refining the model. Utilizing this co-production of

knowledge can supply more necessary data for SWMM LID module flexibility and complex

diagnostic analyses improvements.

31

7 Tables

Table 1. Characteristics of rainfall events.

Event

Number

Start of

Event

Rainfall

Duration

[h]

Rainfall

Depth

[mm]

Peak

Intensity

[mm 5min-1]

Antecedent Dry

Weather Period

[h]

Return

Period

[yrs]

Initially

Saturated

[%]

15 May 1,

2017 17:50 7.1 20.6 4.9 31.2 <1 12.2

37 Jun 5,

2017 10:50 33.0 31.3 3.8 13.5 <1 34.9

105 Oct 28,

2017 19:25 38.0 90.0 1.7 42.2 >5 11.1

Table 2. SWMM Parameters: Initial and best fit values for simulations.

Description Initial

Value Units Symbol Data Source

Best Fit

Value

Subcatchment

Area 5600.00 m2 A Squier and Davidson (2016) 5600.00

Width 111.00 m W Squier and Davidson (2016) 111.00

Surface layer

Berm height 0.00 mm D1 Jacobs Estimate 0.00

Vegetation volume 0.00 fraction 1-Φ1 Jacobs Estimate 0.00

Surface roughness 0.40 manning’s n n Jacobs Estimate 0.00

Surface slope 1.00 % S1 Squier and Davidson (2016) 0.00

Soil (substrate)

Thickness 7.60 cm D2 Squier and Davidson (2016) 7.60

Porosity 0.57 volume fraction Φ2 Penn State AASL 0.47

Field capacity 0.20 volume fraction ΘFC Jacobs Estimate 0.30

Wilting point 0.10 volume fraction ΘWP Jacobs Estimate 0.18

Conductivity 32,400.00 mm/hr Ksat Yang and Davidson (2017) 2032.00

Conductivity slope 10.00 [unitless] S Jacobs Estimate 55.00

Suction head 41.70 mm Ψ Jacobs Estimate 0.00

Storage

Thickness 304.80 mm D3 Jacobs Estimate 0.25

Void ratio 0.02 voids/solids Φ3 Jacobs Estimate 0.01

Seepage rate 0.00 mm/hr f3 Jacobs Estimate 0.00

Clogging factor 0.00 [unitless] CF Jacobs Estimate 0.00

Underdrain

Drain coefficient 0.08 mm/hr C Jacobs Estimate 0.38

Drain exponent 0.50 [unitless] b Jacobs Estimate 0.50

Drain offset height 0.00 mm Hd Jacobs Estimate 0.25

32

Table 3. Parameter interactions between saturated hydraulic conductivity (Ksat) and porosity (Φ2) generate equifinal

results (e.g., similar but high objective function values) when varied in tandem.

Φ2 [volume fraction]

Ksat

[mm/hr]

NSE

1-May 5-Jun 28-Oct

0.38 2032 0.930 0.942 0.968

0.40 2540 0.938 0.903 0.954

0.45 50,800 0.938 0.935 0.962

0.50 1,778,000 0.956 0.939 0.959

0.60 25,400,000 0.345 0.936 0.920

33

8 Figures

Figure 1. Conceptual diagrams of (a) the OnCenter green roof layers, drainage system, and monitoring equipment

locations and (b) conceptualization of green roof processes in the SWMM LID bioretention module including model

parameters, storage zones, and fluxes.

Figure 2. Flowchart showing a frugal sensitivity analysis approach for assessing whether model parameters are

sensitive or insensitive with parameter value changes.

34

Figure 3. Timeseries of rainfall, observed runoff, and modeled runoff from the best fit parameter set for (a) October

28, 2017 (calibration) and (b) May 1, 2017 (validation) and (c) June 5, 2017 (validation).

35

Figure 4. To demonstrate that SWMM is prone to parameter interactions, we have documented seventeen parameter

sets, visualized as a parallel coordinate plot (top), and corresponding runoff simulations (bottom) all with similar, high

values of model fit (NSE > 0.94). Within the parallel coordinate plot, each parameter set corresponds to a line that

spans the suite of 14 model parameters. These results show that equifinal parameter values largely span each

parameter range, which suggests that SWMM parameters are highly interactive.

36

Figure 5. OAT sensitivity analysis results documenting change in NSE with respect to (a) the initial (pre-calibrated)

parameter set and (b) best fit (post-calibrated) parameter set for increasing and decreasing each parameter by 50%,

and for setting each parameter value to zero. (c) We qualitatively summarized these results with respect to the initial

(left) and best fit (right) parameter sets, with color indicating storage or flux each parameter is most closely associated

with and shading documenting most sensitive, highly sensitive, and insensitive parameters. Comparing results reveals

fundamentally different process dominance based on location within the parameter space, indicating model non-

linearity.

37

Figure 6. To assess how limited soil moisture information can impact runoff simulations, we compared simulated

runoff for two storms using uninformed (e.g., no observed soil moisture; initial soil moisture set to zero) versus

informed (e.g., observed soil moisture available; initial soil moisture set to observed value coincident with model start)

estimates of soil moisture. While informed versus uninformed estimates yielded similar model fits for (a) an event on

October 28, 2017, model fits were severely degraded for (b) June 5, 2017 in the absence of information on initial soil

moisture.

38

Figure 7. While simulations from a calibrated (orange) versus a laboratory-based (gold) estimate of porosity (Φ2)

were similar for runoff (top), these two porosity estimates produced very different simulations of soil moisture content

(bottom) that poorly approximated the range of observed soil moisture from the OnCenter green roof.

39

9 References

Alfredo, K., Montalto, F., & Goldstein, A. (2010). Observed and Modeled Performances of

Prototype Green Roof Test Plots Subjected to Simulated Low- and High-Intensity

Precipitations in a Laboratory Experiment. Journal of Hydrologic Engineering, 15(6), 444–

457. https://doi.org/10.1061/(ASCE)HE.1943-5584.0000135

Arnold, C. L., & Gibbons, & C. J. (1996). Impervious Surface Coverage: The Emergence of a Key

Environmental Indicator. Journal of the American Planning Association, 62(2), 243–258.

https://doi.org/10.1080/01944369608975688

Bannerman, R. T., Owens, D. W., Dodds, R. B., & Hornewer, N. J. (1993). Sources of Pollutants

in Wisconsin Stormwater. Water Science and Technology, 28(3–5), 241–259.

https://doi.org/10.2166/wst.1993.0426

Beck, M. B. (1987). Water quality modeling: A review of the analysis of uncertainty. Water

Resources Research, 23(8), 1393–1442. https://doi.org/10.1029/WR023i008p01393

Berretta, C., Poë, S., & Stovin, V. (2014). Reprint of Moisture content behaviour in extensive

green roofs during dry periods: The influence of vegetation and substrate characteristics’’ q.

Journal of Hydrology, 516, 37–49. https://doi.org/10.1016/j.jhydrol.2014.04.001

Beven, K. (1989). Changing ideas in hydrology — The case of physically-based models. Journal

of Hydrology, 105(1–2), 157–172. https://doi.org/10.1016/0022-1694(89)90101-7

Beven, K. (2006). A manifesto for the equifinality thesis. Journal of Hydrology, 320(1–2), 18–36.

https://doi.org/10.1016/J.JHYDROL.2005.07.007

Beven, K. (2008), Environmental Modelling: An Uncertain Future?, Routledge, London.

Beven, K., & Binley, A. (1992). The future of distributed models: Model calibration and

uncertainty prediction. Hydrological Processes, 6(3), 279–298.

https://doi.org/10.1002/hyp.3360060305

Bianchini, F., & Hewage, K. (2012). How “green” are the green roofs? Lifecycle analysis of green

roof materials. Building and Environment, 48, 57–65.

https://doi.org/10.1016/J.BUILDENV.2011.08.019

Blanusa, T., Vaz Monteiro, M. M., Fantozzi, F., Vysini, E., Li, Y., & Cameron, R. W. F. (2013).

Alternatives to Sedum on green roofs: Can broad leaf perennial plants offer better “cooling

service”? Building and Environment, 59, 99–106.

https://doi.org/10.1016/J.BUILDENV.2012.08.011

Booth, D. B., & Jackson, C. R. (1997). URBANIZATION OF AQUATIC SYSTEMS:

DEGRADATION THRESHOLDS, STORMWATER DETECTION, AND THE LIMITS OF

MITIGATION. Journal of the American Water Resources Association, 33(5), 1077–1090.

https://doi.org/10.1111/j.1752-1688.1997.tb04126.x

40

Boyle, D. P., Gupta, H. V., & Sorooshian, S. (2000). Toward improved calibration of hydrologic

models: Combining the strengths of manual and automatic methods. Water Resources

Research, 36(12), 3663–3674. https://doi.org/10.1029/2000WR900207

Broekhuizen, I., Leonhardt, G., Marsalek, J., & Viklander, M. (2019). Calibration event selection

for green urban drainage modelling. Hydrology and Earth System Sciences Discussions, 1–

29. https://doi.org/10.5194/hess-2019-67

Burszta-Adamiak, E., & Mrowiec, M. (2013). Modelling of green roofs’ hydrologic performance

using EPA’s SWMM. Water Science and Technology, 68(1), 36–42.

https://doi.org/10.2166/wst.2013.219

Carson, T., Keeley, M., Marasco, D. E., McGillis, W., & Culligan, P. (2017). Assessing methods

for predicting green roof rainfall capture: A comparison between full-scale observations and

four hydrologic models. Urban Water Journal, 14.

https://doi.org/10.1080/1573062X.2015.1056742

Carson, T., Marasco, D., Culligan, P., & Mcgillis, W. (2013). Hydrological performance of

extensive green roofs in New York City: observations and multi-year modeling of three full-

scale systems. Environ. Res. Lett, 8, 24036–24049. https://doi.org/10.1088/1748-

9326/8/2/024036

Cipolla, S. S., Maglionico, M., & Stojkov, I. (2016). A long-term hydrological modelling of an

extensive green roof by means of SWMM. Ecological Engineering, 95, 876–887.

https://doi.org/10.1016/j.ecoleng.2016.07.009

Clark, M. P., Nijssen, B., Lundquist, J. D., Kavetski, D., Rupp, D. E., Woods, R. A., … Rasmussen,

R. M. (2015). A unified approach for process-based hydrologic modeling: 1. Modeling

concept. Water Resources Research, 51(4), 2498–2514.

https://doi.org/10.1002/2015WR017198

Clark, M. P., Slater, A. G., Rupp, D. E., Woods, R. A., Vrugt, J. A., Gupta, H. V., … Hay, L. E.

(2008). Framework for Understanding Structural Errors (FUSE): A modular framework to

diagnose differences between hydrological models. Water Resources Research, 44(12).

https://doi.org/10.1029/2007WR006735

Cobelli, C., & DiStefano, J. (1980). Parameter and structural identifiability concepts and

ambiguities: Acritical review and analysis. American Journal Physiology Regular

Integrative Comparative Physiology, 239(1), R7–R24

Cook-Patton, S. C., & Bauerle, T. L. (2012). Potential benefits of plant diversity on vegetated

roofs: A literature review. Journal of Environmental Management, 106, 85–92.

https://doi.org/10.1016/j.jenvman.2012.04.003

Dunnett, N. & Kingsbury, N. (2008). Planting Green Roofs and Living Walls. Portland, OR:

Timber Press, Inc.

41

Fassman-Beck, E., Voyde, E., Simcock, R., & Hong, Y. S. (2013). 4 Living roofs in 3 locations:

Does configuration affect runoff mitigation? Journal of Hydrology, 490, 11–20.

https://doi.org/10.1016/J.JHYDROL.2013.03.004

Foglia, L., Hill, M. C., Mehl, S. W., & Burlando, P. (2009). Sensitivity analysis, calibration, and

testing of a distributed hydrological model using error-based weighting and one objective

function. Water Resources Research, 45(6). https://doi.org/10.1029/2008WR007255

Francis, R. A., & Lorimer, J. (2011). Urban reconciliation ecology: The potential of living roofs

and walls. Journal of Environmental Management, 92(6), 1429–1437.

https://doi.org/10.1016/J.JENVMAN.2011.01.012

Gooseff, M. N., Bencala, K. E., Scott, D. T., Runkel, R. L., & McKnight, D. M. (2005). Sensitivity

analysis of conservative and reactive stream transient storage models applied to field data

from multiple-reach experiments. Advances in Water Resources, 28(5), 479–492.

https://doi.org/10.1016/J.ADVWATRES.2004.11.012

Gupta, H. V., Sorooshian, S., & Yapo, P. O. (1998). Toward improved calibration of hydrologic

models: Multiple and noncommensurable measures of information. Water Resources

Research, 34(4), 751–763. https://doi.org/10.1029/97WR03495

Gupta, H.V., Beven, K.J., & Wagener, T. (2006). Model calibration and uncertainty estimation.

Encyclopedia of Hydrological Sciences. John Wiley & Sons Ltd.

Gupta, H. V., Wagener, T., & Liu, Y. (2008). Reconciling theory with observations: elements of a

diagnostic approach to model evaluation. Hydrological Processes, 22(18), 3802–3813.

https://doi.org/10.1002/hyp.6989

Gupta, H. V, Kling, H., Yilmaz, K. K., Martinez, G. F., & Kling, H. (2009). Decomposition of the

mean squared error and NSE performance criteria: Implications for improving hydrological

modelling. Journal of Hydrology, 377, 80–91. https://doi.org/10.1016/j.jhydrol.2009.08.003

Haase, D. (2009). Effects of urbanisation on the water balance – A long-term trajectory.

Environmental Impact Assessment Review, 29(4), 211–219.

https://doi.org/10.1016/j.eiar.2009.01.002

Herman, J. D., Reed, P. M., & Wagener, T. (2013). Time-varying sensitivity analysis clarifies the

effects of watershed model formulation on model behavior. Water Resources Research,

49(3), 1400–1414. https://doi.org/10.1002/wrcr.20124

Hill, M. C., Faunt, C. C., Belcher, W. R., Sweetkind, D. S., Tiedeman, C. R., & Kavetski, D.

(2013). Knowledge, transparency, and refutability in groundwater models, an example from

the Death Valley regional groundwater flow system.

https://doi.org/10.1016/j.pce.2013.03.006

Hill, M. C., Kavetski, D., Clark, M., Ye, M., Arabi, M., Lu, D., … Mehl, S. (2016). Practical Use

of Computationally Frugal Model Analysis Methods. Groundwater, 54(2), 159–170.

https://doi.org/10.1111/gwat.12330

42

Hilten, R. N., Lawrence, T. M., & Tollner, E. W. (2008). Modeling stormwater runoff from green

roofs with HYDRUS-1D. Journal of Hydrology, 358(3–4), 288–293.

https://doi.org/10.1016/J.JHYDROL.2008.06.010

Ibbitt, R., & O’Donnell, T. (1971). Designing conceptual catchment models for automatic fitting

methods, Int. Associated Hydrology Science Publication, 101(2), 461–475.

Jackson-Blake, L. A., & Starrfelt, J. (2015). Do higher data frequency and Bayesian auto-

calibration lead to better model calibration? Insights from an application of INCA-P, a

process-based river phosphorus model. JOURNAL OF HYDROLOGY, 527, 641–655.

https://doi.org/10.1016/j.jhydrol.2015.05.001

Jefferson, A. J., Bhaskar, A. S., Hopkins, K. G., Fanelli, R., Avellaneda, P. M., & McMillan, S. K.

(2017). Stormwater management network effectiveness and implications for urban watershed

function: A critical review. Hydrological Processes, 31(23), 4056–4080.

https://doi.org/10.1002/hyp.11347

Johnston, P. R., & Pilgrim, D. H. (1976). Parameter optimization for watershed models. Water

Resources Research, 12(3), 477–486. https://doi.org/10.1029/WR012i003p00477

Kelleher, C., Mcglynn, B., & Wagener, T. (2017). Characterizing and reducing equifinality by

constraining a distributed catchment model with regional signatures, local observations, and

process understanding. Hydrol. Earth Syst. Sci, 21, 3325–3352. https://doi.org/10.5194/hess-

21-3325-2017

Kelleher, C., Wagener, T., McGlynn, B., Ward, A. S., Gooseff, M. N., & Payn, R. A. (2013).

Identifiability of transient storage model parameters along a mountain stream. Water

Resources Research, 49(9), 5290–5306. https://doi.org/10.1002/wrcr.20413

Krebs, G., Kuoppamäki, K., Kokkonen, T., & Koivusalo, H. (2016). Simulation of green roof test

bed runoff. Hydrological Processes, 30(2), 250–262. https://doi.org/10.1002/hyp.10605

Kuczera, G., & Mroczkowski, M. (1998). Assessment of hydrologic parameter uncertainty and the

worth of multiresponse data. Water Resources Research, 34(6), 1481–1489.

https://doi.org/10.1029/98WR00496

Lee, J. G., & Heaney, J. P. (2003). Estimation of Urban Imperviousness and its Impacts on Storm

Water Systems. Journal of Water Resources Planning and Management, 129(5), 419–426.

https://doi.org/10.1061/(ASCE)0733-9496(2003)129:5(419)

Legates, D. R., & McCabe, G. J. (1999). Evaluating the use of “goodness-of-fit” Measures in

hydrologic and hydroclimatic model validation. Water Resources Research, 35(1), 233–241.

https://doi.org/10.1029/1998WR900018

Li, J., Lam, E., Zhan, J., Li, J., Wai, O., Ho, Y., & Li, Y. (2010). Effect of green roof on ambient

CO2 concentration. Building and Environment, 45(12), 2644–2651.

https://doi.org/10.1016/J.BUILDENV.2010.05.025

43

Li, Y., & Babcock, R. W. (2014). Green roof hydrologic performance and modeling: a review.

Water Science and Technology, 69(4), 727–738. https://doi.org/10.2166/wst.2013.770

Lim, T. C., & Welty, C. (2017). Effects of spatial configuration of imperviousness and green

infrastructure networks on hydrologic response in a residential sewershed. Water Resources

Research, 53(9), 8084–8104. https://doi.org/10.1002/2017WR020631

Mentens, J., Raes, D., & Hermy, M. (2006). Green roofs as a tool for solving the rainwater runoff

problem in the urbanized 21st century? Landscape and Urban Planning, 77(3), 217–226.

https://doi.org/10.1016/j.landurbplan.2005.02.010

Nash, J. E., & Sutcliffe, J. V. (1970). River flow forecasting through conceptual models part I —

A discussion of principles. Journal of Hydrology, 10(3), 282–290.

https://doi.org/10.1016/0022-1694(70)90255-6

Nawaz, R., McDonald, A., & Postoyko, S. (2015). Hydrological performance of a full-scale

extensive green roof located in a temperate climate. Ecological Engineering, 82, 66–80.

https://doi.org/10.1016/j.ecoleng.2014.11.061

Palla, A., & Gnecco, I. (2015). Hydrologic modeling of Low Impact Development systems at the

urban catchment scale. https://doi.org/10.1016/j.jhydrol.2015.06.050

Palla, A., Gnecco, I., & Lanza, L. G. (2012). Compared performance of a conceptual and a

mechanistic hydrologic models of a green roof. Hydrological Processes, 26(1), 73–84.

https://doi.org/10.1002/hyp.8112

Palla, A., Gnecco, I., Lanza, L., Palla, A., Gnecco, I., & Lanza, L. G. (2010). Hydrologic

Restoration in the Urban Environment Using Green Roofs. Water, 2(2), 140–154.

https://doi.org/10.3390/w2020140

Peng, Z., & Stovin, V. (2017). Independent Validation of the SWMM Green Roof Module. Journal

of Hydrologic Engineering, 22(9), 1–12. https://doi.org/10.1061/(ASCE)HE.1943-

5584.0001558.

Pianosi, F., Beven, K., Freer, J., Hall, J. W., Rougier, J., Stephenson, D. B., & Wagener, T. (2016).

Sensitivity analysis of environmental models: A systematic review with practical workflow.

Environmental Modelling & Software, 79, 214–232.

https://doi.org/10.1016/j.envsoft.2016.02.008

Rankinen, K., Karvonen, T., & Butterfield, D. (2006). An application of the GLUE methodology

for estimating the parameters of the INCA-N model.

https://doi.org/10.1016/j.scitotenv.2006.02.034

Razavi, S., Tolson, B. A., Matott, L. S., Thomson, N. R., MacLean, A., & Seglenieks, F. R. (2010).

Reducing the computational cost of automatic calibration through model preemption. Water

Resources Research, 46(11). https://doi.org/10.1029/2009WR008957

44

Rosa, D. J., Clausen, J. C., & Dietz, M. E. (2015). Calibration and Verification of SWMM for Low

Impact Development. JAWRA Journal of the American Water Resources Association, 51(3),

746–757. https://doi.org/10.1111/jawr.12272

Rossman LA. (2008). Storm water management model; version 5.0.022. US EPA National Risk

Management Research Laboratory.

Rossman LA. (2015). Storm water management model user’s manual version 5.1, EPA/600/R-

14/413b. US EPA National Risk Management Research Laboratory.

Rossman LA. (2017). Storm water management model; version 5.1.012. US EPA National Risk

Management Research Laboratory.

Ruth, O. (2003). The effects of de-icing in Helsinki urban streams, Southern Finland. Water

Science and Technology, 48(9), 33–43. https://doi.org/10.2166/wst.2003.0486

Saltelli, A., Ratto, M., Tarantola, S., & Campolongo, F. (2006). Sensitivity analysis practices:

Strategies for model-based inference. Reliability Engineering and System Safety, 91, 1109–

1125. https://doi.org/10.1016/j.ress.2005.11.014

Scott, D. T., Gooseff, M. N., Bencala, K. E., & Runkel, R. L. (2003). Automated calibration of a

stream solute transport model: implications for interpretation of biogeochemical parameters.

Journal of the North American Benthological Society, 22(4), 492–510.

https://doi.org/10.2307/1468348

Seibert, J., & McDonnell, J. J. (2002). On the dialog between experimentalist and modeler in

catchment hydrology: Use of soft data for multicriteria model calibration. Water Resources

Research, 38(11), 23-1-23–14. https://doi.org/10.1029/2001WR000978

She, N., & Pang, J. (2010). Physically Based Green Roof Model. Journal of Hydrologic

Engineering, 15(6), 458–464. https://doi.org/10.1061/(ASCE)HE.1943-5584.0000138

Song, X., Zhang, J., Zhan, C., Xuan, Y., Ye, M., & Xu, C. (2015). Global sensitivity analysis in

hydrological modeling: Review of concepts, methods, theoretical framework, and

applications. Journal of Hydrology, 523, 739–757.

https://doi.org/10.1016/j.jhydrol.2015.02.013

Sorooshian, S., & Gupta, V. K. (1983). Automatic calibration of conceptual rainfall-runoff models:

The question of parameter observability and uniqueness. Water Resources Research, 19(1),

260–268. https://doi.org/10.1029/WR019i001p00260

Spolek, G. (2008). Performance monitoring of three ecoroofs in Portland, Oregon. Urban

Ecosystems, 11(4), 349–359. https://doi.org/10.1007/s11252-008-0061-z

Squier, M., & Davidson, C. I. (2016). Heat flux and seasonal thermal performance of an extensive

green roof. Building and Environment, 107, 235–244.

https://doi.org/10.1016/j.buildenv.2016.07.025

45

Stovin, V., Poë, S., De-Ville, S., & Berretta, C. (2015). The influence of substrate and vegetation

configuration on green roof hydrological performance. Ecological Engineering, 85, 159–172.

https://doi.org/10.1016/j.ecoleng.2015.09.076

Stovin, V., Vesuviano, G., & Kasmin, H. (2012). The hydrological performance of a green roof

test bed under UK climatic conditions. Journal of Hydrology, 414–415, 148–161.

https://doi.org/10.1016/J.JHYDROL.2011.10.022

Susca, T., Gaffin, S. R., & Dell’Osso, G. R. (2011). Positive effects of vegetation: Urban heat

island and green roofs. Environmental Pollution, 159(8–9), 2119–2126.

https://doi.org/10.1016/j.envpol.2011.03.007

Tang, Y., Reed, P., Wagener, T., & Van Werkhoven, K. (2007). Comparing sensitivity analysis

methods to advance lumped watershed model identification and evaluation. Hydrol. Earth

Syst. Sci (Vol. 11). Retrieved from www.hydrol-earth-syst-sci.net/11/793/2007/

US EPA. (2000). Low Impact Development (LID) – a literature review, EPA-841-B-00-005. US

EPA Office of Water (4203).

Voter, C. B., & Loheide, S. P. (2018). Urban Residential Surface and Subsurface Hydrology:

Synergistic Effects of Low‐Impact Features at the Parcel Scale. Water Resources Research,

54(10), 8216–8233. https://doi.org/10.1029/2018WR022534

Vrugt, J. A., ter Braak, C. J. F., Gupta, H. V., & Robinson, B. A. (2009). Equifinality of formal

(DREAM) and informal (GLUE) Bayesian approaches in hydrologic modeling? Stochastic

Environmental Research and Risk Assessment, 23(7), 1011–1026.

https://doi.org/10.1007/s00477-008-0274-y

Wagener, T., & Gupta, H. V. (2005). Model identification for hydrological forecasting under

uncertainty. Stochastic Environmental Research and Risk Assessment, 19(6), 378–387.

https://doi.org/10.1007/s00477-005-0006-5

Wagener, T., McIntyre, N., Lees, M. J., Wheater, H. S., & Gupta, H. V. (2003). Towards reduced

uncertainty in conceptual rainfall-runoff modelling: dynamic identifiability analysis.

Hydrological Processes, 17(2), 455–476. https://doi.org/10.1002/hyp.1135

Yang, Y. & Davidson, C.I. (2017). Green roof performance influenced by growth medium

characteristics. Poster Presentation, 15th Annual New York State Green Building Conference,

Syracuse, NY, March 30-31, 2017.

46

10 CV

Lucie Worthen, ENV SP 136 Roosevelt Ave Apt 2, Syracuse, NY 13210 | Lworthen626@gmail.com | (603) 530-1053

EDUCATION

Jun 2017-Present Syracuse University | College of Engineering & Computer Science | Syracuse, NY

M.S. Environmental Engineering | Expected May 2019

Advisors: Cliff I. Davidson, Ph.D. & Christa Kelleher, Ph.D.

GPA: 4.0

Aug 2009-May 2013 University of New Hampshire| College of Engineering & Physical Science | Durham, NH

B.S. Environmental Engineering, Summa Cum Laude

Study Abroad: Budapest University of Technology and Economics, Hungary

Senior Capstone: The Dominican Republic Clean Water Project in Cumayasa, DR

GPA: 3.79

RESEARCH EXPERIENCE

May 2017-Present Syracuse University | Department of Civil & Environmental Engineering | Syracuse, NY

Graduate Research Fellow

Advisor: Professor Cliff I. Davidson, Ph.D.

Project | Doing a Lot with a Little: A Diagnostic Analysis of SWMM to Simulate Hydrologic

Behavior within LID Systems

Jun 2012-May 2013 University of New Hampshire | Department of Natural Resources & The Environment |

Durham, NH

Undergraduate Research Assistant

Advisor: Professor Rich Smith, Ph.D.

Project 1 | Cultivar diversity as a means of ecologically intensifying dry matter production in

a perennial forage stand

Project 2 | Increased productivity of a cover crop mixture is not associated with enhanced

agroecosystem services

Project 3 | Cover crop species as distinct biotic filters in weed community assembly

Project 4 | Effects of living mulch and fertilizer on the performance of broccoli in

plasticulture

Jun 2011-Aug 2011 University of New Hampshire | Department of Environmental Engineering | Durham, NH

REU Grant Researcher

Advisor: Professor Nancy Kinner, Ph.D.

Project | Evaluate microbe reactions under varying concentrations of chlorinated solvents

during groundwater remediation practices

PUBLICATIONS & PRESENTATIONS

Conference Papers

Sep 2018 Worthen L., Davidson C.I., “Sensitivity Analysis Using the SWMM LID Control for an

Extensive Green Roof in Syracuse, NY”. Presented at the International Building Physics

Conference. Syracuse, NY. September 2018.

47

Conference Posters

Sep 2018 Worthen L., Davidson C.I., “Validation of the PCSWMM LID Module for an Extensive Green

Roof in Syracuse, NY”. Updated version presented at the International Building Physics

Conference. Syracuse, NY.

Sep 2018 Worthen L., Davidson C.I., “Validation of the PCSWMM LID Module for an Extensive Green

Roof in Syracuse, NY”. Updated version presented at the SyracuseCoE Graduate Research

Poster Competition. Syracuse, NY.

Aug 2018 Worthen L., Davidson C.I., “Validation of the PCSWMM LID Module for an Extensive Green

Roof in Syracuse, NY”. Presented at the ASCE LID Conference. Nashville, TN.

May 2013 Im J., Worthen L., Anderson M., Miller B., “Dominican Republic Clean Water Project”.

Presented at the University of New Hampshire Senior Capstone Presentation. Durham, NH.

HONORS & AWARDS

Syracuse University

Sep 2018 First Place – SyracuseCoE Graduate Student Poster Competition | Syracuse, NY

May 2018 Graduate Student Organization (GSO) Travel Grant

Apr 2018-Apr 2019 CHI University Grant

Oct 2017-May 2019 Women in Science & Engineering Future Professional Program (WiSE-FPP) Associate

Aug 2017-Aug 2019 National Science Foundation (NSF) NRT EMPOWER Fellowship

University of New Hampshire

Aug 2011-May 2013 Marie L. Langelier Scholarship

Aug 2010-May 2011 Kenneth J Higson Scholarship

Aug 2009-May 2013 University of New Hampshire Presidential Scholarship

Aug 2009-May 2013 Dean’s List

Aug 2009 Bristol United Church of Christ University Scholarship

TEACHING EXPERIENCE

Aug 2017-May 2018 Syracuse University, College of Engineering & Computer Science | Syracuse, NY

Graduate Teaching Assistant

Treatment Processes in Environmental Engineering | Fall 2017

Sustainability in Civil & Environmental Systems | Spring 2018

May 2015-Apr 2016 Denver Permaculture Design Course | Denver, CO

Coordinator

Taught sustainable urban design utilizing systems within nature as a design framework

Sep 2013-May 2014 HETIC College | Paris, France

Professor of English

Curated curriculum and taught lecture to graduate students

48

WORK EXPERIENCE

May 2018-Present Jacobs | Syracuse, NY

Water Engineer Intern

Provided program management support, design support, and construction inspection

support for the Onondaga County green infrastructure program “Save the Rain” through

project cost analyses, AutoCAD and GIS construction design and standard drawings,

PCSWMM GI analysis, attendance at client and stakeholder meetings, and technical

memorandum, grant application, and report writing

Feb 2015-Apr 2016 CBDRx | Longmont, CO

Indoor Research & Development Farm Manager

Managed breeding research for optimum hemp genetics cultivars, provided data driven

analysis of cultivar breeding program through the implementation of standard protocols and

documentation, curated and maintained critical growing environment based on sustainable

practices and USDA organic certification

PROFESSIONAL AFFILIATIONS & HONORARY SOCIETIES

Sep 2018-Present International Association of Building Physics

Oct 2017-Present Women in Science & Engineering Program | Syracuse University

Aug 2017-Present NSF NRT EMPOWER Program | Syracuse University

Sep 2011- Present American Society of Civil Engineers (ASCE)

May 2011-Present Tau Beta Pi Engineering Honors Society

May 2011-Present Golden Key International Honour Society

Sep 2011-May 2013 Engineers Without Borders | University of New Hampshire

Sep 2009-May 2013 Society of Women Engineers | University of New Hampshire

VOLUNTEER & COMMUNITY SERVICE

May 2015-Oct 2015 UrbiCulture Community Garden | Denver, CO

Weekend farmers market coordinator, harvester, and food bank delivery driver

SOFTWARE SKILLS

Sep 2018 HydroCAD

Jan 2018 ArcGIS

Sep 2017 PCSWMM

Sep 2017 R Studio

Sep 2017 Hydrus 1-D

Jan 2013 HEC RAS

Jan 2012 AutoCAD

Jan 2012 USGS PHABSIM

Jan 2012 EPA NET