Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
The Role of Risk Management Prioritization
Decision Support Tools
An Overview of Barriers, Facilitators, and Recommendations
October 2020
Prepared for
USDA Forest Service
Southwestern Region, Region 3
Prepared by
Melanie Colavito, [email protected]
Ecological Restoration Institute
Northern Arizona University
PO Box 15017
Flagstaff, AZ 86001
NAU is an equal opportunity provider.
This report was funded by a grant from the USDA Forest Service.
Table of Contents
Executive Summary ..........................................................................................................1
Project Purpose and Need ................................................................................................3
Methods ...............................................................................................................................5
Findings ...............................................................................................................................7
Decision Support Tools Referenced in this Report ....................................................7
Decision Support Tool Definitions .............................................................................9
Decision Support Tool Development and Transfer .................................................10
Decision Support Tool Use ......................................................................................11
Decision Support Tool Barriers ...............................................................................14
Decision Support Tool Benefits and Facilitators .....................................................24
Recommendations ...........................................................................................................30
Conclusion ........................................................................................................................42
References .........................................................................................................................43
Appendix. Open-Ended Interview Questions ................................................................45
Acknowledgements
The Ecological Restoration Institute is grateful for the contributions and time of the interview
participants whose knowledge and perspectives informed this report.
Executive Summary
The purpose of this project was to analyze the use and adoption of wildfire risk assessment and
fuels treatment planning and prioritization analysis methods and products—broadly referred to
here as decision support tools (DSTs)—by federal land managers, especially in the US
Department of Agriculture (USDA) Forest Service. This project examines the efficacy of policy
direction articulated in frameworks and documents such as the National Cohesive Wildland Fire
Management Strategy, Forest Service Manual Chapter 5140, and Forest Service Shared
Stewardship Initiative that call for collaborative assessment of wildfire risk to inform fuels
treatment prioritization using the best available science. There is a need to demystify the topic of
spatial fire planning specifically with respect to assessing wildfire risk and determining areas for
fuels treatment prioritization to facilitate effective development and use of DSTs for pre-fire
planning. We acknowledge that the term DST may not adequately capture the complexity of the
different frameworks, methods, and tools referred to in this report, but we selected this term
intentionally while recognizing that new terminology may be warranted to describe the full realm
of approaches for wildfire risk assessment and treatment prioritization.
This report addresses the following key questions and the results are organized according to key
themes that emerged within each of these questions:
1. What decision support tools (DST) are commonly in use for assessing wildfire risk and
prioritizing areas for fuels treatment in the Forest Service?
2. How are DSTs defined by their developers and users?
3. How are these DSTs developed and transferred?
4. How are these DSTs used?
5. What are barriers to the use of these DSTs?
6. What are benefits and facilitators to the use of these DSTs?
7. How can the development and application of these DSTs be improved?
We used semi-structured interviews with key informants to identify common DSTs for assessing
wildfire risk and treatment planning and prioritization, approaches for the development and
transfer of DSTs, examples of DST uses, common barriers and facilitators in the development
and use of DSTs, and recommendations for facilitating the development and use of DSTs.
The audience for this report includes DST scientists and developers who develop analysis
methods and products, the end users of DSTs, and policy and decision makers who develop
guidance for using DSTs. There are recommendations for key audiences at the end of the report.
This report does not provide a comprehensive or technical summary of wildfire decision support
tools. Rather it is an effort to explore the social dimensions of developing and using DSTs for
pre-fire planning and fire response and to develop recommendations to enhance the use of
scientific and technical tools in pre-fire planning and fire response. This report can be used to
better understand the role of DSTs and support their effective use in wildfire risk assessment and
treatment prioritization planning.
1
Although there were many barriers identified to the effective development, integration, and use
of DSTs in pre-fire planning, interview respondents had numerous constructive
recommendations for improving this process, which we categorized into the following themes:
capacity, communication, implementation, question identification, testing, education and
training, and policy, guidance, and authorities. We hope these recommendations can help shape
the perspectives of science, management, and decision-making audiences for how to improve the
use of DSTs for wildfire risk assessment and treatment prioritization in order to effectively meet
the goals of national policies and frameworks.
2
Project Purpose and Need
The purpose of this project was to analyze the use and adoption of wildfire risk assessment and
fuels treatment planning and prioritization analysis methods and products—broadly referred to
here as decision support tools (DSTs)—by federal land managers, especially in the US
Department of Agriculture (USDA) Forest Service. This project examines the efficacy of policy
direction articulated in frameworks and documents such as the National Cohesive Wildland Fire
Management Strategy, Forest Service Manual Chapter 5140, and Forest Service Shared
Stewardship Initiative that call for collaborative assessment of wildfire risk to inform fuels
treatment prioritization using the best available science (Greiner et al. 2020, Stratton 2020).
There is a need to demystify the topic of spatial fire planning specifically with respect to
assessing wildfire risk and determining areas for fuels treatment prioritization to facilitate
effective development and use of DSTs for pre-fire planning. The desired outcomes of this
project are: 1) a characterization of common spatial fire risk assessment and treatment
prioritization planning analysis methods and products, and 2) actionable recommendations and
strategies developed with input from DST developers and users to facilitate the effective use of
DST outputs in decision making. We acknowledge that the term DST may not adequately
capture the complexity of the different frameworks, methods, and tools referred to in this report,
but we selected this term intentionally while recognizing that new terminology may be warranted
to describe the full realm of approaches for wildfire risk assessment and treatment prioritization.
Although there is policy direction to use DSTs for risk-informed, landscape-scale hazardous
fuels reduction treatments and fire response, the specifics for doing so are unclear. For example,
Forest Service Manual Chapter 5140 – Hazardous Fuels Management and Prescribed Fire –
includes two objectives: 1) “understand the role of fire on the landscape …”, and 2) “…
strategically plan and implement on a landscape scale, risk-informed, and cost-effective
hazardous fuel modification and vegetation management treatments …” (USDA Forest Service
2020). As Stratton (2020) notes, this guidance is consistent with broader frameworks such as the
National Cohesive Strategy, which articulates a cross-boundary, collaborative strategy to reduce
wildfire risk, as well as Shared Stewardship, which also promotes a collaborative, cross-
boundary approach to prioritizing hazardous fuels reduction treatments at landscape-scales.
However, none of these guidance or strategy documents include explicit requirements or
recommendations for what risk assessment and prioritization DSTs to use.
There are numerous analysis methods, products, and tools, and geographic and temporal scales
for modeling fire behavior, assessing wildfire risk, and prioritizing areas for treatment. This can
lead to confusion, among other barriers, about what kinds of information different frameworks,
models, and tools can provide for use in decision making (Rapp et al. 2020). A lack of effective
landscape-scale monitoring of completed restoration or fuels reduction treatments and limited or
incomplete data sets can also limit the development of effective risk assessment and treatment
prioritization DSTs (Esch and Waltz 2019). Nonetheless, DSTs can facilitate a risk-informed and
strategic approach to decision making in pre-fire planning, as well as in fire response (Calkin et
al. 2010, Thompson et al. 2019). And because land managers who are working to reduce the risk
3
of catastrophic wildfire and restore the ecological role of fire must balance many social and
ecological considerations, DSTs can provide an added level of support for complex decision-
making processes at different temporal and spatial scales that can allow for effective treatments
and the safe and effective use of fire (Caggiano et al. 2020, Thompson et al. 2019).
It is important to define key terms used in this report, given the need for a common language to
talk about risk in wildland fire management (Thompson et al. 2016) and confusion that exists
around the different approaches to characterizing and analyzing wildfire risk. The term used
most prominently in this report is “decision support tool” (DST). One of the first questions we
addressed with project participants was how to define a DST. Not surprisingly, responses varied
widely, and a summary of those responses comes later in this report. For the purposes of this
report, we use the term DST broadly to refer to the different frameworks, models, analysis
methods, and products and tools (i.e., software programs) that provide information about wildfire
risk assessment and treatment prioritization to support strategic, risk-informed pre-fire planning.
The focus in this report is on DSTs that support pre-fire planning, but certain DSTs referenced
by interviewees, such as the Wildland Fire Decision Support System (WFDSS), are largely
intended for fire response but may accept inputs and data layers developed from pre-fire
planning processes. In this report, the term DST refers to both specific tools, such as WFDSS
and the Interagency Fuel Treatment Decision Support System (IFTDSS), as well as analysis
methods and frameworks, such as Quantitative Wildfire Risk Assessments (QWRA) or Potential
Operational Delineations (PODs).
Other key terms used in the report include “wildfire risk assessment” and “treatment
prioritization.” While we did not ask participants to define these terms, it is important to provide
a frame of reference for how they are used in this report. Wildfire risk assessment refers to the
range of approaches for assigning values to risks and informing decision making (see Scott et al.
2013 and Thompson et al. 2016). The Thompson et al. (2016) definition, more specifically,
states, “A product or process that collects information and assigns values (relative, qualitative,
quantitative) to risks for the purpose of informing priorities, developing or comparing courses of
action, and informing decision making,” and is adopted here. Treatment prioritization refers to
approaches for determining where to treat hazardous fuels to reduce wildfire risk as determined
by a wildfire risk assessment or other prioritization process. It is explicitly referenced in this
report because we also examined DSTs that provide outputs specific to treatment prioritization.
This report addresses the following key questions:
1. What decision support tools (DST) are commonly in use for assessing wildfire risk and
prioritizing areas for fuels treatment in the Forest Service?
2. How are DSTs defined by their developers and users?
3. How are these DSTs developed and transferred?
4. How are these DSTs used?
5. What are barriers to the use of these DSTs?
6. What are benefits and facilitators to the use of these DSTs?
7. How can the development and application of these DSTs be improved?
4
The rationale for this project was informed by the “All-Lands Planning, Coordination, and
Wildfire Risk Reduction Project” conducted by the Ecological Restoration Institute (ERI)
(Colavito et al. 2018). One of the lessons learned from that project was that although DSTs can
provide robust, scientific information to inform land management planning, there are many
social barriers to realizing the effective use of DST outputs in decision making. This project
sought to identify the specific barriers and facilitators to the effective development and use of
DSTs for wildfire risk assessments and treatment planning and prioritization and to generate
actionable recommendations for DST developers, users, and policymakers to improve the use of
DSTs in land management.
The audience for this report includes DST scientists and developers who develop analysis
methods and products, the end users of DSTs, and policy and decision makers who develop
guidance for using DSTs. There are recommendations for key audiences at the end of the report.
This report does not provide a comprehensive or technical summary of wildfire decision support
tools. Rather it is an effort to explore the social dimensions of developing and using DSTs for
pre-fire planning and fire response and to develop recommendations to enhance the use of
scientific and technical tools in pre-fire planning and fire response. This report can be used to
better understand the role of DSTs and support their effective use in wildfire risk assessment and
treatment prioritization planning.
Methods
We used semi-structured interviews with key informants to identify common DSTs for assessing
wildfire risk and treatment planning and prioritization, approaches for the development and
transfer of DSTs, examples of DST uses, common barriers and facilitators in the development
and use of DSTs, and recommendations for facilitating the development and use of DSTs. Prior
to conducting the semi-structured interviews, we reviewed peer-reviewed literature, gray
literature, and other relevant sources to inform our research questions and interview protocol. We
also conducted preliminary, informal scoping interviews and conversations with key contacts
who did not participate in formal interviews to inform our study design.
In order to identify interview participants, we used a purposive sampling approach. We started
by asking contacts from the scoping conversations who they recommended we interview. The
requirement for our purposive sample was that respondents be familiar with the development or
use of the DSTs relevant to pre-fire planning. We asked interviewees to provide suggestions for
additional interviewees, as well as identified interviewees through our networks. We contacted
interviewees in multiple places and roles to ensure our data represented a range of perspectives
(Tables 1, 2, and 3). There were 27 interview participants (n=27). We did not geographically
constrain interviews and had respondents from Regions 1, 2, 3, 5, 6, and 9 of the USDA Forest
Service. Respondent locations included New Mexico (n=6), Colorado (n=5), Montana (n=5),
Oregon (n=4), Arizona (n=4), Idaho (n=1), California (n=1), Minnesota (n=1). We conducted
interviews until we reached information saturation.
5
The interviews averaged one hour in length, were all conducted over the phone, and were
recorded and transcribed for analysis. We used the qualitative data analysis software Nvivo 11
for Windows to organize and code interview transcripts for key themes. The analytical process
we used was a systematic approach called grounded theory, which allowed us to iteratively
generate themes both from prior knowledge and literature, as well as from the data itself. The
findings presented here represent a summary of themes across interviews. Interviewees are not
identified with quotes to ensure anonymity according to the requirements of the Northern
Arizona University Institutional Review Board that oversees this human subjects project.
Table 1. Respondent Demographics by Organizational Affiliation
Organizational Affiliation Number of Respondents Percent
USDA Forest Service 22 81%
External Stakeholder 5 19%
Total 27 100%
Table 2. Respondent Demographics by Title Classification
Title Classification Number of Respondents Percent
Agency Analysis/ Tech
Transfer 7 26%
Agency Researcher 2 7.33%
Fire Management 6 22%
Line Officer 2 7.33%
External Researcher 3 11%
Fuels Management 5 19%
Collaborative Partner 2 7.33%
Total 27 100%
Table 3. Respondent Demographics by General Classification (as assigned or self-identified during
interview)
General Classification Number of Respondents Percent
Boundary Spanner 8 30%
Scientist/ Developer 6 22%
Manager 13 48%
Total 27 100%
6
Findings
Decision Support Tools Referenced in this Report
Currently, there is no comprehensive list of DSTs used for wildfire risk assessment or treatment
prioritization, though USDA Forest Service Research and Development recently conducted an
inventory of common tools. Interviewees referenced numerous DSTs relevant to wildfire risk
assessment and treatment prioritization. The DSTs referenced by interviewees are organized
below according to the kinds of information they provide to help elucidate the different roles that
DSTs play in the realm of pre-fire planning. This is not an exhaustive list of wildfire risk
assessment and treatment prioritization DSTs, rather it is a list of the major tools that were
discussed by interviewees. Some interviewees also referred to DSTs developed for specific
projects or collaborative groups, which are not included in this list.
• Data sets: developing DSTs require data sets that provide the underlying data that are
analyzed as an input for decision making. Any modeling product or DST is only as good as
the data on which it relies. Interviewees referenced three data sets used in wildfire risk
assessment and treatment prioritization DSTs.
o LANDFIRE: a database of more than 20 geospatial layers such as vegetation, fire
regimes, and fuels for the United States.
o FSVeg, or Field Sampled Vegetation: stores data about trees, fuels, down woody
material, surface cover, and understory vegetation.
o FIA, or Forest Inventory and Analysis Program: a national census of forest data in the
United States.
• Wildfire Risk Assessment: many interviewees referenced fire simulation models, each of
which have different parameters and use different fire behavior models to compute
predictions. However, all the models below use the same surface fire spread equation
developed by Rothermel (1972). There are many challenges in modeling wildfire, depending
on weather conditions, patterns of fire ignition across landscapes, and historical distributions
of fires (Ager et al. 2017). All of the DSTs below provide some assessment of wildfire risk,
which can be defined as the likelihood of a fire occurring, the intensity of behavior when a
fire does occur, and effects of fire on highly valued resources and assets (HVRA) (Calkin et
al. 2010, Scott et al. 2013). The DSTs referenced by interviewees are organized below
according to whether they generate fire simulations (i.e., first order products) or use fire
simulations to create products to inform decision making (i.e., second order products).
o Fire probability models and simulators (i.e., first order products)
▪ FlamMap: a fire analysis system that calculates pixel-based measures of fire
behavior; smaller scale but similar inputs to LFSim.
▪ FARSITE: a computational approach to estimating wildfire growth and
behavior, which has now been integrated into FlamMap; tends to overestimate
extent of fire compared with on-the-ground observations.
7
▪ FSim or LFSim: also known as the Large Fire Modeling Simulation System,
is a method for simulating fire modeling across larger scales and across
seasons than FlamMap or FARSITE but using similar parameters.
▪ Fire Spread Probability (FSPro): a probabilistic model that estimates where
fire may go; tends to overpredict final wildfire footprint but underestimate
farthest extent burned.
▪ BEHAVE: a fire modeling system that simulates surface and crown fire rate
of fire spread and intensity, probability of ignition, fire size, spotting distance,
and tree mortality.
o Frameworks or tools that use outputs from fire simulation modeling (i.e., second
order products)
▪ Wildfire Hazard Potential: a national scale map that combines flame-length
probability and burn probability to inform wildfire risk assessment and fuels
management prioritization across large landscapes.
▪ Wildfire Risk to Communities: an interactive website that provides a national
scale assessment of wildfire risk to communities; best used to compare risk
among communities not within communities. This was developed in response
to the 2018 Omnibus Bill calling for a national map of wildfire hazard.
▪ Wildland Fire Decision Support System (WFDSS): an online system that
supports fire managers, agency administrators, and fire analysts in making
decisions on fire incidents; includes fire modeling capability, as well as
decision documentation processes.
▪ Quantitative Wildfire Risk Assessment (QWRA): a fire modeling process that
assesses how HVRAs respond to modeled fire behavior and determines both
benefits and threats to selected HVRAs. The background for this concept is
described in General Technical Report (GTR) 235 (Calkin et al. 2010) and the
process for developing QWRAs is detailed in GTR 315 (Scott et al. 2013).
▪ Potential Operational Delineation (POD): PODs refer to boundaries that
provide different fire response options across a landscape. POD polygons
provide an overview of fire risk according to potential control features and
values at risk (Caggiano 2019, Greiner et al. 2020). They are an approach for
making QWRA relevant in an operational context but can also be developed
without a QWRA (Caggiano et al. 2020). There are also efforts to explore
how to use PODs to guide treatment prioritization.
• Treatment Planning and Prioritization: interviewees referred to models that provide
simulations of forest growth and landscape processes, especially with respect to how
management changes can impact fire behavior. These models can be used to inform
treatment planning and prioritization decisions, though as with fire behavior models, there
are many challenges to simulating forest changes (Ager et al. 2017).
8
o Forest Vegetation Simulator (FVS): a forest stand-level modeling tool that simulates
forest growth in response to natural processes and management actions; used to
quantify effects for National Environmental Policy Act (NEPA) analyses.
• FVS Fire and Fuels Extension (FVS-FFE) simulates how fuels and potential
fires will change over time in response to management.
▪ LSim: a forest landscape simulation model integrates FSim into a modified FVS
Parallel Processing Extension (Ager et al. 2017).
▪ Scenario Investment Planning Platform (SIPP) or spatial scenario planning model
(FORSYS), formerly Landscape Treatment Designer (LTD): Ager Scenario Planning
Framework (scale of large fire events, more for resource allocation and budgeting).
o ArcFuels: an ArcGIS toolbar that uses FVS and fire behavior models to provide
decision support for vegetation management, fuel treatment planning, fire behavior
modeling, and wildfire risk assessments.
o Interagency Fuel Treatment Decision Support System (IFTDSS): an online system
that supports fuels treatment planning and analysis, including fire behavior modeling
under different conditions and processes for testing fuels treatment impacts on fire
behavior to determine priority areas for reducing fire behavior.
• Other:
o Risk Management Assistance (RMA): RMA is not a DST, but it was brought up by
interviewees as it is a form of decision support on fires. RMA teams travel to fires
and provide decision support in the form of analytics and discussion facilitation
(Schultz et al. 2020).
Decision Support Tool Definitions
As mentioned previously, the term DST means different things to different people. There is no
established definition of a DST in fire science or management. Therefore, we asked interviewees
how they define DST to get a general sense of common elements across disciplines and roles.
Interviewees spoke of a clear distinction between data, such as a geospatial data layer, and a
DST, which provides a higher-level function to facilitate risk-informed decision making. One
boundary spanner said, “An important distinction for me is that a decision support tool provides
some kind of capability to either evaluate scenarios or prioritize something. So, it has some kind
of higher-level function that evaluates or analyzes the data in a unique way to help someone
make a decision” (DST11). Another boundary spanner put it this way, “What I think separates
decision support tools versus any old piece of information is they’re set up to answer a specific
question, or help make a specific decision, and so they purposefully have multiple
considerations” (DST14).
Another distinction interviewees made was between an informational resource, such as a
website, and a DST as a tool or software program that can be asked specific questions. For
example, one scientist/developer said, “A decision support tool in my mind is something like
9
WFDSS or IFTDSS that are actual computer programs, software programs that provide people
actionable tools to ask questions and run analyses” (DST16). One manager also identified
programs like WFDSS, and said, “A decision support tool, I think of WFDSS, I guess. Pretty
much because that's what we use, that’s our decision of record” (DST17).
Some interviewees also emphasized that DSTs do not make decisions, rather they support
decisions. One manager stated, “It doesn’t make a decision, but it helps us take a quick look at
what we might expect and consider that when we need to move forward with a decision”
(DST21). Another manager noted, “I really look at them as a piece of information, not a
decision, not something to make a decision based totally on. That is just one of the pieces of
information that I throw into our decisions. It’s the best available science we have, so I’m pretty
adamant we use them, where a lot of other folks, they're like, ‘Oh, it’s just a model, I can
probably guess as well as the model does.’ But if I put that in a document, I feel like I have good
scientific backing behind this decision we’re making” (DST25).
Many interviewees described DSTs in broad terms. One manager described a DST as, “Anything
that helps inform a decision better than what you would have otherwise had available to you is a
decision support tool” (DST24). Another manager said, “So it’s anything from a fire behavior
model to the risk assessment model, the Zimmerman graphs that are in WFDSS right now … I
think those are all decision support tools that just help you think through the process and think
through what’s important to this decision” (DST25). Another manager stated, “A decision
support tool should be something that helps fire managers and agency administrators make
informed decisions using a deliberative process regarding wildfire management” (DST26).
The broad conceptualization of DSTs by interviewees corresponds to the definition used
throughout this report in which DST refers to a broad array of frameworks, models, analysis
approaches, and products and tools. One scientist/developer noted that the term DST is
misleading, as it is often used in this broad sense as anything that, “… helps inform managers
how to make better decisions” (DST4). The same scientist/developer suggested that another term
might be more appropriate. “I think decision support tools help people think about problems.
They shouldn’t be called decision support tools; they probably should be called something else.
Thought, thought analytics, you know? Or, I call it conversational analytics. It’s a way of
starting a conversation that takes advantage of all the tools” (DST4).
Decision Support Tool Development and Transfer
Six interviewees characterized themselves as scientists or DST developers and provided
information about the process of developing and transferring DSTs. Scientists/developers noted
that there is a recurring disconnect between some scientific work and the needs of managers in
the field. This disconnect manifests itself in multiple ways. For example, as one boundary
spanner noted, “Oftentimes scientists work on things that they think are important or the money
dictated they work on, but really aren’t helping meet the needs of the field. But every once in a
while, they make something that they really like, but we don’t have a really great mechanism
10
where scientists can fledge their science and it can go into operations and management”
(DST1). This comment highlights that there can be a disconnect between what scientists think
should be developed in terms of DSTs and what the potential end users of DSTs in the field need
to facilitate decision-making. This disconnect can also occur when an operationally relevant DST
is developed, but there is no clear process for taking that new tool and getting it into management
effectively and the capacity to do that work is limited. As the same interviewee later noted,
“There’s only a few people that can develop it and code it to apply it in management” (DST1).
Scientist/developer interviewees referred to multiple DST transfer approaches, with the most
common including in-person engagement, field trips, workshops, webinars, and online modules.
In-person engagement was highlighted as a critical element of successful DST transfer. One
scientist/developer provided an example of an ongoing, in-person DST transfer process that
resulted in building trust and relationships with end users. “It was just me there at these meetings
all day, twice a month, maybe more, just being willing to answer questions and interject and
explain over and over and over again. Particularly some of the quantitative pieces. And laughing
about it and enjoying it” (DST27). Similarly, field trips were cited as being effective for
communicating how DSTs work on the ground. As one manager stated, “And we brought them
out on a field trip and showed them all the work we had done ahead of time to decide where we
were doing what and why, and we got no pushback from them after that. They were like, ‘Now I
get it.’” (DST25). Workshops also serve a similar function in terms of communicating the utility
of DSTs and building trust, both in the DST outputs and the scientists/developers communicating
about the DSTs. One boundary spanner explained, in reference to workshops for developing
PODs, “Doing these workshops sometimes you get people who are like, ‘Ah, I don’t think this is
going to be very helpful.’ But by the end of the workshop, they see the utility of just having
everybody in the room, working through this stuff together” (DST10).
Decision Support Tool Use
Respondents were asked to describe how they have used or participated in efforts to use the
DSTs with which they were familiar. Interviewees provided examples of failures to use DSTs as
envisioned that nonetheless resulted in useful learning, as well as examples of successful DST
use. Additionally, some respondents referred to collaborative processes to develop context-
specific DSTs that are not referenced in the list of DSTs provided earlier in this report. Finally,
respondents referred to future goals and hopes for the effective use of DSTs.
Interviewees provided several examples in which DSTs were used to the best of a teams’
abilities but still resulted in undesirable outcomes. This highlights that even with the best
available DSTs, it is not always possible to prevent undesirable outcomes, but at the same time,
that does not mean that the tools or information were not valid. As one scientist/developer
explained, “I think that’s an important thing to be able to communicate that, like even when we
have the best information to try to make the best decisions, we’re not always gonna have a
perfect outcome because there are things that we just simply can’t control. But the hope is that
11
that doesn’t lead somebody to just say, ‘Okay, well then that approach to fire management is not
valid anymore because we had one bad outcome’” (DST16).
Another interviewee referred to the challenges of using DSTs in cases where stakeholders
already have pre-existing notions. In instances like this, it can be challenging to use DST outputs
if they do not validate these pre-existing notions. As one manager explained, “So you were going
to have people that disagreed from the initial response from the get-go. So, as it played out over
time, they continued to disagree depending on whose strategy got chosen under any given
condition. And, so they viewed the analytics much more of either validating their opinions, not
helping inform choices or validating the, quote, opposition’s opinions, and not helpful because
they didn’t just give an answer” (DST7). In order to address the issue of DST outputs not
matching the perceptions of fire managers, the same manager emphasized the importance of
having conversations about the use and role of DSTs early in a process and providing space for
local leaders to lead the conversation and build relationships.
In addition to lessons learned from unsuccessful attempts to use DSTs, interviewees provided
examples of successful DST use ranging from scene-of-action on a fire to long-term risk
assessment and planning. Interviewees highlighted the value of using DST outputs from WFDSS
and FSPro on a fire being managed for resource benefit to communicate with the local
community and build support and understanding of the planning that goes into letting a fire burn
for resource benefit. One scientist/developer explained, “And, so with all that information at our
fingertips and being able to show them we’re being really thoughtful about this … Generally, the
community was very happy … I think that for me it’s showed how powerful it can be, just on a
personal level, to be able to relay all of this fire modeling information to people and show them
that we’re managing fire with … the information that we can” (DST16).
Interviewees also talked about how longer-term risk assessments and DST outputs for planning
can also facilitate community understanding and support. As one manager explained, “Having
those maps and those documentations of why we were doing what we were doing and how we
chose to do what we do, I think, made a big difference” (DST25).
PODs were often mentioned by interviewees in describing success stories for pre-fire planning.
For example, one scientist/developer noted, “For specific actions on the ground, we often tout
the example from the Tonto National Forest, the Pinal Fire, and that’s out there in some
published sense as a true example of that fire response (see O’Connor and Calkin 2019). And the
importance of the discussions upfront and the pre-planning to facilitate a response that was
successful and was very beneficial in reducing risk for surrounding areas” (DST27).
Meanwhile, one manager highlighted that another value of pre-fire planning and successful
implementation of PODs mitigates fire hazards in the long-term, “Those are hard successes to
capture. That we haven’t had to mitigate hazards or expose people to putting out other
unplanned ignitions in all that country. And I know the day will come where … we’ll start getting
fires in those areas again, but we haven’t had to yet. And those are big wins in my book. We’ve
12
taken a piece of country that we’re responsible for and changed it, changed the hazard there. I
think it’s just hard to capture, hard to put numbers on that but it’s there. It’s real” (DST6).
DSTs are also used to support grant applications and provide justification for funding. As one
manager explained, “I used it two years ago to submit a proposal for Joint Chiefs project which
we were awarded … I used that as a justification and rationale for why we’re looking at these
various areas for large scale broadcast burn treatment and other types of manual, mechanical
work” (DST18). DSTs can also communicate collaborative agreement that can further boost
funding proposals. As one boundary spanner noted, “A 15-page document that has 20 signatures
of the diverse stakeholders, who’ve all said we buy into this thing, when you attach that to a
grant application or another type of funding proposal, you’re in a much better position to get
funded, because from a funder’s perspective it takes a lot of the guesswork out of whether this is
just one organization or one government opinion of what needs to be done, or 20 organizations
and governments’ opinion of what needs to be done. It gives them an assurance that this is a
well-thought-out proposal that’s part of a bigger strategy” (DST19). DSTs can also improve the
narrative and rationale for how funding will be used. As one scientist/developer explained,
“Managers are motivated and the people that have a better storyline about what they’re going to
do with their budget in terms of improving landscape conditions and meeting targets sometimes,
have a better chance of getting supplemental funding” (DST3).
Interviewees also referred to efforts to develop project-specific DSTs to assist collaborative
groups in prioritization. As one boundary spanner noted, using formal, complex risk assessment
tools is often beyond the means of smaller organizations. “And one of the challenges with
smaller nonprofits or organizations that are building some kind of risk assessment to make the
business case for the kind of investment they need to actually scale things up to have a
meaningful impact, is that they oftentimes don’t have the money on hand to really utilize
something that’s a large landscape scale planning tool” (DST11). One common approach for
smaller groups is a tailored process that brings together local partners and data. For example, one
boundary spanner referred to a process to develop priority areas for treatment that used local data
depicted to a collaborative group through large table-sized maps to facilitate a discussion about
prioritization. “We relied pretty heavily on local and expert knowledge to help us, to guide us
through the prioritization piece of it. And we identified four different areas in what we called the
landscape resilience strategy … It was a completely different, more open discussion, more
collaborative-led prioritization process. And the science served as the foundation or the hub that
conversations could rotate around and it was scientifically much less complex and but socially
more engaged than some of the other risk assessment work” (DST15). Interviewees noted that
some of the benefits of grassroots processes is that they facilitate relationship building and trust
among partners in a landscape but often require sustained effort and time to accomplish.
13
Decision Support Tool Barriers
Persistent tensions, disconnects, and cultural issues impact the use of DSTs
Responses from interviewees across the spectrum of science, management, and boundary
spanning organizations illustrated several persistent tensions and divergent perceptions of the
role, development, and use of DSTs for wildfire risk assessment and treatment prioritization
planning. These tensions manifested in two primary ways.
First, responses highlighted a tension between a desire for top-down directives on what DSTs to
use and how to use them and a desire for a more organic, grassroots, bottom-up development and
integration of DSTs. One manager articulated a desire for the grassroots approach to developing
and integrating new DSTs. “Everybody wants to be the tip of the spear, be the most advanced or
whatever, but you got to organically build that within your fire program for it to be a useful
tool” (DST24). Meanwhile, another manager acknowledged that a lack of interest in a new DST
at the local level warrants more top-down intervention. “I think there needs to be really tight,
good communication from the top down, and each individual district on the forest has their own
culture, and so a lot of times … All it takes is one or two people to say, ‘Hey, I’m not interested
in this,’ to blow the whole thing up. I think it really needs to be a top-down thing where, from the
Forest Service perspective, it forces you to say, ‘Hey, this is what we’re doing, and I want you to
make sure that all of your staff is coming along with this and moving it and staying on tap.’ And
then having those unit line officers, the district rangers giving that same message so everyone’s
on the same page from the communication standpoint” (DST23).
The second tension articulated by interviewees was a tension between a desire for scientists to
co-develop tools in concert with intended end users and a desire to not take precious time away
from intended end users in experimenting with a new DST that may not be adopted. One
manager summarized it this way, “Instead of the big workshops, computers, remote sensing
exercise, if you really want to make this valid, maybe take a few more steps, and bring in the
implementing part and relationship part, and go get into the weeds a little more, if you wanted to
reflect reality more. And then that’s a big rabbit hole, because when I complained about before,
it was like, ‘Well, am I going to take your finite time and ask you to go do something like I’ve just
described? And what is that going to do for you?’ Because … that might not add up to anything”
(DST12). This illustrates an ongoing disconnect between science and management that was
similarly articulated by another manager. “There’s the disconnect between the people that are
running and building these tools and the folks that are ultimately using the tools, and the
understanding of the intricacies of the landscape that they work on and the data that they would
need to run is maybe not there” (DST22).
Interviewees also alluded to persistent cultural issues that impact the use of DSTs. For example,
interviewees referred to the challenge of changing existing mindsets and cultural issues that
impact perceptions of the role of fire. As one manager noted, “We haven’t had that cultural shift.
It’s still, despite the messaging, very much viewed as, fire is a bad thing and the less of it we can
have the better” (DST24). These cultural barriers impact perceptions of the role of science, as
14
well. As one boundary spanner put it, “I’d say there’s a quiet undercurrent in fire to not
embrace science” (DST1). Another scientist/developer referred to it this way, “Almost like, if
you show me a map that says that there's an X percent probability of the fire reaching this place,
I’m going to take it as a challenge, and I’m going to make sure that doesn’t happen … There’s a
real challenge to changing people’s mindset to have a little bit more patience to not always want
to just get in there with the heavy machinery and try to go direct” (DST16).
Scientists/developers also brought up the challenges of trying to shift paradigms and propagate a
risk-informed decision-making approach throughout the agency. For example, one scientist/
developer stated, “It’s a bigger challenge to create decision support concepts and get those
propagated and understood than it is to create a model that goes and does something on some
national forest” (DST4).
Lastly, scientists/developers referred to cultural barriers that impact public perceptions of the
role of fire and the challenge of communicating fire management approaches like allowing fire to
burn. “On the community side, it’s a really hard sell. I feel like those of us in science are always
trying to make the sell and get the public messaging out that wildfire is not always bad on the
landscape … But when people are pounded over and over again in the media and other
messaging that it’s devastation, destroying landscapes, the adjectives are always so terrible
about the effects of fire. And, so it’s really hard to come in and then tell people, “Hey, look at
fire, it’s really actually not all that bad, and we’re going to be patient. We’re going to let it burn
up there …” I would say cultural barriers exists all around smart and thoughtful fire
management” (DST16).
Capacity issues inhibit both the development and use of DSTs
Interviewees across respondent groups referred to numerous capacity issues that impact the
development and use of DSTs. Capacity issues were described in general and specific terms and
occur in science, boundary spanning, and management organizations. As one scientist/developer
noted, “I think we need capacity on both sides. Those folks that can communicate and work with
groups, as well as those folks that can create the analytics. And both lack … There, of course,
are good people out there that can do it, they’re just tasked with other things” (DST27).
One of the most persistent capacity issues that interviewees brought up is simply a lack of time.
As one manager noted, “I think there’s a time component where we have a full program of work
for the people that are managing the forests and doing the work in the woods, and there’s limited
capacity to add anything new onto what they’re required to do” (DST22). Another manager
explained, “As a ranger, we’re so busy that I don’t really have time to take a deep dive into
stuff” (DST13). One boundary spanner said, on a similar note, “They’re so strapped for time that
some people might even have the skills to do some of these things if they actually had the time to
sit down and think and learn and work their way through a process like this” (DST11).
Interviewees also noted that there is a general lack of personnel capacity across land
management agencies, especially the Forest Service, that impacts the development and use of
15
DSTs. As one manager said, “And as you probably know, across the Forest Service, we just
don’t have the capacity to get a lot of work done. We’ve got a lot of vacant positions and that
causes a lot of other issues” (DST23). Turnover in personnel also leads to a persistent need to re-
educate and familiarize new staff with emerging DSTs and strategic fire management paradigms.
As one boundary spanner noted, “Now line officers’ turnover, on average, is about 20 percent of
them every year. So, in five years you’ve got a whole new set of line officers who need to become
knowledgeable at frames et cetera in fire management business” (DST2).
More specifically, interviewees noted a lack of personnel in the agency who have the scientific
background and analytical expertise to learn and use new DSTs. One scientist/developer noted
that existing technical experts on local Forest Service units are often too busy to take the time to
integrate new analytical approaches into their regular program of work. “What we find is that the
GIS professionals within the agencies, they’re so swamped with just their regular GIS path,
managing vector data set, making maps for different staff areas. The idea of doing … analysis
and detailed fire behavior stuff, they’re like, ‘Ah, no, I can’t do that.’ I think there’s a real lack
of people with the level of understanding and analytic skills to be able to help implement this
stuff broad scale” (DST16). Another manager noted that technical experts on local units are
often not directed to use new DSTs. “There’s also a problem with utilizing the folks with
technical skills to facilitate these kind of analyses where we might not be prioritizing or thinking
about what capacity we have to do more technical analyses” (DST22).
Interviewees also noted that existing analysts and scientists within Forest Service research and
enterprise units are oversubscribed. As one scientist/developer explained, “We're a fairly small
community of people doing the analytic work … It’s a lack of capacity building within the public
management agencies on the analysis side. We feel like there needs to be a whole lot more
people like us with enough understanding of the science to be able to explain it well and
understand it and to be able to do these analyses” (DST16). Another scientist/developer noted
that external analysts struggle to generate local ownership in new DSTs. “The biggest gap we
have is there is a small number of people who can actually develop the models and facilitate a
full workshop and most of them are here on our team. So, the other issue with it being located on
our team is not only the lack of capacity to cover all of the areas, but also that issue of
ownership. If it were a process that the regions could own or the forest themselves could really
engage with a little more directly, that would be, I think, a real positive” (DST3).
This lack of capacity also manifests itself in a lack of technical transfer specialists or boundary
spanners, who need to be comfortable working in both the realms of science and management.
As one boundary spanner noted, “I don’t think there’s enough people who act in that tech
transfer role. The managers that I work with the most are the ones who are inclined towards
technological, scientific approaches. There’s a lot of managers out there who aren’t. And I think
you need someone who’s familiar in both worlds in order to make that happen” (DST14).
Meanwhile, one scientist/developer argued that the individuals developing new DSTs may not
always be the best people to teach others how to use those tools. “I might not be the best teacher.
It’s not always the same person developing the tools that should be the champion or the one
16
doing the outreach for the tool, which I think is really true in science and research in general,
researchers aren’t always the best communicators” (DST5).
This lack of analytic capacity within the agency also impacts external scientists/developers and
boundary organizations that work with the Forest Service. Another scientist/developer noted that
the lack of analytical capacity in the agency ultimately leads to a lack of capacity in research.
“Instead of advancing research, we do a lot of the development and application piece because
we know it’s important to do that. We think it’s important to do that to affect real change, and so
we do it. But we should be on the research side, and we could continue to advance these tools if
we had the time to even sit down and do so” (DST27).
Funding issues also present persistent capacity issues and can manifest in different ways. For
example, one manager described a learning experience during the development of a new risk
assessment framework, for which the funding concluded when the final data was delivered but
did not include any funding to support the local unit on integrating that new data into the
program of work on the forest. “I didn’t really think that through when we first started down the
process, and so we’d used up all the money by the time we got to the end of it” (DST25).
Meanwhile, another manager brought up another issue, which is the perspective that the limited
funding that is available should ultimately be focused on getting acres treated. “How to get
funding into projects and get acres treated is my job. I always remind myself not to get wrapped
around the axel with all the other stuff that gets thrown at you, about the fancy new widget for
how to get something done” (DST12).
An ineffective process to integrate a new DST into a program of work can limit its outcomes
Interviewees referred to procedural challenges that can inhibit effective integration and
application of DSTs. One of the most referenced process barriers was a lack of participation or
missing participants, which can lead to a lack of buy-in and trust, especially for those who may
not have participated. For example, managers and boundary spanners alike noted the challenge
of getting collaborators to the table in the first place. One manager noted, “That’s one of the
bigger challenges is just to get people to sit around the same table” (DST18). Another manager
similarly explained, “We’re not quite there yet in terms of having complete buy-in from everyone
on what the priorities are but we’re working towards that, and that’s one of the challenges that
we’re starting to acknowledge, it’s hard to get all of the voices to the table to make sure you
didn’t miss anybody” (DST23). Furthermore, when integrating new DSTs into a program of
work is considered optional, a lack of buy-in can halt progress. As one manager noted, “It’s hard
to get something done that’s optional, yeah. People gotta really buy into it” (DST6). Simply put,
this can result in a lack of creative thinking. As one manager noted, “To some degree there’s
limited desire to re-think the processes that they have been using in the past” (DST22).
Many managers also noted that within the Forest Service, there are diverse perspectives among
personnel from fire to fuels to resource areas. One of the perspectives often missing that several
managers noted was resource specialist perspectives, which can include archeology, biology,
range management, and others. One manager noted that, in many cases, resource specialists have
17
been frequently asked to try new DSTs that have not led to meaningful outcomes, which has
resulted in some resistance to new DSTs. “I think a lot of resource specialists feel that, if it’s not
going to really lead to anything meaningful, they’re not interested in investing a lot of time into
it. And I can identify a little bit with where they’re coming from because sometimes they will go
through some sort of assessment like that and then we fall back internally in our ID teams to
what feels comfortable and what we know and which typically doesn’t always incorporate those
tools” (DST23).
Interviewees also noted that even when there is participation to develop and integrate a new
DST, that a lack of interest in the process and poor communication can create additional barriers.
For example, one scientist/developer described a process that lacked a champion in certain key
leadership positions. “I heard in the meetings … ‘Well, we only did this because we had to,
because 5140 tells us that we have to.’ So, there’s clearly no champion there, and I think that
work was never gonna really be used for anything important” (DST9). These types of issues can
arise, in part, because of poor communication from the appropriate channels about the need for a
DST. As one manager explained, “There wasn’t a lot of communication about the need for this
type of tool, or how this type of information would impact the ultimate users on the districts
coming up with their five-year plan” (DST22).
Several managers also noted that the process to develop and learn new DSTs takes time. An
efficient process can cause challenges by requiring an added time investment. For example, one
manager noted, “There was an ebb and flow in terms of how much engagement I had, and the
rest of the people not doing the analysis day-to-day had. That didn’t make for an efficient
process from my perspective … I had to go back and fix some things that I didn’t notice until late
in the game” (DST22). Furthermore, it takes time to learn new tools, and it takes even longer to
shift paradigms. As one manager noted, “The criticism that we hear is it takes a lot of time … I
went into the meeting thinking this, and I had a really good conversation, I understood why I
thought that better at the end of it, but that didn’t change anything, so I don’t know if it was a
good use of my time” (DST17). As one boundary spanner noted, there are tradeoffs between the
process to develop and integrate a DST and the DST itself. “And it’s always a fine line between
what is the process that you want to develop versus what is a tool you want to develop”
(DST11).
Technical challenges in the use of DSTs can impact perceptions of their accuracy and usability
Interviewees referred to technical challenges that create barriers to the use of DSTs and impact
perceptions of their accuracy and usability. The most frequently cited technical challenge was
limitations with the underlying data or models that serve as input to DSTs. As the famous saying
goes, “All models are wrong, but some are useful.” As one scientist/developer noted, “The best
way to uncover bad data is to start using it” (DST4). Nonetheless, when models are perceived to
be “wrong” it can lead to a series of challenges. As one manager put it, “For some people on the
ground-truthing side of things, it becomes … if it’s not 100 percent, the model has failed. If the
model doesn’t give me the answer of: the fire, will definitely, on day three, at approximately 4
PM, go up that drainage. And if it doesn’t, the model has failed” (DST7).
18
Model or DST limitations were something that scientists, managers, and boundary spanners alike
agreed create challenges. One of the limitations are the data that serve as input to DSTs. One
boundary spanner explained, “The outlook that we’re going to get for these DSTs is heavily
influenced by the data that is available. And if you don’t understand that limitation, you might
just run with the results without factoring in those considerations that are a little more difficult
to map” (DST14). Scientists/developers were also quick to acknowledge this challenge. “We
might have a false sense of knowledge from some of our assessments and those could lead to bad
decisions. So, I think we need to maintain our awareness of the limitations of our assessment and
our knowledge about, especially about natural disturbances” (DST4).
DSTs are limited in terms of how much and what type of information they capture and analyze.
As one manager noted, especially with firefighting, there are often many other considerations
that impact decision making. “As we look at a landscape, you know pre-fire, there’s so many
inputs and factors such as where we are at nationally with firefighting resources, where exactly
are the fuel moistures at, political considerations like fire restrictions … trying to do that stuff in
advance of the of an incident without knowing any of those details that I just mentioned is a
challenge” (DST20). One potential outcome of DST limitations is a resistance to new DSTs and
a reliance on what is already familiar. As one manager put it, “We tend to use models that we
have control over, that we understand better, and we set up ourselves. Because we know the data
coming in, we know the limitations of the data coming out” (DST22).
Data management issues can also impact the use of DSTs. If data are not stored somewhere
easily accessed, it can lead to problems during the development of DSTs or updates to existing
DSTs. One manager explained, “The issue is that as we try to update what makes this corporate
knowledge, there’s often a gap on where things were filed in the T drive, for example. Having
that information at a corporate level and consistent and updated at least annually, maybe more
than that, would be ideal. But I think that’s the struggle” (DST20). This can happen with data
collected during fires, as well. One manager explained, “When we used to make decisions on
wildfires, we would do all this analysis, and all this really good work. And then the fire would go
out, and that data, all that work, these reports, they just got left on someone’s hard drive, or they
got stored in a box in a district office and never to be accessed again” (DST8). Data
management is also a challenge across organizational entities doing longer-term, cross-boundary
planning. As one scientist/developer explained, “State GIS and federal GIS people have tons and
tons of data, but there really lacks a coherent way to bring the data together: stream layers, road
layers, veg layers, this layer, that layer. They’d show us 20 different layers and no one had
thought, well, how does all of this contribute to a decision in a sort of thematic way?” (DST4).
There are also inherent differences in data across geographic and temporal scales that can
contribute to challenges in developing and using DSTs. One manager explained there can be a
tension within Forest Service regions about what data sets provide the most appropriate outputs
for certain scales. “I have heard in the past where some of these regional modeling efforts have
used the best available data from the regional perspective, where individual forests might have a
better perspective or a better data set to use for their specific forest, or better modeling
19
parameters, or assumptions. And when you do it at too large of a scale, you lose that ability to
make it applicable for any particular place on the landscape when you run it across large areas”
(DST20). Certain geographic areas have numerous DSTs that overlap geographic scales, which
can create confusion in selection and interpretation of outputs for decision making. At the same
time, one scientist/developer emphasized that there are tradeoffs in accuracy and outputs across
geographic scales. “It’s an issue that when you downscale and there's some things that maybe
miss or maybe particular for one portion of the landscape. But when you’re making regional-
level decisions of how you might allocate dollars or resources, the Forest Service likes to
maintain that continuity so that the region can do that” (DST27).
Interviewees also noted a desire for one DST that can make decisions across all geographic
scales. One boundary spanner said, “A risk assessment at the national level is not going to look
anything like the risk assessment at the regional level, and that’s not going to look anything like
the risk assessment at the local level. They’re making different decisions, and accordingly, they
have different considerations. And asking one tool to effectively make decisions across multiple
scales … We’re asking more than we can possibly get” (DST14).
Another technical barrier that interviewees mentioned are technology malfunctions and
limitations. One manager noted that if a DST does not work in the field, people are likely to
resort to the tools with which they are familiar. “When you’re running a fire, if you have to
monkey with your iPad, and now this isn’t working and whatever’s going on, you’re just going to
go back to your steno pad. They don’t have time. People that are implementing this stuff don’t
have time to mess with it” (DST17). Integrating new DSTs into existing platforms can also take
time to work out successfully. One manager noted, “When we put PODs in WFDSS, they show
up with the same ugly, fat boundaries and you only can give them one value … It’s really clumsy
right now and hard to use in WFDSS” (DST21).
Lastly, interviewees noted that due to the limited longevity of some new DSTs it can be time
consuming to keep up with changes and can create frustration and confusion. One manager
referred to recent changes in day-to-day computer platforms in this way, as well. “We used to
have the JKNL drive, then we went to Pinyon, now we’re going to Teams ... The tech keeps
changing, but it’s hard for people to keep up quite frankly” (DST17). Meanwhile, one boundary
spanner noted that a possible solution to limited tool longevity is to develop smaller-scale DSTs
tailored to a unique context. “I’m very much well aware of the findings that people have seen
about decision support system use and longevity … Potentially, if you’re not spending a ton of
money on things, then it’s easier to make something that’s very tailored to a specific use in an
organization without making it overly burdensome to develop something that might be replaced
in two or three years for something that’s newer and better and fancier” (DST11).
20
Complex information in DSTs can be difficult to understand and can lead to confusion and
information overload
Boundary spanners and scientists/developers working to develop and apply DSTs acknowledged
that the concepts and information can be quite complex. As one manager explained, there is
simply information overload. “We just have a hard time, almost like information overload. The
fire managers, we feel like we have our hands full just trying to manage people and fleet and
budgets, et cetera. And, so these new tools, or new to some of us, they’re difficult to understand,
and if you don’t use them, they’re easy to forget” (DST20).
Another source of complexity is a lack of a common language to describe risk. One boundary
spanner explained, “Sometimes it just gets confusing for people if there’s so many different
options out there of how you prioritize, what a risk assessment is, and what it means. Our group
has tried really hard to standardize that and tried to make sure that everybody’s using the same
language” (DST10).
Another scientist/developer noted that it can be hard to select a DST without a foundation in
certain scientific and technical concepts. “There’s still understanding the fundamentals. The
basis of a lot of the things we do is fire behavior modeling, and there is definitely, I think, some
barriers to people understanding where to start or where to learn just the basics to fire behavior
modeling. Yeah, you can get into the system and click the buttons, but are you producing
meaningful outputs?” (DST5).
Managers also noted that the large number of DSTs available can be overwhelming. “It
definitely can get overwhelming to even stay up with everything that a ton of different folks are
doing, is really difficult to do. I don’t know if there’s an answer for it. I think every tool could
have a use. I'm not convinced they all do all of the time, but with the ever-growing list of tools
that are available out there, that’s more time that people need to learn a little bit about every
single one of these things” (DST24). Scientists/developers also noted that the sheer amount of
information available and the numerous different approaches for conducting a risk assessment
and prioritization process can be confusing. “It’s really easy, even for somebody like me who
now works in this stuff day in, day out, to get confused. And, so it can be really easily confusing
for the layperson and even for the fire management officer on a forest or a land use planner for a
community or a forest supervisor who doesn’t work in wildfire to understand this” (DST16). In
some cases, there is also a lack of awareness that certain DSTs are available in the first place.
Interviewees noted that even when there is an awareness of existing DSTs, there may still be a
lack of understanding of how to use them. As one boundary spanner explained, “Most people are
somewhat familiar with the different fire models that are in existence. But it still seems to be
challenging for some incident management teams to get their heads around how exactly they’re
going to use that information” (DST8). One scientist/developer noted that it takes time and a
concerted effort to explain the logic behind certain DSTs and develop an understanding of how
to use the outputs. “There’s a lack of understanding of the information fundamentally, and then
a lack of its intended use, I would say both … It’s fairly complicated, and it took a long time to
21
really slowly explain and work with everybody on explaining that” (DST27). Indeed, there is a
learning curve associated with any new concept. As one manager noted, “The short version is it
took them two years to get the local folk online and on board” (DST21).
Finally, interviewees noted that there can also be a lack of understanding or misunderstanding of
model outputs, which can be problematic for decision makers. As one boundary spanner noted,
“The pro is, you can do some predictive modeling, and get an answer, and have something as a
discussion point. The con is, maybe there’s more people in there that are running models that
don’t necessarily know exactly what they’re doing. And maybe they’re making some mistakes,
and that can be misleading to a decision-maker” (DST8). One boundary spanner explained that
this can turn people off to using DSTs in the future. “The danger though is people start using a
lot of different analytics and if some of them don’t work then some people will get put off by any
analytics … because they’ve had a bad experience with maybe one of them” (DST2). Or, as
another boundary spanner argued, if DST outputs are problematic, decision makers may move
forward without them. “It either validates what they’ve been thinking and then they’ll use it to
support … or it is contrary to their beliefs, and they move forward without it” (DST15).
Institutional issues and governance barriers can inhibit the use of DSTs
Institutional issues within the organizations responsible for developing, transferring, and using
DSTs can inhibit their use in several ways; although, the barriers within organizations tend to
vary widely based on the context. As one boundary spanner noted, “There’s probably different
kind of barriers to use within different types of organizations. And I spoke a lot about a non-
profit using risk assessment tools. I engage with agencies using risk assessment tools within
collaborative processes. And then I think I gave the example of someone trying to use IFTDSS
within their own organization. And I think there’s unique barriers in all those different contexts
depending on exactly how they view their use of those tools” (DST11).
Within land management agencies like the Forest Service, interviewees explained that one of the
major institutional barriers is a lack of direction on which DSTs to use and how to use them.
Across the administrative levels of the Forest Service, interviewees expressed a lack of clear
direction and alignment on DST use. As one scientist/developer noted, “The leadership intent,
they don’t do intent until they’re educated about a potential or an opportunity to do intent with a
decision support tool. But they won’t get educated unless a scientist spends a lot of time
somehow educating them on these, which is a difficult process” (DST4).
A lack of leadership intent regarding DST use can result in confusion across different
administrative units of the Forest Service. As one scientist/developer noted, “There just hasn’t
been a lot of help out there to direct forests on, okay, so now how do we take that next step of, we
have all this great information, how do we integrate it into our program of work?” (DST9).
Although there is a direction in Forest Service Manual 5140 to use risk-informed decision
making for pre-fire planning, some interviewees noted there can be a reluctance to use the
outputs of those analyses once they are available across all levels of the Forest Service. Or in
other cases, as the same scientist/developer noted, DST outputs just sit on a shelf. “It just seems
22
that a lot of them, they get done, and they get put on the shelf, and then they’re not really
integrated into a statement of work or a program of work of any particular unit” (DST9).
Another issue that interviewees cited specific to the Forest Service are restrictions about the use
of technology. One boundary spanner described this as CIO governance. “The way I would
describe it is just general computer technology. We call it CIO governance that enables us to be
able to do the tech transfer of research to mobile devices or to computer applications. There just
seems to be a delay on the government side to effectively support emerging research to get some
of their ideas or some of their tools to the masses … One of the biggest impediments is
technology and the advancement of things is moving so fast. It’s as if the government can’t keep
up with those changes because we’re not able to adapt as quickly as technology changes. So,
you’ve got researchers that are coming up with really cool stuff, and they’re usually able to
leverage the new IT technologies” (DST8). There is also limited IT capacity that results in
analysts devoting time to IT issues. As one boundary spanner noted, “If I had half a dozen folks
in IT to help, then I could free up my great analysts and tech transfer folks to do more tech
transfer … To be fair to IT, they’ve been shorthanded, and it’s very difficult for them to hire
folks” (DST2).
There are also barriers within scientific organizations. Specifically, interviewees noted that the
science reward system does not incentivize scientists to work on development and application of
DSTs for end users. As one boundary spanner noted, “If a scientist makes something really
useful that the field needs, there’s not a lot of incentives on their end to keep it going because
that’s not really how they’re structured in research” (DST1). Another scientist/developer noted,
“The struggle is, someone who’s supposed to be in research is balancing that and the cost of
doing that with carrying out a research function because you’re not rewarded” (DST4).
Lastly, one boundary spanner noted, infrastructure with which new DSTs can be incubated and
tested is largely nonexistent. “We struggle with, how do we develop stuff? How do we answer the
fields’ questions? Once we’re onto something, how do we grow it, and then implement it, and
teach it, and maintain it? We just don’t really have a mechanism to do that” (DST1).
Strategies for integrating DST outputs into decision making processes are often lacking
Many interviewees highlighted that one of the final challenges in the process of developing and
using DSTs for risk assessment and treatment prioritization pre-fire planning is a lack of
implementation plan for how to integrate DST outputs into existing decision-making processes.
As one scientist/developer noted, there is a tendency for efforts to develop new DSTs to end with
the data itself, and less attention is paid for how to integrate the new information into a planning
process or program of work. “A big issue is, then, the forest and regions, they get the data, and
they say, ‘We have our QWRA, now what?’… ‘We have our PODs, now what?’ And people don’t
know how to interpret it, so they need a little bit of assistance or a little bit of additional training
on how to interpret it” (DST9). One manager had a similar comment that highlighted that there
is often a lack of implementation plan for using new information. “We have a host of people
using this data that aren’t as savvy with the data and all the uses … there’s so many different
23
ways to use it, and I just am skimming the surface with what I put together for our folks to use”
(DST25).
One boundary spanner noted that even when end users can run new DSTs, they may not know
how to effectively use the outputs. “You might be able to run something like a linear
optimization tool and get a table output, but then for someone to take that and attach it spatially
to that data, visualize it, put it into a GIS and those kinds of things. Those are often next
barriers” (DST11). Another scientist/developer had a similar observation. “The tools aren’t
challenging, I mean they take a while to build, but that’s not the hard part. It’s the interpretation
of the outputs that you’re using to make your decisions and really having solid foundation and
training to understand what you’re looking at is really important. And that’s something we
struggle with a lot, being the ones developing the tool because our goal is to develop the tool.
It’s not necessarily to teach the science. We have to, of course, teach and train to a certain level,
but I feel like getting other people to really take the torch and understand and teach the
foundation and the understanding of how you use these outputs and maps and charts to make
more informed decisions is really important, and it’s really challenging for those developing the
tools” (DST5).
Interviewees also expressed concerns that implementation of DST outputs, especially during
fires, can be prevented by decision-makers at a higher level for various reasons. For example,
one boundary spanner relayed a concern that had arisen at a PODs workshop. “What I’ve heard
from the field … is that they are concerned that they’re going to have these plans in place and
then upper management won’t let them implement it. And even though they’re drawing these
PODS, they’ve got these great plans in place, they feel like if they get a fire in certain areas, and
they’re like, "Oh, we should let it burn,” that somebody much higher than them is going to come
in and say, ‘No’” (DST10). The same boundary spanner noted that there are also concerns about
a lack of resources to implement a DST like PODs during a fire. “They’re concerned that if they
have a POD where they feel like they can manage a fire successfully, but managing that fire
requires them to have a certain amount of resources available to do just that, if they decide to
manage that natural ignition, they go very low on the totem pole in terms of resource allocation
need. So, if a higher priority fire needs the resources that they have, it gets pulled off from their
fire, and then all of a sudden, the plan that they had in place can’t be maintained because of
resource shortages … The ability for the forest to maintain resources on these managed fires …
seems critically important in my mind” (DST10).
Decision Support Tool Benefits and Facilitators
DSTs spur new conversations and facilitate communication
One of the most frequently mentioned benefits of using DSTs in pre-fire planning is that they can
help to spur new conversations among fire staff, line officers, resource specialists, and other
decision makers and stakeholders. Interviewees noted that DSTs help to generate discussions
among staff that were not common in the past, especially when the development of a DST
involves collaborative participation from end users. As one scientist/developer noted in reference
24
to PODs, “Through all of this process, one of the biggest and most beneficial outcomes is
conversations between the fire folks and line officers and their resource specialist staff.
Conversations that have rarely happened in the past. The fire shop is even in a separate
building, typically, and they don’t engage and speak with the other Forest Service staff on a
regular basis, so it has helped bridge that, and those are important conversations that can lead
to downstream beneficial decisions” (DST27). When managers from different resource areas or
functions of the agency gather to discuss pre-fire planning, a DST and its outputs can serve as a
common space from which to share varying perspectives.
Although DSTs may have flaws, they can still provide a forum for joint decision making. As one
boundary spanner noted, “I think the value is in using these tools as a communication tool in
order to jointly make decisions. Or, at the very least, understand the perspectives of other people
in the audience” (DST14). Similarly, another boundary spanner noted that DSTs can facilitate
landscape-scale thinking, “I think that the decision support tools work really well as an excuse to
get people to the table and start having conversations at a landscape scale” (DST15). DSTs can
also facilitate communication across levels of the Forest Service. For example, one boundary
spanner provided an example of a case where the use of PODs on a fire had allowed a District
Ranger to communicate decision making rationale to the forest supervisor and regional forester.
Another manager mentioned that they are important for documenting decision making, “Why
folks made the choices that they did early on” (DST7).
DSTs can also facilitate conversations between land management agencies and the public. As
one boundary spanner noted, “We’re talking with [sic] our partners about this stuff long before
the fires ever occur” (DST1). DST outputs can provide communication tools that allow fire
managers to better explain their decision-making rationale to the public. As one boundary
spanner noted in reference to PODs, “It can be a great communication tool to explain why we’re
actively suppressing a fire on one portion of the landscape, while we’re letting a fire burn in
another portion at the same time. Realizing that all fire in all places is not the same” (DST10).
DSTs can also help bring community members into the conversation about pre-fire planning and
suppression during fire events by demonstrating the amount of planning that goes into addressing
fire events. As one scientist/developer noted, “I think the more we can bring public and
community, especially engaged community organizations, into those conversations to show
people that we are really trying to be as thoughtful about how we approach wildfire as possible
and to do as much pre-planning, pre-thinking as possible so that when events happen, we’ve
already thought through how we’re going to deal with them” (DST16).
DST outputs, such as visuals and maps, are essential communication objects that allow DSTs to
facilitate discussions and joint decision making. For example, one manager referred to the
process of developing POD boundaries to capture the collective knowledge of a group. “We had
a bunch of pre-made maps out. And the good part of it was we had the fire management officers,
line officers drawing maps together, putting lines on the maps and thinking about things, from
all the various history that was in their heads in addition to all the data that went into the
PODs” (DST13). Visuals developed from DSTs can also facilitate creative thinking. As one
25
manager noted, “That graphic illustration gives you some pause and makes you ask some
different questions. It doesn’t give you any answers. It just helps understand a frame for the next
choices that you’re faced with” (DST7). Maps and visuals are also helpful in explaining the
potential impact of a fire to the public and can help managers more effectively make arguments
for fuels treatment and restoration work. As one manager said, “Being able to explain the real
need for doing this is really what we’re trying to do, and that risk assessment, to be able to show
them a map that, overall, this one area is very high risk and the potential impact to the values
could be very severe. That really helps, just visuals, really help tell the story to the public, and
that’s what we’re going to use it for right now” (DST 25).
The process of developing and using DSTs impacts their perceived effectiveness
One of the key facilitators in the development and use of DSTs is a collaborative process in
which DSTs are co-developed with multiple perspectives in an iterative process that facilitates
building relationships, trust, and respect. This is often referred to as the co-production of
knowledge, which is a collaborative process to co-develop actionable scientific information for
decision-making. Organizations that span the spheres of science, decision making, and policy,
often referred to as boundary spanning or bridging organizations, tend to have unique functions
that allow them to facilitate the co-development of scientific knowledge and DSTs. Interviewees
frequently referenced these concepts as facilitators to the development and application of DSTs.
As one boundary spanner noted, the process of developing DSTs is critical to their perceived
effectiveness. “I think that all these tools have flaws. I think that it’s really hard to get over a lot
of the flaws. The value that I see in these decision support tools and these planning tools is in the
process. You all gather around what the tool shows you and how that aligns with your current
understanding of the landscape versus your colleagues’ understanding of the landscape”
(DST14). One manager put it this way, “The biggest thing was partnership buy-in and
collaboration … These tools are helping us literally come around the table and agree to doing
the same thing, regardless of who you work for” (DST18).
Interviewees referred to numerous factors in the process of co-developing DSTs that help to
enhance their role in decision making. Interviewees noted that an iterative and interactive process
for developing, testing, and using DSTs can increase their operational relevance. For example,
one scientist/developer noted, “And, so it’s kind of this interactive process of bringing
information to the field and having them confirm or reject … This iterative process of working
with the fire managers so that they’re owning it, it is their information. We’re just helping
support the development of it and learning where our model’s working well and where our
model’s not working well became really powerful” (DST3). One manager also highlighted
positive outcomes from an iterative process, “It was a conversation, yes. Let’s move forward on
it, and then we kind of revisited things as it progressed. But it wasn’t one get together about, ‘we
like this, let’s do this.’ It was an idea … Let’s see where we can work with it and then presented
it to the district ranger and got the support” (DST6).
26
Another important factor in the co-development of DSTs is the value of having multiple
perspectives represented, both in terms of providing necessary information, as well as in
generating support. As one boundary spanner explained, “Some of the factors are that the
managers themselves, the FMOs, the engine captains, up to the rangers and the forest
supervisors, the resource specialists, they were all involved in the process … The fact that lots of
people were involved, the fact that the Forest Service had meetings with their cooperators after
the fact in order to give them updates on what was happening, I think that helped generate some
support and buy-in and trust” (DST14). Furthermore, participation in the co-development of
DSTs from leadership and support from leadership for using DSTs can drive progress. As one
manager noted, “The fact that leadership was involved and pushing to get it done, and to come
up with a new way of looking at the problem was very useful” (DST22). Overall, an iterative,
participatory process helps to generate critical support and buy-in from end users. As one
manager explained, “It was good to have resource specialists involved in the discussions and
working groups to assign response function. And I think it created a fair amount of buy-in
because having the right people in the room to be a part of the process is more or less a verbal
agreement about the end products, especially with agency or line officers, so if that situation
arises where there’s a disagreement about how to manage a wildfire, there’s at least a start
point to say, ‘Hey, you remember we talked about this?’” (DST26).
Boundary spanning or bridging organizations were also referenced by interviewees as key
facilitators. Examples of boundary spanning organizations include the Southwest Ecological
Restoration Institutes in Arizona, Colorado, and New Mexico, The Nature Conservancy, and the
Forest Service Enterprise Team. Boundary spanning organizations and boundary spanners, the
individuals who help to communicate and facilitate across science, management, and policy
spheres, provide added capacity in terms of science and technology expertise, as well as
organization and facilitation. One manager noted, “Having CFRI [Colorado Forest Restoration
Institute] as someone who can do the legwork of a lot of the spatial analysis and with all of the
capacity issues that we have, is really important … I really feel lucky that we have CFRI
available to use, and I know a lot, some other forests don’t have that, but having that is really a
critical piece, I think, to our capacity” (DST23).
Boundary organizations can also help provide access to scientists and model developers who can
answer questions and provide guidance as DSTs are integrated into workflows. One manager
noted, “Being able to talk specifically with the person that created the model and had a
knowledge of how it was supposed to work, and then also the person that was doing the
modeling, to be able to provide guidance in terms of limitations of the data to some degree was
also useful” (DST22). Or as one scientist/developer described it, “It just takes some facilitator
out there working with them to help them explain and understand it,” (DST27). Boundary
organizations can also help to make DSTs more accessible to their intended users. One boundary
spanner explained, “I’m making a lot of these risk assessment tools more accessible so people
can use at, say, low cost … because that seems to be a major barrier, is you can go out and find
a lot of this information, but most people don’t have the skills to link it together. So most of the
work that I’ve done in this space is just making things more accessible” (DST11). Boundary
27
organizations can also help build credibility by providing access to local, place-based science. As
one boundary spanner explained, “We can point not to nearby mountain ranges but literally out
the window and say we collected from right there…do that in a way that’s completely grounded
in experience and science … It makes it really hard for any of those anti-management groups or
especially outsiders who are anti-management to come in and try to derail things” (DST15).
Another key element of the process of co-developing and applying DSTs is the creation of new
relationships, continuation of existing relationships, and facilitation of trust and respect. One
boundary spanner explained the importance of getting to know the people on a team to gain trust
and credibility. “I think what’s critical is getting to know the people that you’re working with on
a team and really gaining their trust and demonstrating that you have competency and
something to offer them as far as the products that I would bring to the table … A successful tech
transfer, it has to be through somebody that’s trusted by the decision-maker and the people that
are in the field doing the work” (DST8). One scientist/developer noted that it takes time and
iteration to build trust and relationships and have candid conversations about what may or may
not work. “So once there’s shared mutual respect across the room, then we could just have
conversations and work through all of these over time and get there” (DST27).
Interviewees also noted that consistency in participants and limited turnover help facilitate
building relationships, trust, and respect. As one scientist/developer explained, “There’s one or
two people on that forest that I would refer to as those champions. They came to us; they wanted
the risk assessment; they saw the application for it; and they haven’t retired or took a new job or
anything; they’re still at the forest” (DST9). Other interviewees highlighted the importance of
local champions who can articulate the need for a DST, drive progress, and create buy-in and
support that leads to use of new tools. One boundary spanner noted, “Another important factor is
the cheerleader. I think in order for these efforts to be successful on each landscape, there has to
be one or two people who really act as a champion, who want to use it, who know how to use it,
who understand how it was constructed … That’s the magic sauce. You need the people to really
want to use the tool. And I think they’re more likely to use the tool if they were invited to help in
its development and implementation from the start” (DST14).
DSTs can enhance the legitimacy of decisions
One primary facilitator to using DSTs is an inherent part of their name and existence, that is,
they support a desire for better information and facilitate the use of that information in decision
making. This is especially important when it comes to decision making about where to provide
funding for hazardous fuels reduction and forest restoration work. Managers and decision makers
are motivated to have a better narrative about why, where, and when they plan to spend money.
DSTs can provide information that justifies funding. As one scientist/developer noted, “They
[Forest Service leadership] want better analytics to understand what outcomes can be produced
from investments” (DST4).
28
Another benefit is that DSTs can serve as a centralized repository for information. As one
manager explained, “There's some utility in having a central spot for all this information that's
preloaded and pre-thought out, with a caveat that it's highly subject to change, right?” (DST12).
In a similar vein, DSTs can help to quantify existing knowledge and provide access to
information in a more effective format. As one manager noted, “One of the big benefits was
coming up with the data set across the landscape that allowed us to quantify all this information
to a degree that no one's had access to before” (DST22). Developing DSTs with local input can
help to capture and institutionalize local knowledge and experience, which can otherwise be
difficult to provide to new staff when there is turnover or retirement. As one manager explained,
“Some of the resource folks, maybe they're not from the area or haven't been in their job for a
decade. The line officers as you know, tend to come and go fairly frequently. We don't have any
line officers that have been here for 10 years, for example. And, so that institutional knowledge, I
feel it lies in a few hands on the forest. Some folks have that. But if those folks retire … that
information is lost. But if you capture it in a process like PODs or something, then it's more
institutional. It's more corporate knowledge, it will be retained” (DST20).
DSTs are also seen as facilitators when there are apparent outcomes on the landscape with
respect to planning. For example, when DSTs are used in a planning process, they can
demonstrate how previous work has helped lower risk on the landscape. For example, one
boundary spanner noted, “That was an interesting tool to be able to have that … area in the tool
because it really allows people to look, take a step back, and say, wow, this work does show up,
at least on the map, as being significantly reduced. And there’s this nice hole in the middle of the
map that’s relatively low risk compared to everything else” (DST15).
DSTs can also facilitate increased decision quality or confidence in decision making. For
example, during a fire, DST outputs can facilitate trade-off analyses, create understanding of
what really makes a difference in a decision-making process, and generally support risk-
informed decision making. When fire outcomes are positive and DSTs were used to support
decision making on that fire, there is often a positive feedback loop whereby there is increased
confidence in the DST itself. For example, one boundary spanner noted, “These fires might have
been managed the same even if PODs and the risk assessment were not in place. I think they
provided the justification. I think in doing so they helped provide confidence to the decision
makers, that they were making the right call” (DST14).
Interviewees noted that DSTs can also help to create public confidence in the decision-making
process. One boundary spanner explained, “Doing this painstakingly tedious planning and
priority identification ahead of time gives the agency a lot more flexibility when it comes to
setting up a plan of work for five years that include areas where they’re going to do a NEPA
analysis” (DST19). One manager explained how the science behind a DST provides an added
level of credibility to decisions. “Having more of that scientific or analysis background and
being able to show that to some of the concerned people about your project, it really helps clear
the air and make it clear to them that you’re not just willy nilly picking things to meet a timber
29
volume or because it’s a fire program that really likes to put fire on the ground. It really helps
put that solid reasoning to it” (DST25).
Lastly, interviewees noted that DSTs can help to encourage creative or strategic thinking. One
manager put it this way, “It feels like a new generation, a new culture in resource management
that’s recognizing that we’ve got some really cool technology and a lot of smart people that can
help us make better decisions” (DST18). And this is especially important today as fires continue
to increase in frequency, severity, and complexity. As one scientist/ developer noted, “The vision
that I’ve always had for it is: we are bringing a fire manager perspective forward and into the
planning environment. Because it’s such a critical issue today” (DST27).
Recommendations
Interviewees were asked to provide recommendations for improving the development,
integration, and use of DSTs in wildfire risk assessment and treatment prioritization planning.
Their recommendations are organized below according to their major topical areas.
Analytical capacity and funding to develop, integrate, and use DSTs needs to be increased
One of the most common recommendations across interviewees from science, management, and
boundary spanning organizations is a need for more people with analytical expertise to assist in
the development, integration, and use of DSTs. This is consistent with other research findings, as
well (Greiner et al. 2020, Rapp et al. 2020, Schultz et al. 2020, Thompson et al. 2019). One
manager explained, “Nationwide, having more analysts is important because those are the
people that can, at least, if they can’t run them all, they can understand what went into it and
explain it and interpret it. Right now, there’s not very many folks that are able to interpret a lot
of these tools out there” (DST24). One boundary spanner noted, “We really need to have more
analysts that can create some of the analytical tools that we use to develop these PODs and the
Quantitative Risk Assessment … And we need somebody that can help update these and keep
them relevant and maintain them year to year” (DST10). Another manager noted the importance
of having access to an analyst when making decisions. “I want that analyst in the room listening
to me, having these conversations with the stakeholders, and be thinking about what could help
them, not tell me the answer, but help me have this conversation” (DST7).
The ways in which additional analytical capacity could be realized varied among interviewees,
but in general, there was consensus that there should be, at a minimum, personnel within national
forests or regional offices tasked with learning DSTs, as well as translating and transferring them
to intended end users. As one boundary spanner stated, “We need more analysts. It’s that simple
… I wish we could have that capacity at every one of those forests. And then we would have
national vision and direction to help them” (DST1). Or as one manager put it, “If we want
people to be using these tools, all the forests need to invest in people that can use the tools”
(DST25). One boundary spanner also noted the importance of local analytical capacity in terms
of developing trust. “We need to have more technology transfer people that are positioned
around the region … these people, ideally, would have relationships with the fire community that
30
they’re sitting with, so it’s not just some random person that comes in” (DST8). Another
manager noted that this could also include building more capacity locally. “It would be more
people, or prioritizing people that we have, at the local level, building skills to do this kind of
work” (DST22).
For some managers, the lack of appropriate analytical capacity is due to inconsistency in
positions across forests, as well as a lack of funding for positions. As one manager noted, this
inconsistency warrants investment in positions dedicated to analytical work across national
forests. “It depends on the forest, some forests have people that are good at this stuff, some
forests don’t … If we want people to be using these tools, all the forests need to invest in people
that can use the tools” (DST25). Similarly, another manager noted, “There’s not positions whose
only job, It’d probably be like two positions or so, whose only job is to try to operationalize all
these analysis tools. I mean, we’ve got them at the research station, ERI, CFRI, but it’s getting
the field to trust that side of things that is going to continue to be the difficult part” (DST24).
Interviewees also talked about the need for additional assistance from boundary spanners and
boundary spanning organizations to explain the need for DSTs, interpret and communicate
outputs, and facilitate conversations. As one scientist/developer noted, “If you get a bunch of
analysts, they need to be able to communicate in some fashion, and those skills don’t always
come together. And, so there’s a broader need for those folks that are good communicators and
good workers with public or collaborative groups … We need a capacity on both sides. Those
folks that can communicate and work with groups, as well as those folks that can create the
analytics. And both lack … There, of course, are good people out there that can do it, they’re just
tasked with other things” (DST27). One manager described this as a bridging function, much
like the role that boundary organizations already play. “We need people to learn the computer
side, the data side of things and we also need them to have operational experience so that they
can translate it to people that only have operational experience … Something like a bridge
between that data and that operational, those two sides” (DST24).
One of the major capacity barriers noted by interviewees was that there is simply a lack of time
to devote to learning and integrating DSTs into a regular program of work. One recommendation
to address this barrier is simply acknowledging the time needed to integrate a new DST. As one
manager explained, “My recommendation would be to understand that it could take a full year,
maybe even a little bit more, to complete the process so you can actually start incorporating into
your planning efforts, and the acknowledgement that you’ve got to build in that capacity for
when you hit the field season, and because the people who are the implementers are really
important to making sure that this process is completed” (DS23). Acknowledging the time
needed for integrating a new DST is critical, because as one scientist/developer noted, it also
takes time to develop trust and buy-in with a new concept. “If we get it in a vacuum, simple, fast,
done. But if you really want to bring people along and have a shared stewardship or shared
ownership of the outcome and the direction you’re going, that’s what takes time. And I think
that’s critically important and should be pursued, but that’s where time goes and where
investments are needed” (DST27).
31
Respondents also noted that it is critical to ensure funding to support analytical work and the
integration of new DSTs. “I think a lot of the intent is being met supporting the ground with
these analysis and scientific support for field operations. But money could be leveraged much
better if there was kind of a cohesive agreement that this was going to be a priority” (DST18).
As one boundary spanner noted, a lack of capacity is due largely to a lack of funding. “The
personnel capacity, I think, is really limiting. Just more funding” (DST11).
Communication about DSTs must be done in a common language and sustained through time to
build relationships and trust
According to interviewees, different elements of communication can be barriers or facilitators to
the development and use of DSTs, but interviewees had recommendations as to what elements of
communication can be improved to better facilitate the use of DSTs. Respondents across groups
emphasized that a common language is needed to discuss risk and DSTs. The importance of a
common language has also been emphasized in GTR349 (Thompson et al. 2016). Developing a
common language also relates to appropriate education and training on structured, risk-informed
decision making, which is addressed later. As one manager explained, “Now also the other piece
that we are leaning into is with agency administrators, decision makers, line officers, to require
some training and education so we get to some common language and understanding about their
role in decision making and what quality decision making is, what risk-informed decision making
is” (DST7). One boundary spanner explained that establishing a common framework and
language is critical in engaging with partners across a landscape. “Have, at least, an established
framework, so that everybody’s cross boundary communication can be the same … Making sure
that everybody’s using the same terminology and similar methodology, the same framework at
least, so that you can talk to your neighbors about what they’re doing” (DST10).
In addition to establishing a common language, interviewees also noted the importance of setting
expectations up front for collaborative, pre-fire planning efforts, especially using DSTs. One
boundary spanner explained, “From the outset, set the expectations, like, ok, we’re going to this
big landscape-scale planning, and then we’re going to use it to find funding; we’re going to use
it to build community support; and we’re going to use this to get some stuff done. And just keep
coming back to the plan and keep coming back to the purpose” (DST15). Respondents also
emphasized the need to ensure that key people who will be using a DST are involved in its
development from the beginning. “Those key groups of people that are developing that and/or
the people that it’s being developed for, I think, they have a good understanding of those
assumptions, limitations. And to me, the best way to communicate is right there on the front end”
(DST20). Another manager echoed this sentiment. “I think it’s coming up with a more robust
conversation before you even enter into running a model. Really talking about what do we need
the model to do, what information do we need to know, what do we need the model to output, and
whether that model is the right one to use for any particular situation. Having those
conversations together with whoever designed the model and the people deciding to use the
model, and then the end user who's actually going to use the output. Because it may be a line
officer that decides to use the model, but it’s the people on the ground at the district level or at
the forest level that are actually going to use those outputs” (DST20).
32
In addition to engaging end users from the outset of DST development and integration,
respondents also highlighted the importance of sustaining that engagement throughout the
process. End user engagement can help to improve scientist/developer understanding of what
elements of a DST are most useful or viable in each context. As one scientist/developer
explained, this is often an iterative process with give and take from both ends. “It was an
interesting process of trying to be the facilitator and the guider of the process with some
background in how what they were doing would affect the outcome. And, so trying to nudge it in
the right direction when I felt like it was appropriate, but then also making sure that the local
specialists participating had ownership in the process. And, so there were times where maybe the
way that they characterized the fire effects on a particular HVRA, I didn’t a hundred percent
agree with, and I would push back a little bit and say, ‘Well, don’t you maybe want to consider
this or that?’ And they would push back and say, ‘Nope. On our forest, that’s how we feel about
it. This is how we want to characterize the fire effects.’ … That was definitely something that was
a learning experience for me to try to figure out how to walk that line, because in the end,
they’ve got to trust and then have ownership and belief in what they’ve produced” (DST16).
Ensuring that there is a common language, as well as communication and engagement,
throughout an effort to develop and integrate a DST helps to build relationships, trust, and buy-
in. Effective communication can help forge relationships between scientists and managers,
which, as one manager noted, can also ensure progress. “Where the rubber meets the road, is
where you have and you forge these individual relationships, either which way the road’s going.
Whether it’s the scientist approaching the manager or the manager approaching the scientist”
(DST12). And as one boundary spanner noted, trust is a critical element of success. “A
successful tech transfer, it has to be through somebody that’s trusted by the decision-maker and
the people that are in the field doing the work” (DST8).
Another communication challenge identified by respondents was the issue of information
overload with new DSTs. As one manager explained, it is important not to overwhelm managers,
“I think PODs is a good thing. If we were to develop something outside of there, I think we
would just want to be careful that we’re trying to integrate it all and not overwhelm fire
managers because it seems like that’s easy to do” (DST20). But as one scientist put it, there also
needs to be a balance between understanding the role that a DST can play in informing decisions
and appreciating how DST outputs can lead to changes in conceptual understanding over time.
“Decisions come in many, many small baby steps. No one makes radical decisions because you
can't in our climate or business climate that is highly collaborative, highly interactive” (DST4).
Interviewees also talked about the importance of in-person engagement to facilitate
communication, though as one manager noted, that was complicated by COVID-19 in 2020. “We
can’t sit down with his folks. It has to be done with the electronic versions of the map, where
normally you’d have all your heads together, and go, ‘What about this one? Well, that one
doesn’t work, but we razed the crap out of this one, so this one will work now.’ You know, that
kind of thing” (DST21). Interviewees also talked about sustaining communication to ensure that
continued attention is paid to new approaches. As one manager explained, sustaining
33
communication is a way to maintain momentum for change after initial planning efforts are
completed. “Being available and continuing to engage with the folks that are going to be the
ones making decisions on these fires, that’s going to be the most important thing” (DST24).
Lastly, interviewees highlighted the importance of sharing success stories and case studies that
provide examples of successful DST integration in new contexts. For example, one boundary
spanner said, “I wish there were better examples of case studies that showed how other places
have gone through the process that I just described us going through, and not to say that it was
better or more appropriate, but to give the folks that I was coaxing to the table more confidence”
(DST19). Success stories can also move the needle in cases where there is skepticism of fuels
treatments. As another boundary spanner explained, “A risk assessment in itself is not going to
convince someone who's dogmatically opposed to forest thinning or chainsaws or prescribed fire
or anything like that. That’s not going to happen, but if you can put a risk assessment in front of
a skeptical city counselor and walk them through it, having that sort of a success case … helped
move some of those conversations forward” (DST15).
Implementation planning can ensure that DST outputs are used
Once a set of DST outputs are developed, interviewees emphasized the need for a plan to ensure
that those outputs are integrated into decision making and that data and outputs are updated as
needed. One manager emphasized the need for an implementation plan and schedule, “The
biggest recommendation I have is once you start and you pick the tool you want to use, whether
it’s PODs or whatever it is, to get your stuff, get your folks on a schedule and stick to it …
because we’ve moved the process along and then stalled out, and people got disengaged. And, so
my biggest recommendation would be just once you started, bringing it all the way through to
the end, keep yourself on a schedule. Don’t drop it” (DST23). Scientists/developers also noted
that identifying ways to use and implement the results of a DST effort is critical. As one
scientist/ developer said, “Listening to the forest or region, those stakeholders ... ‘Okay, so what
does your program of work look like? How do you think this information can be useful? How are
you identifying and prioritizing your fuel treatments now?’ Working with them together in, sort
of, a workshop setting to identify ways in which the QWRA results could be integrated into their
program of work” (DST9).
Respondents also noted that it is important to update data and DST outputs on some sort of pre-
determined schedule in order to keep the information up-to-date and relevant. One manager
explained, “It shouldn’t be a static thing. Conditions on the ground change. There could be
something out there that wasn’t captured in your risk assessment when you made it. They’re all
not static tools. They need to be updated to have relevance. If you had a big fire season and fire
burns across a number of your PODs, you need to look at updating some things because maybe
those outlines weren't in a good spot to start with, but they probably are now” (DST24). Another
manager suggested that starting with annual updates to a DST like PODs would be appropriate
and could be scaled back as needed. “I think a good starting point would be annually, and we
may find that maybe it’s biannually or every three years or something. But you know, last year,
34
as an example, we had 55,000 acres of what we used to call wildland fire use. So, that 55,000
acres represents a big change on many of our landscapes” (DST20).
Determine the question that a DST is meant to inform and develop and/or apply it accordingly
Across respondent groups from science, management, and boundary spanning organizations,
interviewees emphasized the importance of first determining what is the question that a DST is
meant to inform and then developing its technical capabilities so that it is tailored to produce
outputs to inform the key question. As one boundary spanner noted, “Early on in a process,
define what a management question actually is that needs decision support … For people who
are developing decision support tools, or applying decision support tools, that’s a necessary skill
beyond just the technical capacity is really helping coach people through a process of helping
them define what are the questions they’re trying to answer, what information do they need and
then how are they going to use that information.” (DST11).
Other boundary spanners articulated a similar process of reverse engineering the development of
a DST by first determining what questions need decision support. As another boundary spanner
explained, “In my work, I start at the other end. I figure, what’s the question that we need to
answer, and what’s the best way to answer that question? I think a lot of these tools are working
backwards. They develop the tool and then ask how can we use this? I think the tools that I work
on don’t take that approach, or at least try not to” (DST14). Similarly, another boundary
spanner explained the importance of understanding a setting before determining what DST to
apply. “We go in and listen and learn and pay attention. And whether it’s like, do a formal needs
assessment process so that we can identify which tools to bring to bear, or just through
conversations over coffee and in meeting rooms and things like that, to understand the
complexity, the social-ecological complexity of a place, and then develop the tools that can help
solve that” (DST15).
However, because there are numerous DSTs available that could be brought to bear on a specific
question, it can be challenging to select from the available tools, especially if developing a more
tailored approach is not feasible. One scientist/developer emphasized that it is, therefore,
incumbent on the scientists/developers to explain what kinds of questions a given DST can
answer. “I certainly have tried, whenever I present on the analyses that I'm working on, to
explain the kinds of things that I just talked about that, hey, look, what I’m giving you provides
one type of information. But doesn’t provide all the answers, and this is what you can do with the
information I’m giving you and these are the types of questions that this information can answer.
But if you want answers to these questions, which are different, you maybe need to turn to
different tools” (DST16). One manager also explained the importance of understanding the
question before selecting from the whole suite of DSTs that are available. “We have this whole
suite of tools available, and a lot of the times, you don’t need a lot of this information. That’s
where you become a choke point in the pipeline … If you can get that question phrased and
written down and you all agree on it, it’s actually a lot easier to pick through all of these tools to
figure out what the right one might be for the situation” (DST24).
35
Another boundary spanner highlighted that it is also important not to create DSTs that try to
accomplish every objective or answer every question. DSTs that try to do everything are often
problematic. “There’s a push to have these tools do everything. And they become multi-tools, the
Leatherman, it can technically do everything, but it’s not really good at anything, right? Once
you over-generalize, it’s hard to be good at all those different things. So, I would prefer multiple
models and multiple decision support tools, which each address one or two problems, instead of
one gigantic tool that is supposedly designed to address all things” (DST14).
In short, there are tradeoffs that need to be considered between applying established, formal
DSTs and developing smaller-scale, context-specific DSTs that are less formalized but use a
common language and fire science framework. As one boundary spanner explained, “I’m not
sure if one model is better or not … But it’s certainly much lower costs to say take this scripting
approach to decision support where you’re really just trying to automate a process for someone
without necessarily making it highly functional complicated software for someone” (DST11).
One scientist/developer explained that in order for a DST to be used to support decisions, a few
key factors need to coalesce, including 1) a standard, guideline, or requirement to conduct a risk
assessment or prioritization process; 2) a biological or physical model that serves as a underlying
framework; 3) software or some other platform on which to run the model; and 4) people who
are trained to run and interpret the model. “All those things have to line up before you can build
a decision support tool. It’s going to get used” (DST4).
DSTs should be validated and tested with end users
Another recommendation from interviewees was that DSTs need to be ground-truthed in order to
validate and monitor the outputs that they provide. The importance of model validation for
decision support has also been identified in other research (Rapp et al. 2020). One boundary
spanner noted, “After you’ve drawn stuff on a map, you need to get out in the woods” (DST10).
The importance of ground-truthing was also identified by managers. As one manager explained,
“In all of our fuels projects, we always went out and ground-truthed everything, too, and I think
any model that you use, you need to ground-truth anyways, because you never know how good
the data is, what’s underlying it” (DST25).
Respondents also emphasized that validating DSTs and their outputs is an essential way to
improve them over time. One manager said, “Verify it on the ground with the folks that are
closest to it, because all the models in the world, applying all of the pieces that we might know in
a geospatial lab sense out on the ground always is improved by ground-truthing” (DST7). One
boundary spanner also emphasized DST verification to improve outputs. “Having some feedback
loop so that those people who end up using information can then tell you how to improve it and
make it better for them. It’s really missing in a lot of cases” (DST11). However, as another
boundary spanner noted, in order to effectively validate DSTs, there also needs to be a thorough
system of documentation of their use and outcomes. “What we need to do is we need to validate
them better than they have been. And, so in order to validate them, we need to use them
extensively and then document not only what they said might happen, but what actually
36
happened. And we need to do a good job of doing that so that we can make some assessments on
do they work. And if they don’t, why? Can we fix that? If so, let’s fix it and try it again … To use
them more effectively, we need to have a rigorous program of documentation” (DST2).
In addition to DST validation, respondents also noted that the caveats of different DSTs need to
be clearly explained to end users as part of testing and integration. One scientist/developer
emphasized, “I think we have a real obligation, those of us that work in this, to try to explain this
kind of stuff as clearly as possible” (DST16). Another manager noted that it is important to also
communicate how exactly DST outputs are intended to be used. “I think the best thing we could
do is be really clear on the front end about what this is and what it is not … I have to remind
folks a lot that this isn’t a decision-making tool. This doesn’t replace our executives, our line
officers” (DST20).
Interviewees also recommended that DSTs be tested with end users in order to ensure that they
are as user-friendly as possible. One manager said, “The thing that I would say is just make sure
that things are tested with folks that are out there doing it … make sure that it’s operationally
sound. If it doesn’t make sense on the ground, then people are going to ignore it or be resistant
to it or whatever. Honestly, we got a bunch of iPads for folks, which is all fine and dandy, but if
they’re glitchy or if the user interface isn’t good, people are … going to use their steno pad and
their topo map” (DST17).
Another manager emphasized that it is important that the people who will be using DSTs are also
the people that help to test them and provide feedback. “The people that are testing it are people
that are pretty good with these tools. They really need to get some people that are the ones using
them on the ground that would use them, testing these and making them user-friendly” (DST25).
Furthermore, many end users want to be able to run DSTs themselves so that they have some
control over the process and feel ownership in the information being provided. As one boundary
spanner explained, “A lot of people want to be able to actually run this tool. They don’t want to
just know that it’s on some scientist's computer, and it's coded in this language that they can't
read, like R or Python. They actually want to be able to click on the buttons, change the
numbers, rerun the analysis, and learn and understand how the tool works” (DST11).
These recommendations emphasize the importance of continuing to improve tools based on user
feedback. For example, one scientist/developer explained how a user advisory group and a user
forum were developed to help improve the use of a new DST. “The user advisory group’s a new
concept, so we’re still figuring out the best way to utilize their time, their skills, and their input
with it, but I think it’s much better than us just sitting in a cave and building a tool that we think
is most important” (DST5). The same scientist/developer also explained how a user forum can
help connect end users with DST developers to solicit feedback and improve the DST, “… a user
forum where anyone can post questions or suggestions, and a lot of times, we use what they have
posted to enhance, update, or change something” (DST5).
37
DST end users need both education and training in the concepts and tools
Another key component of successful DST development, integration, and use is both education
and training. Education refers to developing a comprehensive understanding of risk and risk-
informed decision making. Training refers to the necessary steps to develop and apply DST
outputs in programs of work and decision-making processes. As one boundary spanner said,
“Our line officers and fire managers need to be educated in risk, not trained but educated.
Training tells you how to go through step A to step B, and C or D, or whatever. Education gives
you the ability to work on a problem and apply concepts to the problem. Right now, most of our
folks are not well educated in risk or educated at all in risk … The perfect example is it’s very
difficult to get the agency administrator to look at an individual fire and the risk associated with
it, as well as the risks associated down the road that will occur based on the actions they take on
that fire” (DST2). Another scientist/developer put it this way, “I think we need education at
multiple levels and training at multiple levels. But I think that education in decision making is a
critical component” (DST3).
Education is also a way to change existing paradigms and cultures that exist in fire management
that were identified as barriers by some respondents. One manager explained, “I don’t know
what that education campaign would look like, but I think just as you go to more places that are
doing different things with fire, eventually you change the culture of the people that are
eventually going to be the fire staff and the directors and all of that sort of thing” (DST24).
Another managed shared a similar sentiment, “The other piece that we are leaning into is with
agency administrators, decision makers, line officers, to require some training and education, so
we get to some common language and understanding about their role in decision making and
what quality decision making is, what risk-informed decision-making is” (DST17).
Changing paradigms and cultures takes time, so some interviewees recommended that continuing
education programs and revised or new qualifications could facilitate both education and
training. One boundary spanner said, “I think in the end it would get everybody up to the same
level, that if we knew every five years there was this mandatory continuing education piece, so to
get everybody up to speed on what’s the latest and greatest science and tools and information
that we’re all sharing during wildfires” (DST8). Another manager agreed that qualifications
could be modified to facilitate an understanding of the underlying concepts that shape DSTs.
“We have those IFPM [Interagency Fire Program Management] qualifications and I think if
they went back and revised them to include more of that fire ecology, fire behavior, fire analysis
type of things, they could start there to where you have to have some certain basic classes.
Whether you are an engine captain, or you are a fuels specialist, I think all of our fire people
need more of that” (DST25).
With respect to training, one approach called “train-the-trainer” was mentioned by several
respondents as a useful strategy to deliver new concepts and DSTs to the numerous and diverse
Forest Service units around the country. Trian-the-trainer is a model whereby a local
representative is training to use a DST and then returns to a local unit to assist in training his or
her colleagues. One scientist/developer explained it this way, “Train-the-trainer is a model that
38
we’re really trying to work on, because … we’re so busy trying to develop new content, we don’t
necessarily have the time to get out to every person who wants training, but we are slowly …
trying to find people that are self-motivated and want to learn it and willing to go teach beyond
just their small groups so we’re slowly building that momentum” (DST5).
Respondents also noted that user workshops are critical in training people to use new DSTs. One
of the DSTs referred to by many respondents was PODs, which are often developed in a
workshop setting. In addition to using workshops to develop DSTs, respondents also noted that
there need to be workshops to train people on how to use the DST outputs. One manager noted,
“We’re working on … more of a user type of workshop, how do you use this data in ArcGIS, how
can you put it in a NEPA analysis, how could you use it in a burn plan, and work with the users
of this more” (DST25). One scientist/developer described the need for workshops that focus on
the application of DSTs. “After the results have been presented, sit down with … the key users of
it, sit down with those folks and provide a little bit of additional interpretation or additional
training on how you interpret QWRA results” (DST9).
Respondents also provided additional recommendations about how to structure an effective DST
training. For example, one scientist/developer noted that many people prefer in-person training.
“We have user surveys ... One of the questions was, how would you most like to get training?
And the most common answer is: in-person training” (DST5). Additionally, one scientist/
developer explained that working through exercises for how to use DST outputs has also been
helpful. “I think exercises where they’re actually working through an exercise that shows what
was predicted, what actually occurred … If you have two or three exercises that show how you
work through that, that seems to, in my experience anyway, resonate with folks better and give
them better appreciation for what it is saying” (DST2). Lastly, one scientist/developer noted that
having a user manual can also provide long-term support for local technical experts.
Policies, guidance, and authorities must match leadership intent for integrating and using DSTs
Interviewees also recommended several ways in which policies, guidance, and authorities could
be changed or improved to facilitate a better understand of leadership intent for using DSTs in
wildfire risk assessment and treatment prioritization. One boundary spanner said, “I think we
need national guidance saying, just like we did with LANDFIRE, all right, we need these risk
assessments, let’s start, let’s lead the efforts, start turning the crank nationally, and we’re going
to create these risk assessments for, at least the western states, every few, four years” (DST1).
Although there are policies that articulate the need for doing risk assessments, as one scientist/
developer noted, they are not explicit. “The policies, I think, are there. They’re just not as
explicit as, you should do this and use these tools, which at any point in time is useful in setting
an agenda and a direction for an agency, organization” (DST27).
However, developing guidance for DSTs is challenging, in part, because the science behind
DSTs is constantly changing. As one scientist/developer noted, “Science is constantly changing.
Take PODs, for example. It’s starting to take off, and there’s a lot of interest and new
development, how this is being developed and applied. Is there official guidance from the regions
39
or the Washington Office that are saying to use PODs? No. They’ve had to read between the
lines about what the Washington Office wants, more strategic planning, and identify places to
use fire and risk management. And, so there’s a tool that’s crafted towards that. Now, some
forests are waiting for somebody to tell them, thou shalt do this. Other forests are willing to just
kind of dive in” (DST14).
One of the ways that more clarity could be gained about the need for doing risk assessments is
through a clear articulation of leadership intent. One manager argued, “There has to be some sort
of directive or emphasis, literally stated from top leadership, that this is something that they
want to see completed and give people a reasonable amount of time, you know, five years”
(DST18). A clear articulation of leadership intent is especially important in an organization like
the Forest Service that has dispersed units from the Washington Office to local district offices, so
there needs to be clear and consistent communication across the different levels and units of the
Forest Service to avoid confusion about what is intended.
However, as another manager argued, it is also important that leadership intent consider
perspectives from the local level. “I think it’s more of having leadership know and communicate
their desires to have their people involved in it, knowing the benefits of having local experts
involved from the very beginning and throughout the process, knowing the value of that on the
back end would facilitate the use of these tools and making the outputs more usable and making
people more likely to use them in the end” (DST22). One scientist/developer put it this way,
“Somebody at the national level should set an agenda that is in-line with what the public wants
and aligns their staff to that need” (DST27). As one boundary spanner explained, this requires
communication from local levels up the chain of leadership to help provide that local perspective
across the units of the Forest Service. “It’s really important that these higher-up decision makers
know that there’s a lot of nuance to planning and prioritization at a local level. And there’s also
this big, I would say, trust and understanding that comes from applying these tools within a
localized context. And, so I think making sure that people know that it’s unlikely that some single
national analysis that’s done is going to be somehow downscaled and well-received and easily
applied at a local scale is an important message to come out with or to communicate” (DST11).
To that end, it is important to strike a balance between articulating leadership intent from the top,
while also making sure to communicate local and public perspectives up the chain so that they
can inform top-down directives.
Respondents also recommended that leadership intent and communication about the need for
using DSTs should include an emphasis on why it is important. There needs to be a compelling
“why” in order to garner support across levels of the agency. One of the ways in which that
compelling “why” can be communicated is by supporting and empowering thought leaders and
change agents within the agency. As one manager explained in reference to attending PODs
workshops, “A lot of the people that went to these workshops, they’re not your 25-year seasoned
fire managers who came. And they grew up in a very different world. What you need is your
younger people to come back and inspire the old people. But the younger people coming back
are not in any position of any authority or influence. And, so, you’ve got a bunch of young
40
people coming back. And I say young, but I mean change agents coming back. And then, they
basically just get shoved down because there isn’t any parallel policy pressure placed on it. And,
so, it loses traction” (DST18). Another scientist/developer noted that there is also a need from
leadership to support a reasonable level of risk-taking. “From an outside perspective, it feels like
they just aren’t willing to step up and accept decisions, the risks that come with that uncertainty
in the future and the outcomes. And then support each other when things go awry” (DST27).
In addition to articulating intent to use DSTs, interviewees also noted that it is important to
develop incentives and accountability for doing in order to facilitate the process. As one
scientist/developer explained, “We don’t have incentives and accountability. And I’m talking
about accountability, not when somebody screws up, they’re not getting punished. I’m talking
about just basic, are we measuring and reporting what’s happening? And the incentives are very
driven at a local level. There’s no clear direction on how we’re incentivizing good fire
management. We don’t even really know how to define good fire management. So how do you
incentivize it when you haven’t defined it? I think that’s a critical component” (DST3).
In order to develop incentives and accountability for using DSTs in decision-making, there also
has to be some degree of evaluation and documentation of decisions made using DSTs. As one
boundary spanner argued, “I think line officers should be evaluated. Part of the performance
rating should be on how they manage risk on the landscape. We can quantify risk on the
landscape, right? And, so we should quantify risk on every unit, on the landscape every year.
And a line officer should be evaluated against that … It’s probably the most important thing that
they are charged with managing” (DST2). As another scientist/developer recommended, “And,
so this process of risk management requires that we get away from outcome-based metrics and
get to a decision to really be able to evaluate the quality of our decisions and that requires us to
plan and document” (DST3).
In short, many respondents argued for a more structured approach to decision-making, which has
also been articulated by other researchers (Calkin et al. 2010, Thompson et al. 2019). As one
manager explained, “So I have a dream. We need an integrated approach and commitment to the
structured, risk-informed decision making. And this is where you can start throwing around all
the rest of the buzzwords, evidence-based, data-driven, all the rest of it … And one of the most
next steps is some assessment of the analytic capability in the agency and how we want to
leverage it … We’ve got to bring these pieces together and at the core of it, as a body of
specialty, around analytic capacity. How do we assess it and then determine what we might
need? And if we use a structured decision-making approach, I actually might come up with a
fairly reasonable set of next steps” (DST7).
41
Conclusion
The purpose of this project was to analyze the use and adoption of wildfire risk assessment and
fuels treatment planning and prioritization decision support tools (DSTs) by federal land
managers, especially in the USDA Forest Service. We sought to demystify the topic of spatial
analysis for pre-fire planning, specifically with respect to assessing wildfire risk and determining
areas for fuels treatment prioritization to facilitate effective development and use of DSTs for
pre-fire planning. Although there were many barriers identified to the effective development,
integration, and use of DSTs in pre-fire planning, interview respondents had numerous
constructive recommendations for improving this process, which we categorized into the
following themes: capacity, communication, implementation, question identification, testing,
education and training, and policy, guidance, and authorities. We hope these recommendations
can help shape the perspectives of science, management, and decision-making audiences for how
to improve the use of DSTs for wildfire risk assessment and treatment prioritization in order to
effectively meet the goals of national policies and frameworks.
42
References
Ager, A.A., R.M. Houtman, R. Seli, M.A. Day, J. Bailey. 2017. Integrating Large Wildfire
Simulation and Forest Growth Modeling for Restoration Planning. In. Keyser, Chad E.;
Keyser, Tara L., eds. 2017. Proceedings of the 2017 Forest Vegetation Simulator (FVS)
e-Conference. e-Gen. Tech. Rep. SRS-224. Asheville, NC: U.S. Department of
Agriculture Forest Service, Southern Research Station. 129-137pp.
Caggiano, M.D. 2019. Collaboratively Engaging Stakeholders to Develop Potential Operational
Delineations. Colorado Forest Restoration Institute, CFRI-1908. 10p.
Caggiano, M.D. 2020. Potential Operational Delineations and Northern New Mexico’s 2019 Fire
Season. Colorado Forest Restoration Institute, CFRI-2002. 8p.
Calkin, D.E.; Ager, A.A.; Gilbertson-Day, J., eds. 2010. Wildfire risk and hazard: procedures for
the first approximation. Gen. Tech. Rep. RMRS-GTR-235. Fort Collins, CO: U.S.
Department of Agriculture, Forest Service, Rocky Mountain Research Station. 62 p.
Colavito, M., T. Wasserman, and D. Vosick. 2018. All-lands planning, coordination, and wildfire
risk reduction project report. Ecological Restoration Institute, Northern Arizona
University, 54p. Available online:
https://cdm17192.contentdm.oclc.org/digital/collection/p17192coll1/id/913/rec/2/
Esch, B. and A.E.M. Waltz. 2019. Assessing metrics of landscape restoration success in
Collaborative Forest Landscape Restoration Projects. ERI White Paper – Issues in Forest
Restoration. Ecological Restoration Institute, Northern Arizona University, 12p.
Available online:
https://cdm17192.contentdm.oclc.org/digital/collection/p17192coll1/id/1015/rec/36
Greiner, M., C. Kooistra. C. Schultz. 2020. Pre-Season Planning for Wildland Fire Response: An
Assessment of the US Forest Service’s Potential Operational Delineations. Public Lands
Policy Group, Colorado State University, 16p.
O’Connor, C.D. and D.E. Calkin. 2019. Engaging the Fire Before it Starts: A Case Study from
the 2017 Pinal Fire (Arizona). Wildfire 28 (1): 14-18.
Rapp, C., E. Rabung, R. Wilson, and E. Toman. 2020. Wildfire Decision Support Tools: An
Exploratory Study of Use in the United States. International Journal of Wildland Fire,
Online early: https://doi.org/10.1071/WF19131.
Schultz, C., C. Kooistra, L. Miller, and M. Ferguson. 2020. Findings from a Third-Party
Assessment of the U.S. Forest Service’s Risk Management Assistance Teams. Public Lands
Policy Group, Colorado State University, 40p.
Scott, J., M. Thompson, and D. Calkin. 2013. A wildfire risk assessment framework for land and
resource management. Gen. Tech. Rep. RMRS-GTR-315. Fort Collins, CO: U.S.
Department of Agriculture, Forest Service. Rocky Mountain Research Station. 83p.
Stratton, R. 2020. The Path to Strategic Wildland Fire Management Planning. Wildfire 29 (1):
24–31.
43
Thompson, M.P., Y. Wei, D.E. Calkin, C.D. O’Connor, C.J. Dunn, N.M. Anderson, and J.S.
Hogland. 2019. Risk Management and Analytics in Wildfire Response. Current Forestry
Reports 5 (4): 226–39. https://doi.org/10.1007/s40725-019-00101-7.
Thompson, M.P., T. Zimmerman, D. Mindar, and M. Taber. 2016. Risk Terminology Primer:
Basic Principles and a Glossary for the Wildland Fire Management Community. General
Technical Report 349. USDA Forest Service, Rocky Mountain Research Station. 19p.
USDA Forest Service. 2020. Forest Service Manual 5100 – Wildland Fire Management, Chapter
5140 – Hazardous Fuels Management and Prescribed Fire. USDA Forest Service National
Headquarters (WO), Washington D.C. 17p.
USDA Forest Service. 2018. Toward Shared Stewardship Across Landscapes: An Outcome-
Based Investment Strategy. USDA Forest Service National Headquarters (WO),
Washington D.C. 28p. Available online: https://www.fs.usda.gov/sites/default/files/toward-
shared-stewardship.pdf
Wildland Fire Leadership Council. April 2014. The National Strategy: The Final Phase in the
Development of National Cohesive Wildland Fire Management Strategy. 101p. Available
online:
https://www.forestsandrangelands.gov/documents/strategy/strategy/CSPhaseIIINationalStra
tegyApr2014.pdf
44
Study Contact: Melanie Colavito ([email protected])
Barriers and Facilitators in the Use of Wildfire Risk Assessment and Treatment Prioritization
Decision Support Technology
Open-Ended Interview Questions*
*Additional, relevant questions may be posed to research participants depending on the nature of their
answers to the questions below in order to get clarity or gather additional information relevant to the
study objectives. The order of the questions may vary for each respondent, and respondents will not
necessarily be asked all of the questions below. Some respondents will be asked questions from multiple
sections depending on the nature of their job role with respect to decision support tools and models.
The questions in the introduction will help determine which questions are asked of a given respondent.
Primary Investigator: Melanie Colavito, Ecological Restoration Institute / [email protected]
Verbal Consent
1. Are you over 18 years of age, do you consent to participate in this research, and do you agree to be
audio recorded?
Introduction
2. Please tell me about your job, including what your current role entails, how long you have been in
your current role, and any other relevant background.
3. How would you characterize your job role (e.g., scientist, land manager, policymaker, etc.)?
4. Please describe your involvement in wildfire risk assessment and treatment planning.
5. Please describe your involvement in the use of decision support tools and models for wildfire risk
assessment and treatment planning.
6. How would you describe “wildfire risk assessment”?
7. How would you describe “treatment planning and prioritization”?
8. How would you describe “decision support tool or model”?
Development of Decision Support Tools
9. Are you involved in the development of decision support tools or models for wildfire risk assessment
or treatment planning or prioritization? If yes, please explain.
Potential Follow-Up Questions
a. Which tools or models have you helped develop?
b. Who else is involved in the development of these tools or models?
c. What are the objectives of these tools or models?
d. At what geographic scale do these tools or models provide information?
e. What are the ideal uses of these tools or models?
f. Who are the ideal end users of these tools or models?
g. What are the needs of the end users/ audience of these tools or models?
h. Do you see a need for end user participation in the development of these tools or models?
Appendix. Open-Ended Interview Questions
45
i. Can you provide any specific examples of the tools or models being used that you think
were especially successful or unsuccessful?
j. Overall, what works well in the development of these tools or models?
k. Overall, what are barriers to the development of these tools or models?
l. Have you received feedback from end users about the tools or models with which you have
worked? If yes, please explain.
i. Overall, what works well in the use of these tools or models?
ii. Overall, what are barriers to the use of these tools or models?
m. Are there any improvements for the use of these tools or models that you would
recommend?
n. Are there any improvements for the development of these tools or models that you would
recommend?
o. Do you see any gaps in capacity that would facilitate the development and use of these tools
or models?
p. Do you see any gaps in development or use of these tools or models that could be filled by
new or revised guidance, policy, or authorities? If yes, please explain.
q. What do you see as the future of decision support tools for wildfire risk assessment and
treatment prioritization? Are new tools needed?
Use of Decision Support Tools
10. What do you use to inform your decision making about wildfire risk or treatment planning and
prioritization?
a. What are the biggest barriers you face in decision making about wildfire risk assessment or
treatment planning and prioritization?
1. What do you need to facilitate your decision-making process?
11. Are you involved in the use of decision support tools or models for wildfire risk assessment or
treatment planning and prioritization? If yes, please explain. If no, why not?
a. Are there tools or models with which you are familiar but have not used?
Potential Follow-Up Questions
a. Which decision support tools or models have you used to enhance your wildfire risk
assessment or treatment planning and prioritization work?
b. What are your objectives in using these tools or models?
c. How have you used these tools or models?
d. Who else was involved in using these tools or models?
e. What is the ideal use of these tools or models?
f. Have you interacted with the developers of these tools or models? If yes, please explain.
ii. What works well in the development of these tools or models?
iii. What are barriers in the development of these tools or models?
g. Do you see a need for end user participation in the development of these tools or models? If
yes, what is needed? If no, why not?
h. Can you provide any specific examples of the tools or models being used that you think
were especially successful or unsuccessful?
i. Overall, what works well in the use of these tools or models?
46
j. Overall, what are barriers to the use of these tools or models?
k. Are there any improvements for the development of these tools or models that you would
recommend?
l. Are there any improvements for the use of these tools or models that you would
recommend?
m. Do you see any gaps in capacity that would facilitate the development and use of these tools
or models?
n. Do you see any gaps in development or use of these tools or models that could be filled by
new or revised guidance, policy, or authorities? If yes, please explain.
o. What do you see as the future of decision support tools for wildfire risk assessment and
treatment prioritization? Are new tools needed?
Wrap Up
12. Do you have any other thoughts about decision-support tools or models for wildfire risk
management and treatment prioritization that you would like to share?
13. Is there anyone else that you would recommend I interview for this study?
14. Can I contact you again in the future regarding this study?
15. What kinds of outputs would be most helpful to you from a study like this?
16. Would you like to see the outputs of this study?
17. Do you have any questions for me?
47