Defra Evaluation of the Catchment Based Approach Pilot Stagesciencesearch.defra.gov.uk/Document.aspx?Document=11297_DFAppe… · Evaluation of the Catchment Based Approach ... (Q1

Defra Evaluation of the Catchment Based Approach Pilot Stage

Appendix C: Final Review Report

June 2013

In Association with

YJRees

Consulting

Client: Defra Title: Evaluation of the Catchment Based Approach – Pilot Stage (Appendix

C: Final Review Report) Project No: CC472 Date of Issue: June 2013 Status: Final Version No: 1 Produced By Authorised for Release By

……………………………….. ……………………………………

David Corbelli Dr Kieran Conlan Principal Environmental Scientist Managing Director CONTACT DETAILS

CASCADE CONSULTING

Enterprise House Manchester Science Park Lloyd St North Manchester M15 6SE Tel: 0161 227 9777 Fax: 0161 227 1777

Defra Evaluation of the Catchment Based Approach – Pilot Stage Final Review Final

Cascade Consulting

CONTENTS

1 Introduction ....................................................................................................... 1

2 Milestones ........................................................................................................ 2

3 Support ........................................................................................................... 10

4 Engagement and Collaboration ...................................................................... 20

5 Learning ......................................................................................................... 29

6 Benefits .......................................................................................................... 36

7 Costs and Value for Money ............................................................................ 49

Appendix A: Fourth Quarterly Review and Final Review Form ................................ 53


Cascade Consulting 1

1 Introduction

1.1 The Final Review Defra supported a series of catchment-level partnerships to pilot a new approach to

improving the water environment through catchment-level engagement and planning.

As part of this pilot phase Defra evaluated 25 catchments to learn as much as

possible from those participating. The information will be shared with others who may

be involved in a wider adoption from 2013. As part of the evaluation, pilot hosts have

participated in a final review to reflect on their experience of the catchment pilot

process as a whole. This builds on information gathered through the quarterly review

process, which aimed to review activities, highlight potential challenges and identify

successful practices on a rolling basis over the course of the pilot phase.

This report is intended to summarise the high level messages that have been fed-

back by the pilot hosts regarding their experience of the pilot process as a whole up

to December 2012. Other outputs from the evaluation include quarterly reports from

four quarters, reports from two participant surveys at the beginning and end of the

12-month period, and in-depth case-studies from six selected pilots.

1.2 Overview of Responses This Final Review was launched on 18/12/2012 in combination with the Fourth

Quarterly Review, and closed on 22/01/2013. The combined survey was set up using

an online survey tool and was sent to all pilot hosts by email. A copy of the survey

form is provided at Appendix A.

Complete responses were received from 24 catchment pilots. All jointly hosted

catchment pilots managed to submit their responses in a single form for that pilot.

This report is based on the responses from pilot hosts in the Final Review

covering the period from pilot start to December 2012.

1.3 Report Structure The main body of this report is divided into sections, as per the Quarterly Reviews,

covering: Milestones (Q74 - Q83); Support (Q84 - Q92); Engagement (Q93 - Q110);

Learning (Q111 – Q114); Benefits (Q115 – Q139) and Costs and Value for Money

(Q140 – Q146). Note that the first part of the questionnaire (Q1 – Q73) comprised

the questions for the Fourth Quarterly Review.



2 Milestones

2.1 Outputs from the pilot Initially pilots were directed to progress their work according to a set timetable of

common milestones (see Table 2.1 below). Defra and the EA later clarified that

these “were not set as a measure of success or failure but rather as a guide to what

might be realistically achieved in the time available. The pace of progress is

expected to vary significantly between pilots according to the complexity and number

of problems that they seek to address, and by the extent and value of any

collaborative working that was already taking place prior to the pilot.”

Table 2.1. Initial milestones set out in Defra‟s Expressions of Interest and for EA pilots in

their Principles and Evaluation Pack

2. Milestones/Products

Ref Area EA pilots Non-EA pilots

P2 Baseline stakeholder and activity mapping

Identify current activity and issues, and potential partners and networks. Set up stakeholder group.

Sep 2011 Mar 2012

P3 Stakeholder feedback for each pilot

Ongoing evaluation of stakeholder feedback.

Quarterly reports

P4 Catchment appraisals and agreed catchment priorities for each pilot catchment

A brief description and shared understanding of the problems in each catchment, to inform the catchment „plan‟, and a shared vision.

Dec 2011 Jun 2012

P5 Catchment ‘plan’ for each pilot catchment

To show key problems, shared vision, action required, and commitment from partners. Description of key services provided by the catchment and relative value to users. Register of the outline costs and relative effectiveness of actions identified. Format to be defined locally.

Mar 2012 (draft), Dec 2012

Dec 2012

P6 National evaluation report

Including case studies, recommendations, lessons learned and resource assessment for future approach.

Quarterly reports

Dec 2012 (final)

To determine the pace which pilots have been able to progress towards these

milestones, pilot hosts were asked about the outputs they had produced during the

pilot period and how satisfied they were that these provided a good representation of

the catchment. P3, Stakeholder feedback is covered in Section 4 on engagement

and contribution to P6 has not been evaluated, so in this section we look at P2, P4

and P5.

P2 Baseline stakeholder and activity mapping: 20 out of 24 pilots produced a



baseline and stakeholder activity map usually as part of the catchment plan. The four

pilots not producing a baseline and SH activity map, commented that the required

outcomes had been achieved through a workshop or 1:1 contact/word of mouth. Only

1 pilot was fully satisfied (that with the time, resources and support provided) the

baseline and stakeholder activity mapping provides a good representation of the

catchment, half of the pilots (10 of 20) were mostly satisfied and the others (9 of 20)

quite satisfied (no-one was not satisfied).

Figure 2.1: Question 74 (24 responses) Question 75 (20 responses)

Comparative analysis shows:

Catchment size – All small catchments produced baseline activity and

stakeholder maps, 1 medium and 3 large pilots did not. Large catchments did it

as part of catchment plan or not at all.

Land-use – All urban catchments produced baseline activity and stakeholder

maps, 2 rural and 2 mixed did not.

History of engagement – All those with low levels of previous engagement

history produced baseline activity and stakeholder maps, 3 with medium level

history and 1 with high level history did not.

Host – Host type did not appear to be a factor in whether baseline activity and

stakeholder maps were produced. Of those pilots not producing baseline

stakeholder activity maps, 2 were EA hosted and 2 hosted by others.

Local level up vs. Catchment level down – All working from local level up



produced baseline activity and stakeholder maps, i.e. the 4 that didn‟t were

working from catchment level down.

Run time – This appeared not to be a factor. Of those not producing baseline

activity and stakeholder maps, 2 had been running for less than 12 months, one

for 15-17 months and one more than 18 months.

In terms of level of satisfaction, the catchment fully satisfied with the baseline and

stakeholder activity map was a large rural catchment, reporting a high level of

previous engagement, with a non-EA host that had been running 12-14 months.

Overall, the level of satisfaction does not appear to be determined by catchment size,

history of engagement, type of host, or whether the catchments used a top down or

bottom up approach. There is a slightly higher level of satisfaction with this output in

rural land-use catchments. Interestingly, run time does not appear to be a factor in

the level of satisfaction hosts report for this output, with the majority of pilots running

<12 months being „mostly satisfied‟ and some pilots running for more than 18 months

being only „quite satisfied‟.

P4 Catchment appraisal and agreed catchment priorities: 21 out of 23 pilots

produced a catchment appraisal with agreed catchment priorities, again, usually as

part of the catchment plan. The 2 pilots not producing a catchment appraisal

commented that they used ECM (export coefficient model) and Source

Apportionment Models to provide this information. The level of satisfaction with this

catchment appraisals was higher than for the mapping outputs, with four pilots fully

satisfied (that with the time, resources and support provided) that their catchment

appraisal provides a good representation of the catchment and the majority (16 of

19) were at least mostly satisfied (again, no-one was not satisfied).

Comparative analysis shows that the two pilots not producing a catchment appraisal

were both large catchments with mixed land-use, and a history of high level

engagement previously. Both pilots were starting at the catchment level and working

down to the local level, were hosted by non-EA and had been running for < 12

months.

In terms of satisfaction with the catchment appraisals, pilot hosts reporting higher

levels of satisfaction were generally in smaller catchments, rural catchments,

catchments with low previous engagement, catchments working from catchment

level down. For pilots in this types of catchments hosts were all at least mostly

satisfied that the outputs represented the catchment. Host type did not appear to be

a factor. Interestingly, those running less than 12 months tended to be more satisfied

with the outputs than those running >18 months. Even some starting early 2013 were

fully satisfied that the output represented the catchment and some of the longest



running pilots indicating lower levels of satisfaction.



Figure 2.2: Question 77 (23 responses) and Question 78 (21 responses)

P5 Catchment plan: All pilots who reported in the final review have, or will soon

have, produced a catchment plan. The plans are at various stages of development

as shown in Table 2.1.

Table 2.1: Types of catchment produced by end January 2013

Type of plan Number

Final plan 8

Draft final plan* 3

Draft plan 10

Outline plan† 4

Total 25

*Plans were considered “draft final” when content and structure were finalised but the

document was awaiting formatting prior to publication. Final plans were considered ready for

publication or published.

†Outline plans were plans comprising of a report structure with key headings but most text

still to be written.

Overall, pilots produced 11 plans whose content had been finalised (includes draft

final plans). Comparative analysis shows:

The proportion of rural pilots submitting final or draft final plans was higher (6

out of 8) than from pilot catchments with mixed or urban land use.

The four pilots that submitted a basic “outline” catchment plan were all non-

EA pilots with mixed land use who had requested 6-15 days of facilitation.



Half of the “large” pilot catchments (5 out of 10) submitted final or draft final

plans, compared to less than a quarter of “small” pilot catchments (2 out of 9).

This may reflect this importance of strategic planning work in larger

catchments compared to greater delivery focus in smaller catchments (see

Benefits Section).

There was no clear trend evident between the history of engagement within a

catchment and the status of the catchment plan submitted.

Five pilots were fully satisfied (that with the time, resources and support provided)

the catchment plan provides a good representation of the catchment and the majority

(17 out of 24) were at least mostly satisfied. One pilot, the Eden, was not satisfied

but gave no further comment (possibly something for further investigation in the

follow up interviews).

Figure 2.3: Question 81 (23 responses)

Comparative analysis shows that:

Catchment size - the pilot hosts most satisfied that the output was a good

representation of the catchment were working in small catchments (several hosts

from larger catchments didn‟t comment)

Land-use – hosts in rural catchments tended to more satisfied, but there are

examples of hosts from catchments of all land-use types reporting they are fully

satisfied with that the plan provides a good representation of the catchment.

History of engagement – hosts from pilots with a low history of previous

engagement were all „mostly‟ satisfied that the plan provides a good

representation of the catchment (i.e. not fully or quite satisfied).

Host type and bottom-up vs. top down – do not appear to be factors

determining level of satisfaction with the plan.

Run time – pilots running < 12 months are all mostly satisfied (i.e. none were

fully satisfied, or quite satisfied). Many of those running for >18 months were only



quite satisfied.

Table 2.1. below summarises the common outputs produced as a result of the pilot

programme.

Table 2.1. Common outputs produced as a result of the pilot programme

Output Total Stated level of pilot host satisfaction that it was a good representation of the catchment

Not satisfied Quite satisfied

Mostly satisfied

Fully satisfied

Baseline activity and SH maps

20 0 9 10 1

Catchment appraisals 21 0 5 12 4

Catchment plans 25 0 4 12 5

Comparative analysis shows:

Pilot hosts were more satisfied that the outputs they had produced were a

good representation of the catchment in small and in rural catchments where

all or almost all pilots reported that they were at least “mostly satisfied” with

all three outputs.

Pilot hosts were also more satisfied with their outputs in newer pilots and

those with lower history of engagement; all pilots running for less than a year

reported that they were at least “mostly satisfied” that their catchment

appraisals and catchment plans were a good representation of the

catchment. This may reflect higher levels of expectation (or frustration?) in

pilot catchments with a long history of engagement.

Pilots hosted by the EA and those hosted by other stakeholders showed

similar levels of satisfaction with outputs from their pilots.

2.2 Other outputs Pilots reported a wide range of other outputs which fall into three broad categories:

1. Material supporting the development of the plan: for example, catchment

maps and scoping reports based on data from surveys including walk-over

surveys and/or desk studies, or literature reviews, e.g. of existing management

strategies, policies and projects, various mapping outputs using ArcGIS mapping,

Crowdmap, and Scimap modelling (e.g. in Teme catchment).

2. Material to engage others with the plan and to promote the plan: for

example, posters, maps of catchment issues and activities, newsletters,



presentations, engagement flyers and catchment summary documents, space on

the EA website promoting the approach and current projects, a community film

and giant map (e.g. The River Story in the Bristol Avon to tell the 'story' of the

catchment from source to sea) specialist outputs (e.g. Canoeing report looking at

opportunities for canoeing across the whole catchment, leading to the potential

development of a Canoe Trail on the Lower Ribble).

3. Documents defining working methods: for example terms of reference for the

Project Board and Strategy Group, partnership agreements (e.g. The Adur &

Ouse Partnership Agreement), guidance for Task and Finish Groups (e.g. Adur &

Ouse TFG starter pack), good practice case-studies (e.g. for Thames estuary

presented in an Appendix to the Final Project Report), engagement and

communications plans.

Pilots also reports outputs that may be better defined as “outcomes”. Some

outcomes related to improved strategic working such as better integration with spatial

planning strategies, identification of gaps in work needed in the catchment and

identification of delivery mechanisms linked up to other areas such as development

and flood authority work. Other outcome-based achievements included positive

experiences of partnership engagement, greater interest from the local community,

improved working relationships, enhanced reputations and better stakeholder

understanding of the issues in each catchment. These benefits recognised by pilot

hosts are discussed further in Section 6.



3 Support

3.1 Support from Defra through learning events In addition to the direct funding, Defra offered support to the pilots through learning

events to facilitate sharing of others pilots expertise and the results of the evaluation.

The majority of pilots found the learning support useful, particularly for sharing other

pilots expertise (19 out of 24) less so for sharing evaluation findings (14 out of 24).

Two of non-EA hosting pilots noting this was „not asked for‟. Going forward, most

pilots thought that learning support was preferred many thought it was essential (10

for sharing expertise and 7 for evaluation). Three pilots reported that learning events

to share evaluation results was not needed, all of whom were non-EA hosted pilots.

Figure 3.1: Question 84: Support from Defra. To what extent was this support provided to

you and how useful was it? (24 responses)

0% 50% 100%

To facilitate sharing theCascade evaluation findingsthrough learning events and

communications?

To facilitate knowledge sharing of other pilots’

expertise through learning events and communications?

Provided and useful Provided but not usefulAsked for but not provided Not asked forNo view



Figure 3.2: Question 85: Support from Defra. How important is it that this support is

provided to others taking forward a catchment based approach? (24 responses)

3.2 Support from Natural England and the Forestry Commission

Figure 3.3 summarises how useful the pilots considered the three types of support

promised from Natural England and the Forestry Commission. Pilot‟s views were

variable and in many cases support was not asked for. The most requested support,

from EA and non-EA hosted pilots, was help to „co-ordinate with other landscape

initiatives‟ (18 of 23). Where this was provided, all hosts found it useful, but 4 pilots (2

EA and 2 non-EA) asked for this information and it was not provided. Information on

water quality was asked for less often, particularly from EA hosted pilots (2-4 cf 9-10

of non-EA hosted pilots). Where this information was provided non-EA hosted pilots

in particular found the support was useful but again there were cases (2-3) where the

information was asked for but not given. Only one EA hosted pilot indicated that the

support on water quality and the catchment promised this time from Natural England

and the Forestry Commission was not needed going forward (Figure 3.3).

0% 50% 100%

To facilitate sharing the Cascadeevaluation findings through

learning events andcommunications?

To facilitate knowledge sharing of other pilots’ expertise

through learning events and communications?

Essential Preferred Not needed No view



Figure 3.3 Question 86: Support from Natural England and Forestry Commission. To what

extent was this support provided to you and how useful was it? (23 responses)

Table 3.1. List of support from the Natural England and the Forestry Commission that pilot

hosts noted was asked for but not provided.

No of

pilots Support components most commonly asked for but not provided

4

To help co-ordinate with other landscape level delivery initiatives, including action within

catchment sensitive farming areas and signposting to potential new Local Nature

Partnerships where appropriate.

2 To share information on water quality and the catchment.

2

To share information on water quality and the catchment in a tailored format for the

catchment

0% 50% 100%

To help co-ordinate with otherlandscape level delivery initiatives

To share information on water qualityand the catchment in a format

tailored to the catchment.

To share information on water qualityand the catchment.

Provided and useful Provided but not useful

Asked for but not provided Not asked for

No view

Defra Evaluation of the Catchment Based Approach – Pilot Stage Final Review Draft Final


Table 3.2 Summary of responses to support provided by the Natural England and the Forestry Commission split by EA and non-EA

hosted pilots (Question 86)

EA Other stakeholder

To share information on water quality and the catchment.

To share information on water quality and the catchment in a format tailored to the catchment.

To help co-ordinate with other landscape level delivery initiatives, including action within catchment sensitive farming areas and signposting to potential new Local Nature Partnerships where appropriate.

To share information on water quality and the catchment.

To share information on water quality and the catchment in a format tailored to the catchment.

To help co-ordinate with other landscape level delivery initiatives, including action within catchment sensitive farming areas and signposting to potential new Local Nature Partnerships where appropriate.

Provided and useful

1 1 4 8 5 10

Provided but not useful

1 0 0 1 1 0

Asked for but not provided

2 1 2 1 1 2

Not asked for 3 5 2 3 5 1

No view 2 2 1 1 2 1

Blank 1 1 1 1 1 1

Sum 10 10 10 15 15 15



Figure 3.4: Question 87. Support from Natural England and Forestry Commission. How

important is it that this support is provided to others taking forward a catchment based

approach? (24 responses)

3.3 Support from the Environment Agency The EA offered to provide a wide range of support to the pilot catchments, set out as

16 components. Most support was considered to be „provided and useful‟ in the

majority of catchments. Going forward, all of the support components promised by

the EA was seen as essential by at least 4 pilot hosts (see later).

For EA hosted pilots where support was provided it was always seen as useful. This

was also true for most non-EA hosted pilots but not for all (max 2 pilots for any one

component). Non-EA hosted pilots may need more support to help understand how

to make use of the information provided or that it was not provided in a usable form.

This is supported by a comment from one pilot host who commented “A meeting was

set up with EA to discuss WFD data. It was supplied in a non-useful way and not

explained. If the pilot is to be used for WFD progress, it is essential to have a clear

picture of the WFD data and what it means”.

More often the support offered was not asked for and in some cases the support was

asked for but not provided. Further detail on these is given below. For pilots not

hosted by the EA 11 out of 14 said that the EA had contacted them to agree the level

of support.

0% 50% 100%

To help co-ordinate with otherlandscape level delivery initiatives

To share information on waterquality and the catchment in a

format tailored to the catchment.

To share information on waterquality and the catchment.

Essential Preferred Not needed No view



Table 3.3. Summary of responses to support provided by the Environment Agency split by

EA and non-EA hosted pilots

Q88 Support components provided by the Environment Agency

i) ii) iii) iv) v) vi) vii) viii) ix) x) xi) xii) xiii) xiv) xv) xvi)

Provided and useful 7 8 7 8 8 6 1 2 8 5 8 7 2 4 3 7

Provided and useful 8 5 6 12 12 9 2 6 7 7 9 8 4 2 4 4

Provided but not useful 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Provided but not useful 2 1 0 1 2 1 0 0 0 0 0 1 0 1 0 0

Asked for but not provided 0 0 0 0 0 1 0 3 0 1 0 0 0 0 0 0

Asked for but not provided 0 1 1 2 0 3 1 2 2 3 4 3 1 1 0 2

Not asked for 1 0 1 0 0 1 6 2 0 2 0 1 5 3 2 1

Not asked for 3 7 6 0 0 1 9 3 0 2 0 1 6 8 7 6

No view 1 1 1 1 1 1 2 2 1 1 1 1 2 2 4 1

No view 2 1 2 0 1 1 3 3 6 3 2 2 4 3 4 2

Blank 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

Blank 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1

Sum EA hosted 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10

Sum non-EA hosted 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15

Support components in full: i) Access to tools and a network for continuous learning. This would include all pilots, as well as

other relevant initiatives happening elsewhere. ii) Access to Environment Agency meeting rooms, subject to their availability. iii) Strong endorsement of your hosting role in relevant publicity. iv) A willingness to work collaboratively with others in the catchment, where there are shared

goals. v) Access to environmental quality evidence from River Basin Management Plans packaged for

the catchment, and support in interpreting these data. vi) Access to detailed environmental quality evidence and results from investigations relevant to

the catchment that were not accessible from the River Basin Management Plans. vii) An initial assessment of the additional economic value of reaching good status/potential and

meeting all Water Framework Directive objectives in water bodies across the catchment. viii) A commitment to include new actions developed through this process into River Basin

Management Plans where it is clear that stakeholders have committed to the action. ix) A commitment to participate in actions and projects relevant to us, that arise from these new

approaches. x) Regular updates of changing information that may arise from investigations or other information

that affects the confidence we assign to reasons for failure in the catchment. xi) Information on all initiatives and on-going activity being undertaken by the Environment Agency

that affects the catchment. xii) Assistance in decision making on priorities. xiii) Helping to devise measures that provide value for money. xiv) Providing quality assurance around the effectiveness of measures. xv) Co-ordinate information from Natural England through our local office contacts. xvi) Access to facilitation support to engage stakeholders and members of the public.



The most useful support provided by the EA was:

A willingness to work collaboratively with others in the catchment, where there

are shared goals (20 out of 24).

Access to environmental quality evidence from River Basin Management Plans

packaged for the catchment, and support in interpreting these data (19 out of 24).

Information on all initiatives and ongoing activity being undertaken by the

Environment Agency that affects the catchment (17 out of 24).

The support most commonly not asked for was:

An initial assessment of the additional economic value of reaching good

status/potential and meeting all Water Framework Directive objectives in water

bodies across the catchment. This could form the basis for discussing alternative

cost and benefits valuations that might emerge during engagement

processes"(15)

Providing quality assurance around the effectiveness of measures" (12)

Helping to devise measures that provide value for money" (11)

It is likely that may this support was not asked for as many pilots did not reach this

stage during the 12 month pilot period.

Support promised this time, that pilots report they asked for but it was not provided is

listed in Table 3.4. This situation is reported by both EA and non-EA hosted pilots

though for EA-hosted pilots there were fewer unfulfilled requests (5) covering fewer

support components (3; namely vi,(1) viii, (3) x, (1)) than for non-EA hosted pilots

where there were 26 unfulfilled requests across 13 support components.

For example looking at “x) Regular updates of changing information that may arise

from investigations or other information that affects the confidence we assign to

reasons for failure in the catchment”, 4 non-EA hosted pilots reported that this was

asked for but not provided whereas all EA pilots and 9 non-EA hosted pilots said this

was provided and useful.



Table 3.4 List of support from the Environment Agency that pilot hosts noted was asked for

but not provided.

EA hosted

pilots

Non-EA

hosted pilots

Support components most commonly asked for but not

provided

1 4 x) Regular updates of changing information that may arise

from investigations or other information that affects the

confidence we assign to reasons for failure in the catchment

1 3 vi) Access to detailed environmental quality evidence and

results from investigations relevant to the catchment that

were not accessible from the River Basin Management Plans

3 1 viii) A commitment to include new actions developed through

this process into River Basin Management Plans where it is

clear that stakeholders have committed to the action

0 4 xi) Information on all initiatives and ongoing activity being

undertaken by the Environment Agency that affects the

catchment

0 3 xii) Assistance in decision making on priorities

0 2 iv) A willingness to work collaboratively with others in the

catchment, where there are shared goals.

0 2 ix) A commitment to participate in actions and projects

relevant to us, that arise from these new approaches

0 2 xvi) Access to facilitation support to engage stakeholders and

members of the public

0 1 ii) Access to Environment Agency meeting rooms, subject to

their availability.

0 1 iii) Strong endorsement of your hosting role in relevant

publicity.

0 1 vii) An initial assessment of the additional economic value of

reaching good status/potential and meeting all Water

Framework Directive objectives in water bodies across the

catchment. This could form the basis for discussing

alternative cost and benefits valuations that might emerge

during engagement processes

0 1 xiii) Helping to devise measures that provide value for money

0 1 xiv) Providing quality assurance around the effectiveness of

measures



Going forward, all of the support components promised by the EA was seen as

essential by at least 4 pilot hosts. Table 3.5 below identifies the support components

most commonly identified as 'essential' and includes support provided across the

Defra family.

Table 3.5 Support elements seen as essential moving forward

Support component

No of pilots considering

this

Essential Preferred

iv) A willingness to work collaboratively with others in the catchment,

where there are shared goals. 23 0

ix) A commitment to participate in actions and projects relevant to us,

that arise from these new approaches 21 2

viii) A commitment to include new actions developed through this

process into River Basin Management Plans where it is clear that

stakeholders have committed to the action 20 2

xi) Information on all initiatives and ongoing activity being undertaken

by the Environment Agency that affects the catchment 20 3

vi) Access to detailed environmental quality evidence and results

from investigations relevant to the catchment that were not

accessible from the River Basin Management Plans 19 4

x) Regular updates of changing information that may arise from

investigations or other information that affects the confidence we

assign to reasons for failure in the catchment 18 5

From NE/FC: To help co-ordinate with other landscape level delivery initiatives, including action within catchment sensitive farming areas and signposting to potential new Local Nature Partnerships where appropriate. 16 6

iii) Strong endorsement of your hosting role in relevant publicity. 15 8

v) Access to environmental quality evidence from River Basin

Management Plans packaged for the catchment, and support in

interpreting these data 15 8

From NE/FC: To share information on water quality and the catchment. 14 5

From Defra: To facilitate knowledge sharing of other pilots‟ expertise through learning events and communications? 11 10

xiii) Helping to devise measures that provide value for money 9 13

From NE/FC: To share information on water quality and the catchment in a format tailored to the catchment. 9 9

xii) Assistance in decision making on priorities 8 13

From Defra: To facilitate sharing the Cascade evaluation findings through learning events and communications? 8 11



Support offered this time but identified as „not needed‟ was: xv) Co-ordinate

information from Natural England through our local office contacts (by 6 pilots); xvi)

Access to facilitation support to engage stakeholders and members of the public (5);

ii) Access to Environment Agency meeting rooms, subject to their availability (4); To

facilitate sharing the Cascade evaluation findings through learning events and

communications? (3) i) Access to tools and a network for continuous learning. This

would include all pilots, as well as other relevant initiatives happening elsewhere.(2);

vii) An initial assessment of the additional economic value of reaching good

status/potential and meeting all Water Framework Directive objectives in water

bodies across the catchment. This could form the basis for discussing alternative

cost and benefits valuations that might emerge during engagement processes (2);

xiv) Providing quality assurance around the effectiveness of measures (1); 87.

Support from Natural England and Forestry Commission to share information on

water quality and the catchment, in a format tailored to the catchment (1).

OTHER SUPPORT

Other support that pilots noted as essential going forward was:

Funding for the host to continue with the CaBA and to pay for systems, web

portals and facilitation etc.

Funding to pump-prime future delivery. These need to be flexible and not

overburdened with red-tape.

Opportunity to input into the PR14 process

In-kind contribution i.e. staff time dedicated to the CaBA

County Council support for assisting in public engagement.

NFU support for assisting in landowner engagement.

Managerial support, across the board within the Environment Agency, for the

catchment approach (i.e. a geographically focused way of working) and the need

to invest time in building collaborative relationships.

Accreditation of FIELD advisers by the government (who has the shared

intellectual property on ILD) to support every community.

Single source for all the learning tools from EA engagement advice, Cascade,

and catchment hub.



4 Engagement and Collaboration

4.1 Introduction This section focuses on three areas (Q93 – Q10):

Engagement structures and composition: What was the structure and

composition of the pilots‟ engagement process? Was it considered to be a good

balance of stakeholders? Were the public engaged? Were there clear lines of

accountability?

Collaborative working

4.2 Engagement structures and composition Through the evaluation we have collected information on what groups each pilot had

formed and how those groups worked together. From that information we could see

three general structures across the pilots. In this final review we asked pilots the

extent to which they would describe their pilot as fitting into one of these structures.

Figure 4.1 provides the answers. It can be seen that all three structures and some

in between (the “other” category) are represented by the pilots, although the second

structure is less frequent. What is clear is that the structure of the engagement

processes develops to meet the needs of the catchment rather than there being a set

structure which works better than another. Whilst we can fit the majority of the pilots

into three groups, in reality there are subtle differences between them all.


With respect to who has been engaged across the pilot, the majority of respondents



(16) felt that there had been a balance across public, private and community sectors.

In the comments the key groups mentioned as not engaged with were community

groups and the wider public (10) with some mention of the private sector (3) and one

mention of the local authority.

With respect to the comparative analysis few pilots of any size had a „small core

group of stakeholders‟. Small catchments described themselves similarly across

several options as having a „core group of around 15 stakeholders who meet

regularly‟, a „pilot host and project officer acting as coordinators‟ and „other‟, a

smaller proportion formed a „small core group‟. Medium sized catchments were most

likely to have a „core group of around 15 stakeholders who meet regularly‟, whilst

large catchments were most likely to have a coordinator.

Of the non-EA pilots, the majority of those with high facilitation support (15 days)

reported having a coordinator, those with less support (6 days) were equally split

between having a „core group of 15 stakeholders‟ and „other‟, whilst there was no

discernible pattern for those with the lowest support (0-2 days).It might be that those

who had just a coordinator needed more support in terms of direction and asked for

that.

The majority of EA pilots had a „core group of around 15 stakeholders‟, whereas the

majority of „Other‟ pilots had a coordinator . This is perhaps not surprising as the

suggested approach for the EA pilots was to set up a catchment group of

stakeholders.


With respect to comparative analysis all those with a low history of engagement

reported that there had been a balance across public, private and community group

involvement in the pilot. The majority of those with a medium history said that they

agreed with this statement, whereas the majority of those with a high history said that

they did not .

Q109 picked up on whether respondents thought that there were key stakeholder

perspectives missing from the pilot outputs and 14 respondents did say that there

were key stakeholder perspectives missing from the pilot outputs. The perspectives

identified as missing were the following:



Table 4.1 Perspectives thought to be missing from the pilot outputs

Riparian owners 2

Industry/business 3

Local authorities 4

Angling 2

Public 4

Natural England 2

Forestry Commission 1

Infrastructure managers 1

An overwhelming majority of large catchments reported that they thought that key

stakeholder perspectives were missing from the pilot outcomes. Small and large

catchments were more likely to say that they „don‟t know‟ if any key stakeholder

perspectives were missing from the pilot. The proportion of small catchments that

said that they „didn‟t know‟ was only slightly less than the proportion that said that

there were perspectives missing – suggesting a large amount of uncertainty for small

catchments here.

The comments also reflected that pilots were either going to make efforts to engage

with those that they thought were missing, or that they had known within the time it

would not be possible to get particular perspectives incorporated into their outputs

(i.e. members of the public).

In terms of engagement with the public (Q97) the majority of the pilots (21) have

engaged with the public during the pilot. In terms of how those activities have

influenced the pilot over all, comments fell into three main categories:

1: Influence had been minimal or had not yet had a chance to influence (3)

2: Influence had been to some degree and quite specific (5): e.g.

“landowners and farmers probably more so than the wider community”



“but only with Angling clubs, it is anticipated that true public engagement will come

after the plan is launched and they will be able to get involved in delivery and

developing new actions/outcomes and contribute to the plan as it evolves”

3: Influence has been on plans, development of projects, cross referring to other

plans and in giving confidence and support for actions (6)

“Guided the development of projects that are included in the Catchment Plan”

“the types of views obtained tended to be more general in nature and supportive of

the approach and objectives for improving the water environment.”

In terms of comparisons all of the non-EA hosts felt that they had engaged the public

during the pilot and a majority of EA hosts felt the same, however a third of EA pilots

said that they had not.

Finally, all respondents thought that their methods of stakeholder engagement had

been appropriate or very appropriate to their objectives.


4.3 Collaborative working “Collaborative working involves sharing power, risks and ownership of the

process” (Project definition of collaborative working)

We would suggest that indicators of collaborative working include:

Openness and responsiveness of the process – if power and risks are being

shared then the process should have a process for stakeholders to express their

feedback and to be responding to the needs of stakeholders. If the process is

owned by a number of stakeholders we would expect the objectives of different

stakeholders to be met through the process.

Indication of a shared identity - the attitude of “we are all in this together”

comes from sharing power and risks and indicates shared ownership of the



process. This maybe expressed tangibly through names, logos etc or less

tangibly through the way groups discuss and refer to the catchment process.

Collaborative advantage - Finally, we wanted to know the extent to which hosts

felt that they had achieved collaborative working and whether there was a

“collaborative advantage”, that their achievements were due to collaborative

working and if the collaborative nature of the pilot had motivated stakeholders to

get engaged in the process.

4.3.1 Openness and responsiveness of the process

A group of questions were asked that focus on the openness and responsiveness of

the process (Q96;95;99;102). The majority of respondents (20) said that they

thought their pilots had been open and accessible to all those who wanted to engage

with the pilot (Q96). Pilots commented that they had made efforts to be open and

accessible:

“We have never turned anyone away from a stakeholder exchange meeting or

workshop and have actively approached stakeholders to be involved.”

“We have attempted to be as universally welcoming as possible”

In terms of responsiveness, ensuring that the process acknowledges all views both

for and against is important and 18 of the respondents thought that had happened

“very much” within their pilot. Further, the majority (16) felt that the process had

been quite effective in meeting the objectives of the majority of the stakeholders and

all respondents thought the process was able to change in relation to the needs of

the stakeholders. Taken together these findings suggest that pilot hosts thought their

processes were open and quite responsive to the needs of stakeholders.






4.3.2 Indication of a shared identity

The creation of a new identity or working that goes beyond organisational boundaries

can be a marker for collaborative working. 21 of the pilots said that they had

“somewhat” or “very much” created their own unique identities which went beyond

organisational boundaries. In the comments 9 of the pilots reported new names and

logos and going beyond organisational identities, 3 felt there was a sense of a unique

identity but it had not been put into a brand (one deliberately had not done this to

reduce cost), but was “just in stakeholders minds so far. Quite a strong group feel?

No logo yet!”. Clearly these identities are evolving and as one comment suggested it

may be less indicative of collaborative working if having a new name/brand became

an objective of collaborative working.


In terms of the comparative analysis the majority of large catchments thought that

their pilot had „very much‟ created its own unique identity that went beyond

organisational boundaries. The majority of medium and small catchments thought

their pilots had „somewhat ‟.



4.3.3 Collaborative advantage

The majority of the pilots thought that “collaborative working” has been achieved very

much or to a degree. In their comments some are unreserved in it having worked in

the planning stage, others were waiting to see if it would be maintained through to

the implementation stage, with others feeling it was too early to tell and some with

partners who are working much as usual.

“Yes, to a large degree. It works well as a steering group if we need to discuss

anything in particular or to share news of activities. We think it stills needs time to

evolve and to make progress on an action plan before we will be able to see results

of collaborative on the ground action for specific projects brought about by the Pilot

Plan.”

“Very much so in the planning phase. Wait to see if continues into the

implementation stage.”

“Still too early in the process. We are still hand holding a number of stakeholders.”

“Many partners continue to behave as "business as usual" in regard to their work -

hence key phase of coordination now required if a piece-meal approach to delivery is

to be avoided”


For the majority of pilots they thought that the collaborative nature of the pilots “to a

degree” or “very much” was a key motivation for stakeholders to get involved,

suggesting that there was an appeal to the process, that it would be different from

the usual approaches.





All the pilots who responded felt collaborative working had led to the achievements of

the pilot “to a degree” or “very much”, and that was reflected in the outputs of the

pilots, specifically in the catchment plans.

“The catchment plan is/will be a highly collaborative piece of work”

“Plan is collaborative, reflects views of all”

“It’s very apparent in the professionally designed catchment plan that this is

something different – the output of a collaborative process. Not an EA document”

Part of collaboration is the extent to which skills and experience of stakeholders have

been drawn on (Q110). All 24 who responded felt that had been quite or very

successful, providing a lot of different examples ranging through the general to the

specific, but with some comments about the inability of stakeholders to engage

completely due to limited time and resources.

“Local knowledge and experience from anglers as part of Angling workshop is

leading to their involvement in habitat walkover surveys of individual fishing beats

and is an action on the Catchment Action Plan.”

“We are currently planning volunteer-led catchment walkovers within the catchment,

which will bring a broad range of perspectives to this technical process.”

“Third sector organisations/ colleagues - in this catchment specifically Wildlife Trust

and the Rivers Trust - are highly experienced in delivering environmental outcomes

by working together with others. the spirit of collaborative working is already

understood, which helps the process evolve faster.”



“They contributed their time, understanding and practical help like meeting rooms.

However many stakeholders were struggling to meet 'day job' commitments and

were limited in the resources they contribute though would perhaps have liked to”.

“It has struggled as it's not a high priority catchment for most, but individuals involved

have been as committed as possible and provided valuable skills and experience to

date.”

Those with a higher (high or medium) history of engagement were more likely to say

that collaborative working has „very much‟ led to the achievements of the pilot. No

trends were able to be drawn from analysing facilitation support. An overwhelming

majority of EA pilots said strongly agreed („very much‟), whilst only a smaller majority

of „Other‟ pilots strongly agreed.



5 Learning

5.1 Introduction This section covers factors that facilitate or act as barriers to an increase in the

knowledge and capacities of the catchment partnership. The questions looked at:

The advice and support that pilots received from internal and external sources

The way in which sharing knowledge and experience has helped to find creative

solutions to catchment problems and improved the work of the pilot

The role and impact of cross-catchment learning and knowledge sharing

5.2 Advice and support from internal and external sources

The majority of pilot hosts said that they had received advice and support which had

helped the catchment partners to develop shared understandings of the catchment.

Of the three hosts that said they had not had this kind of support, one explained that:

All shared understanding [was] achieved by the steering group and work of the pilot

host.


The pilot hosts described a wide range of sources and types of support, including:

Support from key catchment stakeholders like the Environment Agency (7

mentions), water companies (4 mentions), Natural England (1 mention), the

Rivers Trusts (2 mentions) and the Wildlife Trusts (1 mention).

Local Authorities (3 mentions)

Experts and consultants (3 mentions)

Facilitators (2 mentions)



Two pilots mentioned SAGIS (Source Apportionment GIS model) as a tool for advice

and support.

Some pilots described how this advice and support is brought into the pilot‟s work:

Use of internal and external technical experts to get a better picture of issues

and problems in the catchment. Specific experts have sat on the sub groups

to help with this.

Sub-groups or task-and-finish groups appear to be forums in which knowledge is

shared:

Advice and support has been provided by steering group organisations and

partners sitting on task and finish groups.

For some, increasing knowledge and expertise has been important not just for

improving the work of the partnership but also enabling better communication with

communities:

Technical information from EA/UU has built understanding, and enabled WFD

issues to be better interpreted to community stakeholders.

...the advice and local expertise of Pond Conservation has been important to

engage stakeholders.

All the pilot hosts felt that they had been successful in drawing on the skills and

expertise of partners (9 considered this had been very successful, and 15 said it had

been quite successful). There were many examples of stakeholders contributing

specific technical skills:

Excellent publicity copy writing by NFU rep!

Spatial planning as a key example

We are currently planning volunteer-led catchment walkovers within the

catchment, which will bring a broad range of perspectives to this technical

process.

Broader expertise and experience of working together were also seen as an

important skill to share:

Third sector organisations/ colleagues - in this catchment specifically Wildlife

Trust and the Rivers Trust - are highly experienced in delivering

environmental outcomes by working together with others. The spirit of



collaborative working is already understood, which helps the process evolve

faster.

However, several respondents sounded a word of caution about the ability of

partners and others stakeholders to contribute all the time and expertise needed for

effective catchment management:

They contributed their time, understanding and practical help like meeting

rooms. However many stakeholders were struggling to meet 'day job'

commitments and were limited in the resources they contribute though would

perhaps have liked to.

NFU involvement due to "Pilot Status" [and] may not able to contribute to the

same extent in the future.

5.3 The role of knowledge sharing in facilitating creative solutions

Almost 80% of pilots said that knowledge sharing had helped find creative solutions

to catchment problems. Just over 8% felt that it had not helped very much and

12.5% said that they didn‟t know.


The majority of respondents felt that sharing knowledge and experience helped to

find creative solutions to catchment problems (19 out of 24) with 8 respondents

saying that this had helped „a lot‟).

There was a slight difference in responses between catchments of different sizes:

smaller catchments were more likely to say that sharing experiences had helped „a

lot‟ than to choose other options, whereas medium and large catchments were more

likely to say it had helped a bit. In a similar way, catchments with a high level of

historical engagement were slightly more likely to say that sharing knowledge had

helped a lot, while those with medium or low levels of historical engagement were



slightly more likely to say that it had only helped „a bit‟. Some of the examples of the

ways in which creative solutions had been facilitated included:

The proposal to develop an Upstream Thinking (CSF style approach)

Business Case with one of the water companies in the Partnership.

Action cited for Tidal Ribble sub-catchment includes engagement of kennels

and catteries in tackling diffuse pollution. Action cited for Tidal Ribble sub-

catchment includes engagement of kennels and catteries in tackling diffuse

pollution.

The FIELd adviser designs the restoration of the river bank with the help and

knowledge of the landowner, integrating any other specialist knowledge from

partners [and] match funding from all together, [to] fix the river bank and

deliver a project with multiple benefits that delivers for all.

Even some of those who responded „Don‟t know‟ suggested that it might be too early

to see the results of knowledge sharing and that solutions to catchment problems

would come into their own once the plan had been completed:

This potentially could progress with a complete action plan in assessing

solutions.

We have not reached the detailed solutions stage yet but I am expecting

some creative solutions to be initiated through shared knowledge and

experience exchange.

Only one respondent suggested that creative solutions were not really needed for

their pilot:

Due to the experience of participants and the nature of the priorities of the

catchments the solutions have not been required to be creative.

Over 70% of the pilot hosts also said that sharing knowledge and experience had

improved the work of the pilots in other ways. One pilot said that knowledge sharing

had not brought any improvements, while the remaining 25% were unsure about

whether knowledge sharing had made any contribution to improvements. Of this

25%, one pilot host suggested that this might happen in the future.




The majority of respondents (71%) said that, as well as helping to find innovative

solutions, sharing knowledge and experience had also improved the work of the pilot

in other ways. Only one person felt that this had not helped.

The pilot hosts provided a wide diversity of examples of ways in which the pilots had

been improved. Some gave quite specific examples of things that had improved,

including data sharing processes, information for stakeholders and projects (reed

bed development). One said their work had seen a broader improvement in

„Understanding [of] the technical side of catchments’.

For many, the improvements had been as a result of better understanding of each

other, leading to better ways of working together:

A greater understanding of all organisations and their roles and

responsibilities and the opportunities to deliver together.

Better joint understanding people’s aspirations and constraints.

Several also described how sharing knowledge had increased transparency and trust

in the decision-making process, leading to better decisions being taken:

During a meeting a respected land owner, who also sails in Poole Harbour

gave his own personal account of changes that have occurred over 40 years -

this appeared to be a very powerful driver in other landowners accepting

there was an issue.

Improved quality of decisions and shared engagement and support for the

solutions and decisions e.g. brining local experts together to discuss definition

of coastal lagoon feature.

... sharing information promotes transparency, thus trust, and the confidence

that well informed, justifiable decisions are being made. Confident decisions

are more likely to result in effective action.



5.4 The role and impact of cross-catchment learning and knowledge sharing

Learning can also happen across catchments and during the pilot phase some efforts

were made to bring together hosts and partners from different pilots to share ideas

and experiences in order to spread learning. Almost 80% of respondents said that

they had been involved in this kind of cross-catchment learning. Of the 5 pilots that

said they hadn‟t been involved in this kind of knowledge sharing, all had attended

more than one pilot learning event, so it would seem they interpreted the question as

referring to other kinds of cross-catchment contacts.

65% of those who reported participating in cross-catchment learning activities said

that these had made a difference to their own work.



Some of the examples of the impact of cross catchment learning referred to quite

specific topics of information, such as Payments for Ecosystems Services or the

Water Companies Periodic Review 2014. However, a recurrent theme seemed to

be the value of learning from others who were in a similar situation in their own

catchments:

Especially in the early days, this was a useful way of seeing what worked and

what didn't, and for gauging how other pilots were developing, and for sharing

ideas and helping each other.

Two pilots gave examples of one to one contacts with another pilot that had made a

difference to their work: one was the „study buddy‟ relationship between the Eden



and Tyne pilots and the other was the Wey Pilot which took lessons directly from the

work of the Adur and Ouse pilot:

We lent on experience of Adur & Ouse when initially drafting our Partnership

Agreement

The value of these exchanges was often reported by others as being not in the direct

lessons that one pilot could take away from the experience of another, but in less

tangible impacts, such as improving morale and recognition of their own pilot‟s

uniqueness:

It has been very helpful for morale to understand that the main challenges we

have experienced are similar to those faced by other pilots.

Appreciation of the uniqueness of the pilot areas, and the differing

approaches taken.

It has provided appreciation (and therefore confidence) of the different type(s)

of product that can result from collaborative working, depending on the scale

of the geography involved. This reflects an overall learning point that no-one-

size-fits-all and catchment approaches can each look very different.

One of the pilots where cross-catchment learning had not made a difference still felt

that this aspect of the work had been valuable:

Sharing knowledge and experience with other catchment pilots was very

useful and provided encouragement to our work but it did not directly

influence the work of our pilot.



6 Benefits

6.1 Planning and Engagement Q115 asked participants to consider the pilot as a whole and to identify up to three

main benefits achieved in terms of planning and engagement. The 22 respondents to

this question identified a wide range of benefits from their pilots. The strongest

message coming from answers to this question was in relation to the establishment

of stakeholder networks and a new approach of joined-up working.

A majority of respondents recognised some kind of benefit from the creation of the

stakeholder network with comments noting, for example, they had been able to draw

in a wide range of stakeholders (including new partners in some cases) with a shared

vision, openness, ideas sharing etc. Members of the stakeholder networks felt

confident that they had all the right people around the table and that a long term

commitment to this approach was established. The creation of these networks has

led to better communication between key players, a greater understanding of

priorities and improved relationships and the development of trust. These benefits

mare expected to have a wider impact beyond the extent of the phase. The

stakeholder networks created for this project have also acted as a gateway to other

stakeholders through contacts made within the core network.

Through the development of improved stakeholder networks, the pilots reported that

they were able to work in an integrated manner to build up their catchment plan from

common ground with a consensus on the key issues and solutions. The pilots

recognised the value of a plan developed in this joined-up and integrated way and

through many of the responses there was a sense of pride in the plans that have

been produced through this pilot. One respondent noted that the “Stakeholders have

remained engaged and have faith something real will really be delivered by the

process”. This was another common theme seen in the responses to this question

with many respondents noting there was a real and genuine belief that something

good was actually going to come from the catchment based approach.

Another common benefit noted by the respondents was the sharing of information

amongst the stakeholders to provide a sound technical evidence base, with some

referencing to the use of shared GIS portals specifically. Many of the pilots reported

that through this engagement, there was now a better understanding amongst the

groups of the WFD objectives and a wider understanding of the benefits of WFD

delivery.

Some respondents noted in addition to the stakeholder engagement benefits that



were specific to this pilot project, the pilot had also been a learning experience in

terms of planning and running stakeholder events.


The response to Q116 shows how widespread the agreement of these benefits is

across the catchment group. Over 90% of pilots reported widespread agreement of

these benefits, one pilot said these benefits had been discussed together and agreed

within a steering group meeting. Additional comments note that agreement of the

benefits is widespread because everyone has benefitted in some way and many are

optimistic of opportunities coming from the new relationships developed in the pilot.


The majority of respondents (over 65%) do not think the planning and engagement

benefits can be quantified. The main reason provided for this is that there are no

agreed methods for doing so, recognising that many of these benefits are hard to

quantify. Some respondents do think this is possible, to some extent, one pilot said

they were in the process of mapping / quantifying the benefits. A few pilots

suggested potential methods for quantifying benefits, with one noting that an

ecosystem services analysis may be a useful tool for doing so.

The pilots were asked what barriers remained to assessing the planning and

engagement benefits of the pilot and for potential methods to overcome these

barriers (Q118). Key themes that were raised in these answers included a lack of

time and resources and the fact that these benefits are “inherently qualitative” and

therefore difficult to quantify Some suggested it may be possible to quantify benefits

by assessing the number (and cost-benefit) of projects initiated by the pilot that

deliver WFD objectives and other outcomes. Another barrier that was suggested was

that it is hard to quantify benefits amongst different organisations with different

cultures and priorities.



The Tame pilot suggested that in their case the identification and quantification of

benefits was made difficult by the size, complexity and diversity of the catchment and

recognised that it may be easier to assess benefits if the catchment was broken

down further for planning.

6.2 Environmental Benefits Pilots were asked (Q119) to consider the whole pilot phases and assess what the

main environmental benefits achieved by the pilot were. A common response across

participants was that it is too early as yet to identify environmental benefits but many

were optimistic that environmental benefits would be realised in the coming year

(2013) and beyond. Some have already seen an improvement in water quality and

many expect to see future improvements in water quality as the actions identified in

the catchment plans are implemented.

Another common benefit identified by the pilots was a greater understanding of the

environmental problems and issues in the catchments brought about by the

catchment based approach and the move away from the previous, piecemeal and

single-issue approach. The Tame pilot noted that there was now a greater

understanding of the role of the catchment in terms of ecosystem services and a

better understanding of the links between the rural and urban parts of the catchment.

Some pilots provided specific examples of environmental benefits already seen in

their catchment, for example the “Wey Farm Advice Project” and the “River Story”

project in the Bristol Avon and North Somerset Streams which has improved public

environmental awareness. In the Ecclesbourne catchment three examples of

environmental benefits were provided – habitat creation, fish pass construction and

water quality improvements through phosphorous reduction.


A significant majority of pilots (more than 80%) reported widespread agreement of

the environmental benefits across the catchment group. The Lower Lee pilot noted

that although some stakeholders will gain more than others from these environmental

benefits (e.g. biodiversity and water quality improvements) there is now a much

greater understanding among the group of how they impact on each other.



Q122 asked the pilots what they thought the future environmental benefits would be

as a result of the collaborative approach and catchment plan and when they

expected to see the benefits emerging. All pilots were optimistic about seeing some

kind of environment benefit with some expected in the short term (2013), some in the

medium term (3-5 years) and some taking longer to emerge (>5y).

The most common general benefits referred to by pilots included improved

ecosystem services, improved awareness of environmental issues and problems and

behaviour change as a consequence. Specific environmental benefits expected by

the pilots include:

WFD improvements and achievement of Good Ecological Status;

improved water quality (better sediment management, phosphorous reduction

and nitrogen stabilisation);

habitat creation (and continuation);

river restoration and physical improvements to channel morphology, removal of

weirs (construction of fish passes);

local flood alleviation; and

restored species populations and control of invasive species.


The 22 pilots who responded to Q122 were evenly split on whether or not these

environmental benefits can be quantified. Some said that although they could not be

quantified yet, they could be in the future (subject to the availability of time and

resources) but to do so would require a clear establishment of the baseline

conditions and development of a suitable monitoring / survey programme.

Some specific methods for quantifying the benefits were referred to (e.g. no. of trees

planted, water quality data etc) with one pilot noted that some quantification of

benefits is commonly required for funding applications. The value of trout fisheries

was suggested by the Wissey pilot as a good measure of the benefits achieved



through phosphorous reduction.

A range of barriers to assessing the environmental benefits were identified by pilots,

as before, many pilots noted that a lack of time and resources was a significant

barrier, this is a similar response to the same question when asked in the planning

and engagement benefits section. This shows how important a long term

commitment to the approach (and associated funding) is required to maintain

progress in these pilots. Some referred to a lack of research and available tools to

assist in this quantification of benefits but recognised that there were still some

actions they could take anyway such as establishing a water quality monitoring

programme.

6.3 Social Benefits Q124 asked what the main social benefits achieved by the pilots were. The key

themes emerging from this question are the improved awareness and understanding

of the river and issues within the catchment and a greater feeling of „ownership‟

amongst stakeholders and local communities of the river and the catchment plan.

Many responses included words such as “goodwill”, “trust” and “empowerment”.

There seems to be a greater understanding amongst the groups of what can be

achieved by stakeholders and by local communities and several pilots described a

“connection” or “reconnection” between various partners including farmers,

landowners and the EA. One respondent noted that they had been able to join up the

ecological and social value of the river and were considering the social benefits of

access and recreation. One pilot in particular noted that by working as a group they

had broken down previous barriers.

The general tone of the responses is nicely summarised in one response that said

“The spirit of integrated catchment management is more widely accepted as being

the right direction of travel, as opposed to 'implementing the WFD'”.


75% of respondents reported widespread agreement of these social benefits four

pilots felt there was not widespread agreement in this area. The Ribble pilot reported

that not all stakeholders agree on recreation and access issues; this is likely to be

seen in other pilots too. The Adur and Ouse pilot reported that there was enough



recognition of and value to these benefits for the group to continue its work beyond

the evaluated pilot initiative. One pilot that said there was not widespread agreement

(the Tidal Thames) said that they recognised large gaps in their engagement process

due to lack of time and resources and noted that their Strategy Group recognised the

need for an engagement plan for "hard to reach" sectors (e.g. the commercial sector)

in order to fill in some of the existing gaps.


The majority of respondents (>65%) did not think that these social benefits can be

quantified but some were optimistic that whilst this was not possible at the current

time, it would be possible in the future. Suggested measures for quantifying the

social benefits included attendance at steering group meetings, number of public

attending engagement events, number of website hits, Facebook connections,

school visits, number of volunteers etc. In the Irwell pilot, social benefits had been

measured as the number of NEETs (not in education, employment or training) who

have completed environmental training through the pilot and gone on to full time

employment or further education.

One particular case was reported of an individual who said that being involved in the

project had provided “a sense of purpose and direction”. The benefits are hard to

quantify but the pilot noted that people benefit from being valued and socio-economic

benefits are delivered through bringing people together.

As with planning and engagement and environmental benefits, many respondents

noted that a lack of funding and resources was a significant obstacle to the

identification and quantification of social benefits as well as the fact social benefits

are “quite intangible”. Some were daunted by the scale and complexity of the

catchment and suggested targeted sampling as a potential method for evaluating the

social benefits.

6.4 Economic Benefits Question 128 asked pilots to identify up to three economic benefits across the pilot

as a whole. Some respondents noted that it was too early to say what the economic

benefits were but they would be realised in the future. However, many pilots have



already identified economic benefits. The most popular responses related to

improved efficiencies through working together, sharing data, expertise, evidence

and previous work which can lead to more targeted spending. Collaborative working

prevents overlap and duplication of effort, producing one, evidence-based catchment

plan rather than multiple plans.

Some respondents recognised that economic development and regeneration can

drive catchment opportunities and that “a clean, green and naturally functioning

environment is needed to support the local and regional economy”. The catchment

based approach can lead to improved skills and training and job opportunities. At a

local scale there have already been benefits to the local communities through venue

hire and catering for stakeholder and community events.

The majority of pilots believed that there is widespread agreement of the economic

benefits described above.


Pilots were asked what they thought future economic benefits of the catchment

based approach would be. A variety of suggestions were made covering benefits that

would be seen in the short-term (2013) and in the medium and long-term (>5 years).

Again, the most common theme was economic benefit through efficiency savings by

sharing data and working together to get things right. Many commented that the

value of economic benefits seen would depend on the implementation of the

catchment plans. Pilots anticipated a range of economic benefits including tourism

benefits (e.g. recreation, fisheries), the potential recovery of failing businesses and

reduced water company operating costs. The pilots noted that through collaborative

action, more projects were being initiated and they were confident in a greater

success rate from joint funding applications. The Bristol Avon and North Somerset

Streams pilot specifically noted that they anticipated seeing economic benefits

because of the improved understanding of linkages within the catchment, i.e. the

influence that activity in one part of the catchment has on the rest of the catchment.




The majority of respondents think that these economic benefits can be quantified, but

a significant number do not think this is possible. The comments and reason given

were often that it is too early to do so or that the pilots lack the tools and resources to

do, although there was some disagreement about this point with some pilots thinking

they have the tools but not the resources while others think they are lacking the tools

too.

Some pilots provided specific examples of where economic benefits had already

been seen and quantified, such as cost savings made in the Ribble catchment by the

public sector (rather than consultants) providing training to Lead Local Flood

Authorities in Flood Defence Consenting. Other suggestions for quantifying these

benefits included using house prices for riparian properties or changes in water bills.

The Welland pilot highlighted that as well as quantifying economic benefits, it was

important to link to ecological status improvements.

The main barriers identified to quantifying economic benefits (Q132) were, as before,

a lack of time and resources and particularly a lack of expertise in this area. Other

barriers highlighted included the fact it is hard to quantify an avoided cost and the

sensitivity of financial data.

6.5 General Benefits The next section of the questionnaire focussed on general benefits seen across the

whole catchment pilot. Q133 asked each pilot what they thought the single, most

important achievement of the pilot was. There were a wide variety of responses to

this question but some clear, common themes emerged.

The two most common achievements noted by the pilots were:

1. the development of a stakeholder group, bringing together stakeholders who may

previously had little contact and in some cases were already suffering from

„consultation fatigue‟ but who ended the process committed to the approach and

seeing the benefit of continuing in the same way beyond the end of the pilot,

keeping up the momentum established in the initial year; and

2. the delivery of a catchment plan with clear actions and outcomes, signed up to by



a range of stakeholders within such a short time-frame and getting some good

projects initiated already.

Aside from these two common themes, other comments noted the value of adopting

a „truly integrated catchment approach‟ for the first time by sharing evidence,

engaging wider society and combining resources. One pilot recognised the value of

working at a broader scale than within the normal administrative boundaries and

other were pleased at how successful the pilot had been in raising awareness of the

importance and benefits of a catchment based approach and recognising the

interconnectedness of problems and opportunities at this scale. On a similar theme,

pilots noted that a significant achievement had been aligning catchment issues with

other agendas and partnerships and aligning funding streams to realise efficiencies.

The next question (Q134) asked about the single most important lesson learnt from

the pilot, and secondly, how this lesson would influence their future work. Again,

there was a wide range of answers, but some common themes emerged.

Many pilots recognised that taking this approach was hard work and took a lot of time

and effort, but noted that it should not be rushed. Many commented on the need for

good preparation, starting the process with a strategic and planned approach but not

beginning with too high expectations (in order to manage capacity). The pilot leads

recognised the importance of taking time to understand individual issues and

priorities and to identify individual values and expertise and the value of not using a

traditional top-down approach when trying to understand local priorities.

A couple of pilots noted an important lesson in making sure they identified the right

people – not just the right stakeholders but also the right individuals from within the

partner organisations and that those individuals needed to have the backing of their

organisation. One pilot commented that the stakeholders were not altruistic, i.e. they

needed to see what the benefits were for themselves before they would co-operate.

Some pilots commented on the importance of having a strong catchment co-

ordinator to “act as the glue to maintain and motivate collaborative work” with one

saying specifically that this lead should be “professional, paid, skilled and adequately

resourced”.

The Lower Lee pilot responded that the biggest lesson they learnt was that they

should have used different methods to engage the community as they found out that

some of the smaller community groups were put off from attending events through

fear they would not be heard in the presence or large, more powerful organisations.

Q135 asked how many new actions had been identified to improve or protect the

catchment. Many pilots simply referred to their catchment plan in answer to this



question. Others provided a number of actions, ranging from 12 up to 200 with some

commenting on how their actions had been categorised or listed. For example, the

Adur and Ouse pilot reported that they had 200 actions underneath 12 overarching

objectives. Some pilots noted that some of their actions were already underway prior

to the pilot and others were new and therefore jointly agreed and based on the new

engagement work. More than one respondent commented along the lines of “too

many to count”.


45% of respondents reported with they had been able to gain commitment to

undertake these actions either in full, mostly or partly. This has been achieved

through aligning the agendas of different stakeholders through working together

closely, including steering group meetings, the use of working groups (technical task

and finish groups), through workshops and consultation and through integration with

other projects and partnerships. For those who answered “partly” there were

comments regarding the need for wider engagement to achieve greater buy-in and

commitment.

55% of respondents have not yet been able to gain commitment to undertake these

actions for a variety of reasons. Many pilots commented that this was currently

underway and in progress but that this was dependent on availability of time and

resources. One pilot commented that they have some verbal commitment but they

needed to get the action plan formally agreed.

One pilot commented that the strategy group were not able to prioritise and commit

to actions without more data and agreed criteria for assessment in order that they

could move forward in a fair and transparent way. The Don and Rother pilot

specifically said that “We have not sought commitment to take action; rather we have

sought to identify a joint vision, a joint agreement on problems and consensus on

possible options. We felt asking for commitment from stakeholders at this stage

would have stifled this debate. We wouldn't have got the free exchange of views we

did if people though they were legal/contractually committing their organisation to

actions or expenditure”.

The response to Q136 “Have you been able to gain commitment to undertake these



actions?” has been analysed in more detail to see if there are any differences in

response between the types of pilots. There is no significant difference in the

response to this question based on land-use types, catchment size or whether the

pilot worked primarily at the catchment level or local level. Similarly, the history of

engagement within each catchment does not seem to make a significant difference

to the level of commitment to action.

Overall, the level of commitment to action was lower in those pilots that had been

running for a shorter time (<14 months), which may be expected as many

respondents commented on a lack of time being an obstacle to their pilot‟s progress.

It does appear that those pilots that had been running longer were able to develop

stronger engagement and a greater commitment to action. There does not seem to

be a strong correlation between the level of facilitation support and the commitment

to action. The pilots who requested the greatest level of facilitation support (15 days)

reported either no commitment to actions or only part commitment to actions.


Just over half of respondents said that their awareness or use of Ecosystem Services

had changed during the pilot. The host of the Teme pilot, the Rivers Trust

commented that whilst they already knew about Ecosystem Services, the pilot had

given them the opportunity to share this knowledge with others in the pilot group who

became interested and would like to take this further (but don‟t have the resources,

funds or expertise to do so at the moment). Others made similar comments about the

value of the pilot for increasing awareness of Ecosystem Services but recognised

they had not been able to actively use the approach in the pilot.

The Cotswold pilot has been selected as one of the “National Payment for

Ecosystem Service Pilots”. Other pilots discussed the use of Ecosystem Services as

“a tool to get funding by taking it from an abstract concept to a subject matter that

needs site-specific evidence and qualification”. Some pilots hope to use the

approach moving forward as they deliver their catchment plans. This is shown in the

answer to Q138 when almost 70% of respondents said they have or would adopt an

Ecosystem Services approach going forward.




Some pilots said the Ecosystem Approach was already at the core of the way they

worked and others commented that this was probably being done in some cases

without specifically being referred to in this way, for example, the New Forest pilot

said that “Many stakeholders on the ground are very sceptical of the term ecosystem

services and regard it as a piece of unintelligible jargon. They are however fully

committed to an approach that delivers a better quality environment & benefits for

nature and for people … and would see it being delivered in a way that relates to

practical improvements rather than an academic exercise”. The Ecclesbourne pilot

recognised this as “an essential approach for demonstrating the success of any

catchment based approach” but like many others implied that it needed a significant

input of time and resources to do so. Another barrier to the application of this

approach is the concern among some pilots that the approach is too difficult for

people to understand and apply.


Seven pilots recognised they had achieved unexpected benefits from the pilots,

some specific examples including:

involvement in a Landscape Partnerships bid within the upper catchment,

securing funding from a source not previously considered accessible (Ribble);

creation of permanent catchment co-ordinator posts within the Environment

Agency; and

getting the Birmingham and Black Country LNP to take on a role as a “strategic

sub-catchment partnership” in the urban Thame catchment.

The Cotswold pilot said that even though they had not seen any unexpected benefits,

they benefits they had achieved were greater than originally anticipated and made

particular reference to the involvement of the farming community. The Douglas pilot

said they had no idea what to expect at the beginning of the process but have seen

successes in terms of new relationships and positive discussions with the real



benefits being realised over the coming months/years as the plan is implemented.



7 Costs and Value for Money

7.1 Funding Funding for the pilots has come from a variety of sources, including:

Defra – typically in region of £50K per pilot and in some cases more (for example

where pilot has made a successful application to the Catchment Restoration

Fund);

EA – for some catchments, this has been substantial (up to £50K or more); and

Others on the steering group or with interest in the plan (e.g. water company,

NGO).

In addition, facilitation support has been provided by Defra and averages about 6

days per pilot (range of 0-15 days).

A few pilots have received no external funding (effectively carrying out the pilot with a

zero budget) and have relied entirely on in-kind support.

7.2 Costs to date The main financial costs of running the pilot have been:

Staff time or funding of coordinator;

Communication and meetings (e.g. workshops, web sites, travel to meetings);

and

Contractor costs (e.g. facilitation, feasibility studies, data collection/monitoring).

The greatest cost burden has been on pilot hosts (so the majority has fallen on the

EA). Aggregate costs for the 25 pilot hosts are around £200K per quarter (approx

£8K per catchment) or £800K for the year (£32K per catchment). About three-

quarters of this is associated with staff time (coordination, meetings, data

management, reporting, etc), with the remainder split fairly evenly between travel,

sub-contracts, consumables and equipment. (Note that these are costs for hosts only

and that they generally only cover the process of developing the plan. Spending by

non-hosts is not included here – see participant survey report – and spending on

delivery of measures has generally not yet been incurred).

The costs associated with staff time for pilot hosts are shown in Figure 7.1.



Figure 7.1: Staff costs for the first year (hosts only)

Whilst the amount of time spent is generally greatest in the largest catchments, this

is not always the case. Figure 7.2 compares the amount of time spent by catchment

and local level pilots. Whilst the larger, catchment level pilots tend to have spent

more time, a significant minority of the smaller, local level pilot lead organisations

have spent more than 220 days (equivalent to one full-time employee).

Figure 7.2: Staff costs for the first year (hosts only) by catchment size

Other (non-staff) costs incurred by pilot hosts over the year are shown in Figure 7.3.

0% 50% 100%

Less than 40 days

40 to 80 days

80 to 220 days

More than 220 days

Percentage of respondants

0% 50% 100%

Less than 40 days

40 to 80 days

80 to 220 days

More than 220 days


Local level

Catchment level



Figure 7.3: Non-staff costs for the first year (hosts only)

Surprisingly, the amount spent on non-staff costs does not appear to depend on the

size of the catchment. Indeed, the same percentage of small and large catchments

claim to have spent more than £4,000. This is shown in Figure 7.4.

Figure 7.4: Non-staff costs for the first year (hosts only) by catchment size

The majority (around two-thirds) of pilots feel they have not been able to quantify all

costs. This is shown in Figure 7.5. The main reasons for this are:

Staff/volunteer time is not always recorded or attributed to the pilot; and

Coordinators tend to complete budgeting at year end (i.e. March 2013).

0% 50% 100%

Less than £400

£400 to £1000

£1000 to £4000

More than £4000


0% 50% 100%

Less than £400

£400 to £1000

£1000 to £4000

More than £4000


Local level

Catchment level




7.3 Future costs Most pilots expect the costs of the pilots to rise in future. Whilst staff costs are

expected to remain broadly the same, implementation and delivery costs will be

incurred and these could be substantial. There is considerable uncertainty and a

large range around these costs, but some pilots think these will be in range £10-50K,

whilst others estimate hundreds of thousands or potentially millions of pounds. Most

pilots expect delivery costs to be better understood later in 2013, and most indicated

strongly that they could do more (e.g. engagement, publicity, delivery) if funding was

available and secure.



Appendix A: Fourth Quarterly Review and Final Review Form See separate file