Issues and strategies in ex-post evaluation of intervention against animal disease outbreaks and

spreadsMohamadou Fadiga & Hikuepi Katjiuongua

Mainstreaming Livestock Value Chain Conference.5-6 Nov. 2013. Accra, Ghana

2

Impacts of animal diseases

• Animal disease outbreaks: can be devastating

Direct costs- Death of animals- Lower productivity – slow growth, reduced efficiency of input use- Control costs

Indirect costs- Reduced access to markets- Long term macroeconomic effects- Effects of price changes on supply chain actors- Spillover effects such as effects on tourism

3

Impacts of animal disease

• Highly pathogenic avian influenza: Nigeria (2006/2008) 1.5 million birds lost (747,000 culled)

• Foot and mouth disease: Botswana - trade ban led to $ 33 million (USD) losses at processing level alone

• Tick & tick borne diseases: undermine livestock productivity

• Impacts differ by production system, coping & risk management capacity of VC actors, state of veterinary service delivery

• Require costly interventions: culling, quarantine & movement restrictions, and vaccinations…

• Important to evaluate animal disease intervention

4

Issues in ex post evaluation of animal disease outbreaks

Four key issues in ex post economic assessment of intervention against animal disease outbreaksDefining the counterfactual scenario

Accounting for the losses (under counterfactual)

Handling data uncertainty and specificity of epidemiological data

Dealing with the issues of attribution

5

Issues in ex post evaluation of animal disease outbreaks

• Governments are generally risk averse with respect to animal diseases

• Design intervention interventions to minimize losses in expected social welfare

• or some expected damage function

• Measuring these losses accurately requires integrated epidemiology and economic models

( ) [1 ( )] ( ) i i D i i F i iEW p W p W r

( )i i iED DC IC

6

On the counterfactual scenario

• Probability of outbreak is a composite risk estimate– A product of or risk of introduction, risk of spread, and mortality risk

– Defining the counterfactual risk is akin to answering the question about what would have been the trajectory of the disease in the absence of intervention

– Use a combination of experimental, historical, participatory data on the disease, etc…

• Use total death relative to population at risk to derive mortality risk

• For risk of spread– Develop an SI (susceptible-infected)model

– Data on transmission rate, incubation period, infectious period, and lapse between depopulation and restocking are used to solve the differential equations that illustrate the SI model

7

On the counterfactual scenario

1

1

1

1

1 0 0 1

1 1 0 0

0 1 1 1 0

0 0 1 1 1

t t

t t

t t

t t

S S

C C

I I

R R

0 50 100 150 200 250 300 350 4000%

10%

20%

30%

40%

50%

Day

Prevalence

Endemic state

Unstable epidemic

Scenario Risk Estimates

Spread Mortality Spread Mortality Composite

Burn-out Low 0.13 0.01 0.0013

Burn-out High 0.13 0.02 0.0026

Endemic Low 0.27 0.01 0.0027

Endemic High 0.27 0.02 0.0054

8

Calculate expected welfare or expected damage without the intervention at various scenarios

Analyse the net effect of risk reduction on social welfare

Find the socially optimal level of risk that justifies intervention

Evaluate if eradication makes sense economically

0% 20% 40% 60% 80% 100%0

22

44

66

88

110

Net Social Welfare Gain over 2006-2010

Risk Reduction Level

US $ Million

Use of counterfactuals and example from HPAI in Nigeria

9

• Characteristics of the data

– Underlying uncertainty (multiple sources, measurement errors, etc...)– Disease transmission is stochastic– Not all susceptible subjects in contact with infectious ones would catch the

disease– Necessity to test the validity of results through sensitivity analysis

• Stochastic simulation addresses all these points at once

– Use the collected and/or derived data on spread and mortality to simulate their distribution

– Solve for the key output variables using random draws from the distribution of risk parameters

– Generate a distribution of the key output variables – Conduct a probabilistic sensitivity analysis on the key output variables

Stochastic approach

10

Simulated Risks of HPAI Examples of Stochastic Results

Welfare Losses

Disease Cost

Mean 147.44 144.97

St. Deviation 291.93 115.99

Lower 95% 121.85 134.80

Upper 95% 173.03 155.14

0.100247775141082

0.125391740845092

0.150535706549101

0.175679672253111

0.200823637957121

0.225967603661131

0.25111156936514

0.27625553506915

0.30139950077316

0.326543466477169

0.351687432181179

0.376831397885189

0.401975363589199

0.427119329293208

0.452263294997218

0%

3%

6%

9%

12%

15%

18%Simulated Risk of Spread

Risk of Spread

Fre

qu

en

cy

9.77465302472334E-05

0.00672770087994058

0.0133576552296339

0.0199876095793273

0.0266175639290206

0.033247518278714

0.0398774726284073

0.0465074269781007

0.053137381327794

0.0597673356774873

0.0663972900271807

0.073027244376874

0.0796571987265674

0.0862871530762607

0.092917107425954

0%8%

15%23%30%38%45%53%

Simulated Mortality Risk

Mortality Risk

Fre

qu

en

cy

0 500 1,000 1,500 2,0000.0

0.2

0.4

0.6

0.8

1.0

Sensitivity analysis on welfare losses

Social Welfare Effects

Prob

abili

ty

Stochastic simulation

11

• Ideal Scenario: randomization and use RCTs

• But scope for randomization nearly zero in case animal disease outbreak – responding to a crisis. Alternative approaches exist but depend on unit of analysis

• Other factors influence disease spread and the effectiveness of the intervention (e.g. feed, water availability, agent behavior etc.)

• These make attribution of the intervention in a probabilistic sense difficult

Issues of attribution in intervention

12

• Necessity to use qualitative analysis in addition to quantitative assessment targeting indicators through key informant interview about what could have happened in the absence of the disease

• Use participatory epidemiology to capture indigenous knowledge about the disease effects

• Pay attention to the timing of intervention and careful documentation of all inputs used in the intervention and their sources

• Use past information about losses that coincided with the disease outbreak to gain insights how the intervention may have potentially affected the disease dynamics

• Overall: With that one could plausibly attribute observed change in disease trajectory to the intervention that was carried out

Issues of attribution in intervention

13

• Customary to focus on total cost (both direct and indirect)

• But when calculating total cost it is important to focus on avoidable losses – can overestimate of cost savings

• In other words not all risk will be removed as a result of the intervention

• Doing that would yield more reasonable incremental benefits and incremental net benefits that accrued because of the intervention

Accounting for losses

14

Thank you