Improving Cost Effective Air Improving Cost Effective Air Quality ForecastingQuality Forecasting

UPDATE 10/02/2008:UPDATE 10/02/2008:

•New Categorical Metrics (Kang 2007)New Categorical Metrics (Kang 2007)–Interpretation of new metricsInterpretation of new metrics–Use in AQ model evaluationUse in AQ model evaluation–Direct applications to cost-effective AQ Direct applications to cost-effective AQ forecastingforecasting

ReviewReview

• Project GoalsProject Goals– Minimize money spent due to incorrect forecastsMinimize money spent due to incorrect forecasts

• MD/VA Free ride programMD/VA Free ride program– VA cost per day VA cost per day $120,000 $120,000– MDOT cost per day MDOT cost per day $35,000 - $40,000 $35,000 - $40,000

• National Air Quality Forecasting System National Air Quality Forecasting System (NAQFS) Updates(NAQFS) Updates– Domain, Surface parameters, canopy uptakeDomain, Surface parameters, canopy uptake

• Methods of Model EvaluationMethods of Model Evaluation– Discrete (RMSE, N/MB, N/ME, r)Discrete (RMSE, N/MB, N/ME, r)– Categorical (A, B, FAR, CSI)Categorical (A, B, FAR, CSI)– New Categorical (WSI, aH, aFAR) New Categorical (WSI, aH, aFAR) Kang 2007) Kang 2007)

Traditional Categorical MetricsTraditional Categorical Metrics

• Categorical EvaluationCategorical Evaluation– Ozone threshold exceeded?Ozone threshold exceeded?– Was it forecasted to exceed?Was it forecasted to exceed?

Forecast Forecast

ThresholdThreshold

a a MISSMISS HIGH HIGH

b b HITHIT HIGH HIGH

c c HITHIT LOW LOW

d d MISSMISS LOW LOW

Traditional Categorical MetricsTraditional Categorical Metrics

New Categorical Metrics (Kang 2007)New Categorical Metrics (Kang 2007)

• WSI WSI Weighted Weighted Severity IndexSeverity Index– Like CSI, only it Like CSI, only it

includes (to a includes (to a degree) near hitsdegree) near hits

New Categorical Metrics (Kang 2007)New Categorical Metrics (Kang 2007)

• Modeled vs. Observed O3Modeled vs. Observed O3• Area Hits (aH) and aFAR (Area False Alarm Rate) Area Hits (aH) and aFAR (Area False Alarm Rate)

account for spatial near-hitsaccount for spatial near-hits

New Categorical Metrics (Kang 2007)New Categorical Metrics (Kang 2007)

• Use 3x3 or 5x5 grid spaces to account for areaUse 3x3 or 5x5 grid spaces to account for area• Adjust the Hit forecast (b/b+d) to include the area Adjust the Hit forecast (b/b+d) to include the area

and not just the point…Same for FAR.and not just the point…Same for FAR.

Application to cost-effective AQ Application to cost-effective AQ forecastingforecasting

• Area forecasts are typical in operational Area forecasts are typical in operational

forecastsforecasts

– Warn an area, not a pointWarn an area, not a point

– aH and aFAR account for these!aH and aFAR account for these!

• Assign a percent chance of missed Assign a percent chance of missed

forecast using WSI?forecast using WSI?

• Decide minimum forecast value that Decide minimum forecast value that

statistically produces the best likelihood of statistically produces the best likelihood of

an exceedance?an exceedance?