1

NOAA Testbeds and Proving Grounds and NGGPS

Paula Davidson TBPG Team Lead

Paula Davidson, Ph.D. July 15, 2015 Senior Science Advisor NGGPS Annual Meeting Science & Technology Integration College Park, MD NOAA/National Weather Service

2

Outline

• Background

• NOAA Testbeds and Proving Grounds (TBPG)

• State of the Science

• NGGPS TBPG Team

• Current Priorities, Status and Plans

• Related NGGPS efforts

• Longer-range Plans

3 Web portal http://www.testbeds.noaa.gov (thanks to Rich Lataitis & Barb deLuisi)

Current NOAA Testbeds and Proving Grounds (TBPG)

AWT CTB COMT DTC

GRPG HWT HMT JCSDA JHT OPG SWPT

NOAA Missions TBPG Advancement Focus

To understand and predict changes in climate, weather, oceans, and coasts All

To share that knowledge and information with others All. Public warnings & forecasts: OPG, AWT, CTB, COMT, HWT, HMT, JHT, SWPT

To conserve and manage coastal and marine ecosystems and resources COMT

4

Framework for Transition: NOAA and Partners

5

NOAA TBPG Transitions in 2014

110

10

19

27

50

4

Major TestsConducted

Trans. to Operations Recommended forTrans. to Operations

Advanced to Expt'lTesting

Decision pending ordeferred on advance

Rejected for furtherTesting

6

TBPG Role in NGGPS

NOAA TBPG facilitate phased NGGPS transition through

• Rigorous developmental testing, predeployment testing, operational readiness/suitability evaluation

• Consistent guidelines, based on best practices, for governance, function and execution

• Traditional focus on independent testing; increasing capabilities for coordinated and collaborative testing

• Testing NGGPS priorities: forecast service impacts for NGGPS components, coupled system prototypes

7

TBPG Testing Coordination for NGGPS

1. CODE testing: Global components DTC (new role) potentially TBA: COMT, CTB,

GRPG, JCSDA, SWPT

2. Regional & Storm-scale components:

HWT AWT (?on storm-scale convection?) ------------------------------------------------------------------------------------------------------------------------------- Forecaster and service impacts (including Post-processing and forecaster apps):

Service centers JHT HMT/WPC HWT AWT

Local offices OPG (for WFOs, RFCs)

8

Week 3-4 Forecasts

• Current gap in explicit forecast products despite widespread user demand; NGGPS to provide basis for high-impact forecast information

• Beginning baseline skill assessments in forecast guidance for significant large-scale events, e.g. hurricane, drought/floods, prolonged heat/cold

• Extending object-oriented evaluation, identification of signal from noise in coupled model/ensemble guidance for reliable, consistent forecast information

• TBPG testing forecast service impacts for NGGPS components, coupled system prototypes and feedbacks

9

Day 6-10 Forecasts

• Current NWS detailed forecasts only through Day 7; NGGPS advances in skill/reliability provide basis for extending to Day 10.

• Rapid drop in skill for forecasts of “sensible” weather

• Beginning evaluations of reliable skill in model guidance to be mined for potential useful information for decision support

• TBPG engaged in evaluating guidance and translation into consistent, actionable products & services

10

High-impact weather forecasts (Stormscale) for US

• Current NWS forecasts for high-impact weather including tornado outbreaks, flash floods, major aviation disruptions from thunderstorms do not yet contain the temporal and spatial detail needed to support optimal decision making; NGGPS to provide framework

• Lack of operational guidance from convection-allowing ensembles at high spatial/temporal resolutions

• Beginning to ingest high-resolution, rapidly updating observations from radar; satellites next

• TBPG engaged in evaluating guidance and translation into consistent, actionable products & services

11

NGGPS TBPG Team

Member Organization

Paula Davidson (Lead) NWS/STI

David Novak NWS/NCEP/WPC

Russell Schneider NWS/NCEP/SPC

Jin Huang NWS/NCEP/CPC

John Cortinas OAR/OWAQ

12

Priorities

• Establishing Verification Methods & Benchmark Skill (EBS)

• Testing Advanced Forecaster Tools and Applications (ATA); extracting useful, actionable information for decision support

Above apply to all NGGPS timescales, with TBPG focus initially on improving forecast information to address gaps in several critical service areas

• Global Scale and extended-range forecasts (Weeks 3-4 and Day 6-10)

• Storm-scale & high-resolution applications (high-impact)

13

Progress, Status and Plans: Programmatic

• High-level testing criteria and performance targets for testing, identifying critical gaps in metrics were developed for use in plans and Round I R2O FFO. Q4 FY14. Completed

• Testbeds-related projects: Reviewed, selected testbeds projects submitted for Round I FFO; enhanced support for testbed infrastructure/personnel for robust involvement of NOAA forecast expertise. Q2 FY15. Completed. Projects started: May 1, 2015.

• Refinement and enhancement of the testing infrastructure and testing approach by adjusting the basic structure shown (slide 7). Q4 FY15. In progress

14

Examples: Objective Testing Targets

• QPF: Current Day 2 threat score for 1" threshold attained at Day 3 by 2018

• Aviation: Improve the numerical prediction of instrument meteorological conditions for the NGGPS by 25% over the baseline skill (to be determined) of GFS upgrade of Q2FY15: T1534L4

• Severe Convection: Extend current severe weather forecast service skill by 24 h (Goal FY18; e.g. current Day 1 skill at Day 2; Day 3 skill at Day 4)

• Fire Weather: Extend current fire weather forecast service skill by 24h (Goal FY18; e.g. current Day 1 skill at Day 2; Day 3 skill at Day 4)

Note: Subjective testing target examples by service areas in backup

15

Round I Sponsored Projects: Testbeds-related activities

Global-scale and extended-range weather applications (Days 6-10 and Weeks 3-4)

• Exploitation of Ensemble Prediction System Information in support of Atlantic Tropical Cyclo-genesis Prediction (Thorncroft) EBS, ATA

• Application of a Hybrid Dynamical-Statistical Model for Week 3-4 Forecast of Atlantic/Pacific Tropical Storm and Hurricane Activities (Schemm) ATA

• An Investigation of the Skill of Week Two Extreme Temperature and Precipitation Forecasts at NCEP-WPC (Bosart) EBS

• Validation of Significant Weather Features and Processes in Operational Models Using a Cyclone Relative Approach (Colle) EBS

Storm-Scale and High-Resolution Applications

• Test and Evaluation of Rapid Post-Processing and Information Extraction from Large Convection 3hr Tornado Outlooks (Correia) ATA

• Data Mining of High-resolution Storm-scale Data Sets (Smith) ATA

• Information Extraction and Verification of Numerical Weather Prediction for Severe Weather Forecasting (Jirak) EBS, ATA

• Improvement of Convective/Severe Weather Prediction through an Integrative Analysis of WRF Simulations and NEXRAD/GOES Observations over the CONUS (Dong) EBS

Cross-cutting

• Incorporation of near real-time Suomi NPP Green Vegetation Fraction and Land Surface Temp data into the NCEP Land modeling suite (Csizar) ATA, EBS

16

Summary: TBPG NGGPS Activities

Focus Area Projects Metrics

Week 3-4 Global-scale and extended range weather (Thorncroft, Schemm, Csizar)

GAPS; project deliverables

Day 6-10 Extended range weather (Thorncroft, Bosart, Colle, Csizar)

GAPS; Service targets: extreme precip events and winter weather; project deliverables

Day 0-3 high impact weather

Storm-scale and high-resolution applications (Correia, Smith, Jirak, Dong, Csizar))

-QPF Threat score, -Aviation IFR conditions accuracy -Severe wx lead time -Fire weather lead time GAPS

17

Related NGGPS Efforts

• NOAA TBPG facilitate testing of NOAA pre-operational service improvements: reaching out to partners across NGGPS teams

• Cross-cutting efforts in progress to improve land surface modules for NGGPS through assimilation of new satellite surface observations (Csizar project)

• TBPG efforts to increase NGGPS collaboration will focus first on NGGPS teams for evaluation methodology (Stajner, lead), nesting (Tallapragada, lead), post-processing (Hamill, lead)

18

Longer-Range Plans

• Increasing partnerships with related activities (among TBPG, across NGGPS, other e.g. USWRP)

• Expand testing activities to include additional NWS service areas impacted by NGGPS

• Expand involvement to include OPG in assessing service impacts (WFOs/RFCs)

• Refining and testing overarching performance targets for critical NWS service areas; address gaps

• Continuing phased testing of advanced forecaster tools and applications: rigorous and conclusive evaluation of accuracy, reliability, utility and services impacts in providing actionable decision support

19

Backup

20

Examples: Subjective Testing Targets I

Precipitation

• QPF Service Goal: Improving extreme event prediction from Day 0 to Day 10. For inclusion in AO: This entails improvements in warm season mesoscale prediction in the short range, as well as extending skill out in time (day 6, 7, ...10)

• QPF Effective, novel ways to synthesize information for the forecaster to communicate forecast impacts.

• Sample Performance Measure: Decision managers using week 2 QPF information for planning.

Winter Weather

• Examine alternative objective skill goals, based on analysis of reforecast and soon-to-be available gridded snowfall analysis, and develop recommended forecast skill target, Q4 FY15

• Achieve Winter Storm Warning lead time of 24 hours while maintaining POD of 0.90 by 2018. This lead time is 4 hours longer than the current GPRA goal. For inclusion in AO: This entails improved prediction of explosive cyclogenesis, improved snowfall algorithms, extending skill out in time (day 6-10), and finally novel ways to synthesize information for the forecaster.

Aviation weather

• Advance the spatial and temporal consistency of NWS products and decision support services provided to the aviation community based on increasing global model resolution (e.g. 13km, T1534L64 projected in Q4FY14) and future higher-resolution windows. A mesoscale global model with higher-resolution windows is a major step toward meeting NTSB Safety Recommendation A-14-17 on achieving consistency across NWS aviation and non-aviation specific statements, forecasts, advisories, and warnings that provide information on hazards important to aviation safety such as low clouds, ceilings, visibility, wind, convection, and precipitation.

• Improve the basis for decision support services re instrument meteorological conditions, with numerical prediction guidance of clouds and visibility, and particularly hydrometeors

21

Examples: Subjective Testing Targets II

Severe weather

• Severe Convection: Develop effective objective measures of severe weather forecast skill appropriate for both initial NGGPS (evolved GFS) and future convection-allowing nests through work with forecast specialists and numerical modelers. Baseline current global forecast system and Storm Prediction Center objective severe weather forecast skill, and refine objective goals for NGGPS project and associated Hazardous Weather Testbed testing (Goal: 4QFY15).

• Severe Convection: Improve Day 2-10 national severe weather and convection sensitive services exploiting NGGPS forecasts of severe storm supporting mesoscale environments, boundary layer thermodynamic structures, and convective precipitation (objective metrics established in FY15).

• Severe Convection: Develop specialized NGGPS information extraction to support severe weather service improvement goals, and approaches for effective communication and use of improved NGGPS predictive skill by NWS forecasters and community decision makers

Fire weather

• Develop effective objective measures of fire weather forecast skill appropriate for both initial NGGPS (evolved GFS) and future convection-allowing nests through work with forecast specialists and numerical modelers. Baseline current global forecast system and Storm Prediction Center objective fire weather forecast skill, and refine objective goals for NGGPS project and associated Hazardous Weather Testbed testing (Goal: 4QFY15).

• Improve Day 2-10 national fire weather services exploiting NGGPS skill advances for fuel conditions, large-scale fire weather conditions, and implicit or explicit forecasts of convection and potential new fire starts by associated lightning (objective metrics in FY15).

• Develop specialized NGGPS information extraction to support fire weather service improvement goals, and effective use of NGGPS predictive skill by NWS forecasters and fire weather decision makers.

Tropical weather

• Improve tropical cyclone genesis, track, intensity, and size prediction guidance out through seven days.

• Improve probabilistic wind and storm surge guidance out through seven and three days, respectively.

22

General TBPG Background

• Recent TBPG Testing: FY14 Examples

• FY15 Activities

• TBPG Workshops, Guidelines, Coordinating Committee, Performance Measures

23

Test: Evaluated event-driven terminal and en route air traffic impact decision support methods

• How it was tested: With partners**, developed and tested automated convective forecast guidance and associated end-to-end Meteorologist-Over-The-Loop (MOTL) decision support service guidance based on NWS SREF and convective-allowing model (HRRR, HWRF) data

• What was demonstrated: Performance of automated guidance; effectiveness of MOTL & Collaborative Aviation Weather Statement (CAWS); operational readiness criteria for CAWS

Impact: Major advance in forecasters’ impact-based decision support, bridging FAA and industry’s strategic-to-tactical decision making at 0-4 hour lead time; reductions to weather-related aviation delays

AWT Example: Building a Weather-Ready Airspace

Automated Convective forecast guidance (CCFP)

Collaborative Aviation Weather Statement Impact-based decision support guidance Scheduled operational deployment: 3 Mar 2015

Evaluation: 2014 warm season Partners: FAA, OAR, industry

** Partners: FAA, OAR/GSD, industry

24

Test: MAPP/CTB test of North American multi-model ensemble (NMME) prediction system of major climate models in US and Canada for NCEP operational seasonal forecasts.

• How it was tested: Evaluated skill (model vs. obs) from 30-year hindcasts; daily real-time ensemble forecasts since Aug. 2011.

• What was demonstrated: Improved forecast reliability and forecast skill, e.g. anomaly correlation and rank probability score.

Impact:

• Improved numerical guidance for NCEP/CPC operational seasonal forecasts

• The most comprehensive seasonal prediction dataset available to the public for research and applications.

CTB Example: North American Multi-Model Ensemble

Comparison of NMME and CFSv2 skills based on 30-year hindcasts. July 1 start DJF SST forecast; Ranked Probability Skill Score

NMME

CFSv2

25

Test: Evaluated multiple, coupled wave/surge/inundation models against observations

• How it was tested: Compiled observational data near USVI/PR from CARICOOS and evaluated models

• What was demonstrated: Testing in progress, with all data compiled and model comparisons ~50% complete

Impact:

• Extend present wave/surge forecasting capability to steep-sloped areas such as Caribbean and Pacific islands

• Skillful guidance for forecasters at NHC and WFO San Juan improves decision support high-risk coastal areas

Computed wind field (vectors) and surface water elevation during Hurricane Georges.

COMT Example: PR/USVI Surge & Wave Inundation Model

PR

VI

26

Test: GOES-R Super Rapid Scan 1-min imaging demonstration

• How it was tested: GOES-14 on-orbit spare disseminated real-time 1-min imagery to NWS forecasters for evaluation, May 8-24, August 14-28., 2014. FY15 testing planned.

• What was demonstrated: Rapid dissemination feasibility; subjective benefits to forecaster: Increased situational awareness, forecast confidence, lead time for high-impact weather decision support

Impact:

• GOES-14 1- min imagery provided unique data, trial run for future ABI/GOES-R capabilities

• Increased lead time for thunderstorms: 1-min imagery of cumulous cloud growth helps forecasters identify storms earlier than with current 5-min updates

• Data sets collected for training forecasters

GRPG Example: Super Rapid Scan Imaging

Animation: GOES-14 mesoscale imagery of developing severe storm. Forecasters can monitor dynamic interactions between air masses, outflow boundaries, and related storm features.

DIA Tornadic Storm May 21, 2014

27

Test: Trial of extended winter weather prediction to identify/evaluate skillful forecast guidance candidates, e.g. Day 4-7 probability of >.1” of frozen precipitation • How it was tested: Forecasters subjectively

evaluated probabilities of frozen precipitation and QPF >.1”, extracted from global ensembles: EC and GEFS

• What was demonstrated: – Reliably skillful probabilistic guidance for Day 4 helpful

for decision support; predictability falls rapidly at Day 6 & 7

– Model forecast uncertainty is underestimated; further testing to include additional Canadian ensemble

– More QPF thresholds needed for complete storm guidance

Impact: New winter weather probabilistic forecast products for Days 4-7 extend the range of reliable guidance to aid decision support and mitigate potential societal impacts

HMT Example: Day 4-7 Probabilistic Winter Weather Guidance

Subjective ratings of ensemble guidance from 2014 Winter Weather Experiment

Example: WPC Day 4-7 Winter Weather Probabilities

28

HWT Example: A Forecaster in a FACETs World

Test: In test of concepts for Forecasting a Continuum of Environmental Tests (FACETS), evaluated Probabilistic Hazard Information (PHI) Tool for forecasts of wind, hail, tornadoes

• How it was tested: Forecasters evaluated warning/ forecast decision scenarios for bow echo events, for stability, accuracy, usability, workload impacts, decision support

• What was demonstrated:

– PHI tool effective for building gridded probabilistic forecasts; helps convey tornado probabilities (low, non-zero) in comma head region (circled)

– Forecaster confidence increased with providing more stable, realistic information in probabilistic vs. deterministic form

– Tornado risk communicated better and earlier to decision makers via probabilistic vs. current deterministic approach

– More case studies needed to complete objective evaluation

Impact: Forecasters communicating more timely, probabilistic, information on severe weather to decision makers allows earlier, broader protective and response actions

Bow echo event; comma head region circled

29

Test: Operational Readiness Evaluation of NASA SPoRT/MDL Tracking Meteogram (TM)

• How it was tested: NWS forecasters used prototype in warning/forecast decision scenarios, with both archived & live data, to subjectively evaluate stability, usability, workload impacts, and value for decision support

• What was demonstrated: TM adds value for:

– Tracking and displaying meteorological trends (e.g., mesocyclones, total lightning)

– Interrogating gridded model output

– Mesoanalysis and post-event analysis

– Extracting critical decision support information

Impact: Scheduled TM implementation into WFO AWIPS baseline, Sept 2015, will improve forecasters’ ability to spotlight critical features for decision support

OPG Example: Tracking Meteogram

Evaluation Session, NWS Operations and Services Proving Ground (OPG)

30

Test: Select geospace model to forecast regional impacts of geomagnetic storms to the electric power industry. • How it was tested: NASA Community Coordinated Modeling Center conducted test/evaluation of 5 research geospace models,

independently developed in academia, of Earth’s magnetosphere against observed localized variations in the geomagnetic field near electric power lines

• What was demonstrated: Superior performance and operational-readiness shown by the University of Michigan model (SWMF).

Impact: Once operational (FY16), the model provides 20-40 min forecasts of the regional impact of geomagnetic storms, with enough lead time for electric power industry to mitigate impacts (e.g. transfer load)

SWPT Example: Selecting a Geospace Model for Transition to Operations

Model of Earth’s Magnetosphere

31

FY15 Activities

Combined Special Testing Opportunities: NGGPS initiative**

Multi-year effort to develop NOAA’s next-generation operational model for

global environmental prediction

Foundation for NWP; from storm-scale out to space weather domain

FFO for Round I Projects included TGPG-related activities; 9 awards initiated

May, 2015

Planning for next cycle NGGPS

Coordinating USWRP-funded work (JHT, HWT, HMT)

TBPG Workshop: Boulder, April 14-16 , 2015

Science Theme: Advances in NOAA’s Environmental Intelligence.

Planning for Arctic Testbed

Annual reporting, Monthly telcons/TBPG updates

32

TBPG Annual Workshops

Highlight/promote integrated testing opportunities in NOAA TBPG

Overviews from current/emerging TBPG

Integrating Science Themes and Best Paper Awards:

3rd Annual TBPG Workshop: Intense Precipitation Events (2012)

4rd Annual TBPG Workshop: High-impact Environmental Events (2013)

5th Annual TBPG Workshop: Advances in Environmental Prediction (2014)

6th Annual TBPG Workshop: Advances in Environmental Intelligence (2015)

Best paper (Science-theme) competition

Criteria - excellence, relevance, presentation

33

TBPG Coordinating Committee

Comprised of TBPG managers and LOTM-designated LO focal points:

Co-chairs: Paula Davidson (NWS LO focal point) Mike Uhart (OAR LO focal point)

Initial meeting (NWS and OAR members): Seattle AMS, Jan 2011

Activities:

Monthly virtual meetings featuring TB/PG recent tests/results

Coordination/Outreach: revamped TB/PG coordination website www.testbeds.noaa.gov,

coordinated SEE inputs, one-on-one tutorials on guidelines, testing protocols

Organizes annual workshops on NOAA testbeds/proving grounds

Adopted recommended approach to metrics for recognizing progress, Fall 2012:

Relevance, quality, and effectiveness/efficiency

Accomplishments and performance metrics provided in annual TBPG progress reports

Summary publication (Ralph, Intrieri, et al.): The emergence of weather-focused testbeds

linking research and forecasting operations. BAMS, 94, 1187-1210, 2013

Coordinating announcements of opportunity for 2014 testing

34

Guidelines: TBPG Roles and Responsibilities

NOAA participants

Host facilities:

Develop and maintain Charter and/or Terms of Reference (see governance)

Establish and lead management team, to oversee, support and facilitate testing operations (see function/execution)

Lead management team, establish executive oversight committee

Participate in NOAA-wide coordinating TB/PG coordinating committee

Research partners (outside host facility):

Participate in peer-review and provide testing support

Operations partners (outside host facility):

Provide statement of needs/requirements and testing support

External participants

Respond to announcements of opportunities for testing advanced S&T to support

NOAA’s operational mission requirements

Participate in testing and evaluation

35

Guidelines: Testbed Functions

Testbeds/testbed personnel, under local testbed management:

Conduct controlled testing of peer-reviewed capabilities to determine if

they can work with operational systems

Provide announcements of opportunity for testing

Prioritize tests through peer review recommendations, subject to oversight

Assist/facilitate testing

Report plans and results at least annually (Management Team)

36

Guidelines: Proving Ground Functions

Proving grounds personnel/local management function similarly to testbeds

Conduct controlled real-time testing of capabilities proven to work with operational

systems Assess workflow, workload options and impacts; including collaborative operations

Determine operational readiness

Provide announcements of opportunity that identify additional criteria for Proving Ground consideration: e.g. having passed testbed/developmental testing, and demonstrated impact on meeting operational requirements

Prioritize testing

Assist/facilitate testing and participate in review/approval processes for implementing into operations

Report plans and results at least annually (management)

37

Guidelines: Governance

Major aspects

Local management teams: conduct/support testing operations, report results

Executive committees/boards: apply strategic and funding considerations in oversight/review of activities, selection of tests, and quality of results

Coordinating Committee of TB/PG managers and LO focal points: facilitate communication and to provide coordination and consistency among TB/PG

Charters/Terms of Reference: Each TB/PG should develop and maintain

Outlines scope, operations, and governance, including general procedures,

infrastructure requirements and availability of staff and other testing support

Authority for charters should be the executive oversight committee, or its designee

38

Governance

. NOAA Research

Council

LO Transition

Managers

Testbed 1

Exec Steering

Committee

Proving Ground

1

Exec Steering

Committee

Testbed 2

Exec Steering

Committee

TPGCC

Other Councils

NOAA NEP/NEC

39

Working Definitions: Testbed

Testbed:

• A working relationship for developmental testing, in a quasi-operational framework among researchers and operational scientists/experts (such as measurement specialists, forecasters, IT specialists) including partners in academia, the private sector and government agencies, aimed at solving operational problems or enhancing operations, in the context of user needs.

• A successful testbed involves physical assets as well as substantial commitments and partnerships.

What is tested?

• Advances to be considered include peer-reviewed candidates for more effective observing systems, better use of data in forecasts, improved forecast models, and applications for improved services and information with demonstrated economic/public safety benefits.

40

Working Definitions: Proving Ground

Operations and Services Proving Ground:

• A framework for NOAA/NWS to conduct testing of advanced operations, services and science and technology capabilities that address the needs of both internal and external users. Successful testing demonstrates readiness to implement into operations.

What is tested?

• Capabilities that have already passed developmental testing. Such capabilities include advanced observing systems, better use of data in forecasts, improved forecast models, and applications for improved services and information with demonstrated economic/public safety benefits.

41

Guidelines Development

NOAA LOTMs approved the guidelines prepared by an ad hoc committee: NWS: Paula Davidson, Jason Tuell, Louis Uccellini OAR: John Gaynor, Steve Koch, Roger Pierce, Marty Ralph

Followed guiding principles: Policy Context:

o NOAA and NWS Strategic Plans, NOAA Research Plans, S&T and Services Roadmaps, NOAA research/laboratory reviews, NOAA science review policy

o Complement activities in existing transition processes

Roles and responsibilities; Function and execution o Incorporate best practices from current TB/PG charters and terms of reference

Governance:

o Promote consistency, but not uniformity, among testbeds and proving grounds

o Involve appropriate stakeholders o Propose formal coordination among /across facilities

42

Coordinating Committee

Responsibilities

Facilitates communication coordination and consistency among TB/PG

Assists in programmatic evolution of TB/PGs

Reports overall progress, success, issues to Line Office Transition Managers (annually)

Links to NOAA budget planning and execution processes to ensure TB/PG are included; advocates for resources

Educates/advocates on TB/PG within and external to NOAA

Membership

Each TB/PG manager (or designee)

Line Office focal points for TB/PG, appointed by AAs or LOTMs

43

TBPG Performance Measures

Outcome of Spring 2012 Workshop: TBPG should take

more credit for progress

Development:

Working group developed whitepaper on TBPG Performance

Measures (based on NOAA guidance documents, 5/12- 8/12)

Performance Measures adopted by TBPGCC, 9/12

LOTMs (TBPGCC Oversight body) approved on 10/12

Recommendations:

Each facility should tailor performance measures from generalized

language, in areas of relevance, quality, effectiveness/efficiency

Guidance for annual TBPG reports includes documenting testing

activities, summary results, performance metrics