11

Archiving Requirements – Archiving Requirements – Current Requirements Current Requirements

A. Juliann MeyerA. Juliann MeyerSr. Hydrologist – Data SystemsSr. Hydrologist – Data Systems

Missouri Basin River Forecast CenterMissouri Basin River Forecast Centerandand

RAXUM Team LeaderRAXUM Team Leader

RFC Verification Workshop, 08/15/2007

22

What Will be CoveredWhat Will be Covered

• How to archive forecasts and observations for 4 different variables, these are: precipitation, temperature, height and discharge.

• How to archive forecasts from different scenarios, such as forecasts using actual QPF vs. forecasts using no QPF (zero precip), or actual forecasts vs. forecasts from raw model (without any MODs).

• RFC Archive DB quality control capabilities

33

NWS Directive 10-911NWS Directive 10-911

• Section 4.7 covers verification requirement

• Section 4.8 covers archiving requirement

This directive can be found at

http://www.nws.noaa.gov/directives/sym/pd01009011curr.pdf

44

Flow Chart for SHEF DecodersFlow Chart for SHEF Decoders

55

ob7.2ob7.2

The verifcation software only looks for data in the following tables:

pecrsep and pephsep for observed data

pedfsep for forecast data

66

Everything from this point Everything from this point on is about ob8.2on is about ob8.2

77

Where Data Is StoredWhere Data Is Stored

88

““oops” table, peoosepoops” table, peoosep

• Verification software does try to use this table when looking for observed data that does not fit into the other SHEF data value tables.

• This is the table the raw shefdecoder post SHEF Type code “R” data to that has duration and/or extremum codes that don’t fit the other raw data value tables.

• It is also where the raw shefdecoder post data with a SHEF Type codes of “H”, “M” and “P”.

99

Distinguishing Between ForecastsDistinguishing Between Forecasts

• Use SHEF, primarily the Type/Source Codes

• The latest SHEF manual can be found at http://www.nws.noaa.gov/directives/sym/pd01009044curr.pdf

1100

RFC Archive DB Quality Control RFC Archive DB Quality Control CapabilitiesCapabilities

• Gross Range & Reasonable Range Checks can be done at posting time, uses info in db tables datalimits and locdatalimits

• sensok table, entries in this table are used by other applications

• rivercrit table, if using level 1 processors, allows the user to specify addition range checks

1111

IVP and SHEF quality codesIVP and SHEF quality codes

• IVP uses the SHEF data value tables SHEF qualifier code fields during the pairing process.

• The observed data value's SHEF qualifier code is the first item checked when making a pair.

• IVP has a hardwired, ordered string that defines a quality code preference. The order is: G, M, P, V, S, Z, T, F, Q.

• IVP pairing will never use observed value that has SHEF qualifier code of B, E or R.

1122

Raw shefdecoder and adjustfactor Raw shefdecoder and adjustfactor TableTable

• Only the raw shefdecoder must apply the adjustfactor. • Forecast data (i.e. data for which the SHEF Type

Code is 'F') is not affected, but all other SHEF Types Codes are.

AFTER posting a value to the database, the decoder checks to see if the apps_defaults token adb_shef_raw_add_adjust is not set to OFF.  If it is not OFF, then the decoder will search the adjustfactor table rows for one to use.  If it finds one (i.e. matching the primary key and the date associated with the adjust precedes the obstime of the value to post), then it will find the best one (the date precedes and is closest to the obstime) and apply the adjust.  It will then post the adjusted record with a SHEF Source Code of 9 (i.e. s = '9').

1133

Raw decoder and adjustfactor Raw decoder and adjustfactor Table Table continuedcontinued

1. Create an appropriate record in the adjustfactor table.

2. Add an appropriate record to the ingestfilter table, where everything is the same as the existing applicable record, but the source is '9'.

3. Set the token adb_shef_raw_add_adjust to 'on‘ and stop and restart the raw shefdecoder. Need only do this step if the adjustfactor table has not previously been used.

In order for the adjusted data to make it into the same table as the original data, an appropriate record must be added to the ingestfilter table with a source of '9'. In summary, the user must do the following:

1144

…And that’s all there is!!!

1155

SHEF Code BasicsSHEF Code BasicsPhysical Element CodesPhysical Element Codes

HG – height, river stage

HT – elevation, project tail water stage

HP – elevation, pool

PP – precipitation

TA – air temperature

QT – discharge, computed total project outflow

QI – discharge, inflow

For full list, see SHEF Manual, Table 1.

1166

SHEF Code BasicsSHEF Code BasicsDuration CodesDuration Codes

I – instantaneous (0000)

D – 1 day (2001)

H – 1 hour (1001)

Q – 6 hour (1006)

For full list, see SHEF Manual, Table 3.

1177

SHEF Code BasicsSHEF Code BasicsType CodesType Codes

C - Contingency

F – Forecast

P & 1 thru 9 – Processed

R – Reading (observed data)

For full list, see SHEF manual, Table 4.

1188

SHEF Code BasicsSHEF Code BasicsExtremum CodesExtremum Codes

Z – null character (filler)

X – Maximum of day

N – Minimum of day

D – Maximum of 1 hour

F – Minimum of 1 hour

R – Maximum of 6 hours

H – Minimum of 6 hours

For full list, see SHEF Manual, Table 5.

1199

SHEF Code BasicsSHEF Code BasicsData Qualifier CodesData Qualifier Codes

Used by IVP

G - Good, Manual Edit

M – Manual Edit

P – Passed level 2, 3

V – verified level 1, level 2

S – screened level 1

Z – No QCD Performed

T - triggered

F – flagged by sensor or telemetry

Q - questionable

Never Used by IVP

B – Bad, Manual QC

E – Estimated

R – rejected by level 1