Upload
devi
View
24
Download
0
Tags:
Embed Size (px)
DESCRIPTION
IHOP_2002 DATA MANAGEMENT UPDATE Steve Williams UCAR/Joint Office for Science Support (JOSS) Boulder, Colorado 2 nd International IHOP_2002 Science Workshop Toulouse, France 14-17 June 2004. IHOP_2002 Field Catalog Statistics. http://www.joss.ucar.edu/ihop/catalog/. - PowerPoint PPT Presentation
Citation preview
IHOP_2002 DATA MANAGEMENT UPDATE
Steve WilliamsUCAR/Joint Office for Science Support (JOSS)
Boulder, Colorado
2nd International IHOP_2002 Science Workshop
Toulouse, France 14-17 June 2004
IHOP_2002 Field Catalog Statistics
• Status Reports and Operations Summaries(600+ documents)
• Mission Related Reports and Products(2700+ documents)
• Near Real-Time Operational Image Products•Satellite, Surface, Upper Air, Forecast, and Model
•Nearly 200,000 files/images using 20 gigabytes
• Report Authoring and Submission Tools
• Product Ingest Tools: upload forms, email and ftp
submissions, automated web downloading
• Mirror Site in Boulder, CO, with near real-time synchronization
•Links to other project related real-time data sources
http://www.joss.ucar.edu/ihop/catalog/
Composites: What and Why
A composite dataset is a collection (over some time period and region) of similar data (e.g. surface met) from a variety of sources, put into a common format, and passed through a uniform quality control.
Why does UCAR/JOSS develop composites?
- Provides data in a uniform format with QC.
- Allows determination of network/site problems.
- Useful for model applications.
- Prevents duplication of effort.
Composite Data Sets Developed by UCAR/JOSS for IHOP-2002
• Surface Meteorological - 1 min, 5 min, and Hourly
• Precipitation - 15 min, Hourly, and Daily
• Soundings - High Resolution and 5 hPa
• Complete Station List for all Composites
1-min Surface Meteorological Data Composite (65 stations)
ASOS (41)
ARM SMOS (14)
ABLE AWS (5)
NCAR Supp. (5)
Oklahoma Mesonet (115)
NCAR ISS/ISFF (10)
West Texas Mesonet (34)
1-min sites (65)
5-min Surface Meteorological Data Composite (224 stations)
Hourly Surface Meteorological Data Composite (743 stations)
CoAgMet (*) (23)
AWOS (*) (110)
RAWS (+) (10)
SCAN (o) (6)
GWMD (*) (18)
Ameriflux (+) (3)
HPCN (o) (72)
Texas ET (o) (19)
KVII-TV (+) (4)
MADIS (o) (246)
NMSU (*) (7)
5-min sites (o) (225)
15-minute Precipitation Composite
NCDC Coop (429)
5-min surface (224)
Hourly Precipitation Composite
NCDC Coop (509)
NCEP hourly (1155)
Hourly Met (743)
Daily Precipitation Composite
NCDC Coop (1581)
Hourly Met (743)
NCEP Daily (1611)
ASOS
AWOS
ARM
Others (7)
OKMESO
ABLE
SCAN
LTER
HPCN
WTXMESO
NCAR (4)
MADIS
Ameriflux
KVII
GWMD
NMSU
CVT
HRLY
HRLY
HRLY
HRLY
HRLY
HRLY
HRLY
Visual
Visual
Visual
Visual
Visual
Visual
Visual
Visual
Visual
Visual
Visual
Visual
Visual
Visual
Visual
Visual
MERGE
Gross LimitChecks
HorizontalQC
Visual
IHOP_2002 Hourly Surface Composite
IHOP_2002 Hourly Surface Composite Development
ExamineStatistics
CVT
CVT
CVT
CVT
CVT
CVT
CVT
CVT
CVT
CVT
CVT
CVT
CVT
CVT
CVT
Gross Limit Checking Procedures
• Verify file format.
• Data values reasonable for the Earth.
• Negative wind speed or precipitation values.
• Wind direction between 0o and 360o.
• Dew point less than or equal to air temperature.
• Precipitation values gross limit checked based on temporal resolution of data (i.e. allow
higher precipitation rates at higher temporal resolutions).
Horizontal QC Statistical Summary
• Overall statistics for all data included in a composite.
• Summary statistics for entire networks.
• Summaries of the 10 worst performing stations for each network and each parameter.
• Summary statistics for each individual station.
• Summary statistics for each day.
• Summary statistics for each hour.
• Summary plots of variance values by parameter.
• All occurrences of zero variance noted.
What Can Lead to Surface Data Being Flagged?
• Instrument failure, bias, drift, random errors• Incorrect latitude, longitude, or elevation• Differences in siting or height of instruments• Misidentified units, dates, or stations• Terrain impacts• Data problems at surrounding stations• Unique meteorological phenomena
Example of Applied HQC Data Flags
Severe Problems Detected
Major Biases Detected
Subtle Problems Detected
Embedded Accumulative Precipitation
High Resolution and 5 hPa Radiosonde Composites
Fixed Sites: Mobile Sites:
NWS (14) ARM (5) ISS (1) Falcon Drops (86) Learjet (330) NCAR GLASS (146) NSSL CLASS
(80)
UCAR/JOSS Atmospheric Sounding Processing Procedures
Observing System Output e.g. NWS Micro-ART
Convert to JOSS Quality-control Calculate Derived
Format (ASCII) Parameters
Vertical Consistency Checks
QC Flags
Gross Limit Checks
Automated Checks QC Flags
e.g., u, v, dz/dt
Visual Examination
Error Files
High-resolution and/or Interpolated Vertical Composite e.g., 5 hPa
Inventory, Archive, and Develop Metadata
Data and Metadata Available Online
IHOP DATA WORK IN PROGRESS…..
• Transfer model output files from NOAA/FSL (e.g. new LAPS and WRF model runs)
• Transfer LEANDRE Lidar data and imagery files
• Transfer new Boundary Layer data/products
• Obtain OK Mesonet OASIS data and documentation
• Link to or transfer research radar data and imagery (PSU/NSSL)
IHOP DATA MANAGEMENT – http://www.joss.ucar.edu/ihop/dm/
Questions – contact Steve Williams ([email protected])
• Transfer additional PI data/products
• Complete IHOP Instrument Intercomparison Tables
FUTURE IHOP DATA WORK?
• Produce CD ROM(s)?
• Development of IHOP GIS datasets?
• Design, Develop, and publish IHOP 4-D Water vapor data set?
• Other?
IHOP DATA MANAGEMENT – http://www.joss.ucar.edu/ihop/dm/
Questions – contact Steve Williams ([email protected])