Upload
alexia-horn
View
221
Download
0
Embed Size (px)
DESCRIPTION
Brussels, Belgium Nov. 2003CLIMAR II, I-COADS Data and Products 3 Background Three-way collaboration since 1981Three-way collaboration since 1981 Partner organizations and PI’sPartner organizations and PI’s –NOAA/NCDC, (Dick Reynolds, Joe Elms) –NOAA/CDC, (Scott Woodruff) –NSF/NCAR, (Steven Worley) U.S. Government and startU.S. Government and start Now, International-COADS (I-COADS)Now, International-COADS (I-COADS)
Citation preview
I-COADS Data and I-COADS Data and ProductsProducts
Steven J. WorleySteven J. WorleyScott D. WoodruffScott D. WoodruffRichard W. ReynoldsRichard W. Reynolds
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 22
TopicsTopics• BackgroundBackground• StatusStatus• PlansPlans
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 33
BackgroundBackground• Three-way collaboration since 1981Three-way collaboration since 1981• Partner organizations and PI’sPartner organizations and PI’s
– NOAA/NCDC, (Dick Reynolds, NOAA/NCDC, (Dick Reynolds, Joe ElmsJoe Elms) ) – NOAA/CDC, (Scott Woodruff)NOAA/CDC, (Scott Woodruff)– NSF/NCAR, (Steven Worley)NSF/NCAR, (Steven Worley)
• U.S. Government and Academic @ startU.S. Government and Academic @ start• Now, Now, InternationalInternational-COADS (I-COADS)-COADS (I-COADS)
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 44
Current Status, Release 2.0Current Status, Release 2.0I-COADS Release 2.0, 2003DM 1784-1997, RT 1998-2002
0
1
2
3
4
5
6
7
8
1790 1810 1830 1850 1870 1890 1910 1930 1950 1970 1990
Rep
orts
/Yea
r (M
illio
ns)
DM Archive 1784-1997
RT Archive 1998-2002
185 Million Observation Records
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 55
Current Status, Release 2.0Current Status, Release 2.0I-COADS Release 2.0, 2003DM 1784 -1950
0
0.2
0.4
0.6
0.8
1
1.2
1790 1800 1810 1820 1830 1840 1850 1860 1870 1880 1890 1900 1910 1920 1930 1940
Rep
orts
/Yea
r (M
illio
ns)
WWI and WWII
100K per year, 1854-1885
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 66
Data AvailabilityData Availability• ObservationsObservations
– NCARNCAR•1784 - 2002 binary formatted archive is 1784 - 2002 binary formatted archive is
onlineonline•ASCII format available by online requestASCII format available by online request
– Select regionSelect region– Select timeSelect time– Select variablesSelect variables– Select quality flags and optionsSelect quality flags and options– Delayed mode delivery via pull FTPDelayed mode delivery via pull FTP
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 77
Data Availability, Observations @ Data Availability, Observations @ NCARNCAR
http://dss.ucar.edu/pub/coads/forms/
27
43
0
10
20
30
40
50
60
70
1
I-COADS Observations
NCAR, Unique Users, Past 12 Months
All Other NationsU.S.
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 88
Data AvailabilityData Availability• ObservationsObservations
– CDCCDC•Derived from NCEP GTSDerived from NCEP GTS•1991-current, online, ASCII1991-current, online, ASCII•Updated monthlyUpdated monthly•Only Basic Variables Only Basic Variables •Primary Usage Primary Usage
– Extend I-COADS to near real-time Extend I-COADS to near real-time
http://www.cdc.noaa.gov/coads/nrt.html
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 99
Data AvailabilityData Availability• Monthly Statistics Monthly Statistics
– 11°x1°, 1960 - °x1°, 1960 - 19971997– 2°x2°, 1800 – 2°x2°, 1800 – 20022002
– NCARNCAR•Access equivalent to observationsAccess equivalent to observations
– Full binary format archive is onlineFull binary format archive is online– ASCII format with subset selection capabilityASCII format with subset selection capability
•Very popular user interfaceVery popular user interface
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 1010
Data Availability, MSG @ NCARData Availability, MSG @ NCAR
http://dss.ucar.edu/pub/coads/forms/
26
79
0
20
40
60
80
100
120
I-COADS MSG
NCAR, Unique Users, Past 12 Months
All Other NationsU.S.
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 1111
Data AvailabilityData Availability• Monthly StatisticsMonthly Statistics
– CDC CDC • I-COADS 1I-COADS 1° and ° and 22° netCDF format files° netCDF format files
– Through 1997 now, to be updated through 2002Through 1997 now, to be updated through 2002•22° extension based on NCEP GTS° extension based on NCEP GTS
– ASCII 1998 - currentASCII 1998 - current– NetCDF 1991 - currentNetCDF 1991 - current
http://www.cdc.noaa.gov/coads/coads_cdc_netcdf.shtml
http://www.cdc.noaa.gov/cdc/data.nmc.marine.html
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 1212
Data AvailabilityData Availability
• Monthly StatisticsMonthly Statistics– CDC, Live Access Server (LAS), CDC, Live Access Server (LAS), Public Public
DebutDebut• Interactive interfaceInteractive interface•Database – CDC I-COADS netCDF archiveDatabase – CDC I-COADS netCDF archive
– To be updated through 2002 soonTo be updated through 2002 soon•visualize visualize •subset variables, choose output file format subset variables, choose output file format •compare (difference) data gridscompare (difference) data grids
http://www.cdc.noaa.gov/coads-las/servlets/dataset
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 1313
Data Availability, CDC LAS, Constraints Data Availability, CDC LAS, Constraints InterfaceInterface
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 1414
Data Availability, CDC LAS, Output OptionsData Availability, CDC LAS, Output Options
Choose Option2 Graphic
NetCDF
2 ASCII
2 SpreadsheetGIS
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 1515
Data Availability, CDC LAS, 1Data Availability, CDC LAS, 1° 01/1970 SST ° 01/1970 SST MedianMedian
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 1616
Data Availability, CDC LAS, 1Data Availability, CDC LAS, 1°, SST Median,°, SST Median,Jan.1983 – Jan.1989, Niño – Niña differenceJan.1983 – Jan.1989, Niño – Niña difference
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 1717
Data AvailabilityData Availability• Advanced Products and AnalysesAdvanced Products and Analyses
– New Adaptive QC trimmingNew Adaptive QC trimming•Monthly SST, Monthly SST, 2°x2°, 1800-19972°x2°, 1800-1997•A few detailsA few details
– Authors: Tom Smith (NCDC) , Authors: Tom Smith (NCDC) , Xiao-Wei QuanXiao-Wei Quan (CDC) (CDC)– Observational outliers are trimmed according to a Observational outliers are trimmed according to a
reference climatology that varies with the climate reference climatology that varies with the climate signalsignal
– See link for other details and publication referencesSee link for other details and publication references•Access from NCAR, binary or ASCII (subsets)Access from NCAR, binary or ASCII (subsets)
http://dss.ucar.edu/pub/coads/forms/msg/msga.form.html
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 1818
Data Availability, Adaptive QC SSTData Availability, Adaptive QC SST
Original Trimming
viafix climate
Adaptive Trimming
viavarying climate
Decembers 1970-1997, 8-2S, 80-88W , Red = Trimmed Observations
20
30
10
0
30
20
10
0
82-83 97-98
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 1919
Data AvailabilityData Availability• Advanced Products and AnalysesAdvanced Products and Analyses
– Extended Reconstructed SST and SLPExtended Reconstructed SST and SLP•Authors Tom Smith and Dick Reynolds (NCDC)Authors Tom Smith and Dick Reynolds (NCDC)
http://lwf.ncdc.noaa.gov/oa/climate/research/sst/sst.html
http://www.ncdc.noaa.gov/oa/climate/research/slp/index.html
• Monthly SLP, 2°x2°, Global, 1854-1997Monthly SLP, 2°x2°, Global, 1854-1997–Land and OceanLand and Ocean
• Access from NCDC, ASCII formatAccess from NCDC, ASCII format
• Monthly SST, Monthly SST, 2°x2°, Global, 1854-2002, continuing2°x2°, Global, 1854-2002, continuing
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 2020
PlansPlans• Transitioning archive to IMMA formatTransitioning archive to IMMA format
– International Marine Meteorological International Marine Meteorological Archive (IMMA)Archive (IMMA)
– Why?Why?• IMMA is an ASCII format – easier to use?IMMA is an ASCII format – easier to use?•Gaining acceptance by WMO – JCOMM Expert Gaining acceptance by WMO – JCOMM Expert
Team for Marine ClimatologyTeam for Marine Climatology•Encourage I-COADS data contributors to use it Encourage I-COADS data contributors to use it
– Insures accuracy from data providersInsures accuracy from data providers– Easier to grow I-COADSEasier to grow I-COADS
•Well documented, and proven successfulWell documented, and proven successful– Project examples: CLIWOC, RT at NCDC, NCEP GTSProject examples: CLIWOC, RT at NCDC, NCEP GTS
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 2121
Plans, IMMAPlans, IMMA• Basic format concepts Basic format concepts
Core segment – fixed length
Original data attachments
+
Historical dataGTS dataSpecial formatsAux. QC info.
Working Principles• Core segment
All commonly used variables Fixed length ASCII (108 characters) Alone will be sufficient for most users
• Original data attachments Contains special fields not in Core Can be used to improve, repair, and
recreate CoreGreat format for:• an irreplaceable data archive• ensure easy software access (ascii based)http:/www.cdc.noaa.gov/coads/e-doc/imma/
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 2222
Plans, Real-time data for I-Plans, Real-time data for I-COADSCOADS• Observations from NCDC Observations from NCDC
– Features Features •Based on NCDC GTS data streamBased on NCDC GTS data stream•Web and FTP accessWeb and FTP access•Pilot activation date: September 2004Pilot activation date: September 2004
– Fast access b/c database systemFast access b/c database system•Will include the full historical archiveWill include the full historical archive
– Preparation so farPreparation so far•Cross-checks NCDC GTS and NCEP GTS streamsCross-checks NCDC GTS and NCEP GTS streams
– show good agreement and mutual accuracyshow good agreement and mutual accuracy
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 2323
Plans, Real-time I-COADS dataPlans, Real-time I-COADS data• Steps beyond real-time data provisionSteps beyond real-time data provision
– Archive two GTS data streamsArchive two GTS data streams– Compute Monthly Statistics Compute Monthly Statistics
•Generate @ CDCGenerate @ CDC– Lag time ~ 1 monthLag time ~ 1 month
•Benefit: Benefit: – More complete recordsMore complete records– Standard QCStandard QC– Standard formatStandard format
•Currently not high priorityCurrently not high priority
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 2424
Plans, Future data InclusionsPlans, Future data Inclusions• Details given by Scott Woodruff in a Details given by Scott Woodruff in a
presentation for 150presentation for 150thth Celebration Celebration• Light overview hereLight overview here
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 2525
I-COADS Release 2.0, 2003DM 1784-1997, RT 1998-2002Future Candidates: red(undigitized), yellow (partly), green (fully)
0
1
2
3
4
5
6
7
8
1790 1810 1830 1850 1870 1890 1910 1930 1950 1970 1990
Repo
rts/
Year
(Mill
ions
)
Polish Baltic: 1961- 90; 360K
US Navy Hourly: 1952-64; 3M
UK Marine Data Bank: 1950-79 Japanese Whaling Data: 1946-84;
30K Japanese Kobe (2003 ed.): 1889-1940; 3.2M
Arctic Drift Stations: 1952-76
U.S. Meteorological Journals: 1878-94;1.8M
Arctic Norwegian Data: 1867-1912; 125K
Fully digitized components to be added to I-COADS
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 2626
I-COADS Release 2.0, 2003DM 1784-1997, RT 1998-2002Future Candidates: red(undigitized), yellow (partly), green (fully)
0
1
2
3
4
5
6
7
8
1790 1810 1830 1850 1870 1890 1910 1930 1950 1970 1990
Repo
rts/
Year
(Mill
ions
)
Ukrainian Hydrometeorological: 1958-85 Russian R/V Marine: 1936-2002; 4-
6M US Lightship Data: 1891- recent German Data: 1887-90;
30K CLIWOC: 1750-1850; 300K Partly digitized
components to be added to I-COADS
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 2727
I-COADS Release 2.0, 2003DM 1784-1997, RT 1998-2002Future Candidates: red(undigitized), yellow (partly), green (fully)
0
1
2
3
4
5
6
7
8
1790 1810 1830 1850 1870 1890 1910 1930 1950 1970 1990
Repo
rts/
Year
(Mill
ions
)
German Maury Collection: 1845-67; 500-750K
Undigitized component to be added to I-COADS
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 2828
Plans, Future data InclusionsPlans, Future data Inclusions• Early access to new data sourcesEarly access to new data sources
– Availability before merge into I-COADSAvailability before merge into I-COADS•Termed “add on” datasetTermed “add on” dataset
– Features Features •Data are subjected to climate trimmingData are subjected to climate trimming• In an I-COADS formatIn an I-COADS format•Unique and important dataUnique and important data
– E.g. Kobe Observatory and CLIWOC dataE.g. Kobe Observatory and CLIWOC data•From NCAR From NCAR
http://dss.ucar.edu/datasets/ds530.0/
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 2929
Plans, Web Status and Plans, Web Status and DevelopmentsDevelopments• New “Data and Metadata Page”New “Data and Metadata Page”
– New lookNew look•Organized by data productsOrganized by data products
– Eliminated the CDC/NCAR/NCDC structureEliminated the CDC/NCAR/NCDC structure• I-COADS metadataI-COADS metadata
– Software and documentationSoftware and documentation– Graphics and figures to show I-COADS inventoriesGraphics and figures to show I-COADS inventories– Ship metadata – Pub47 (1973-1999)Ship metadata – Pub47 (1973-1999)
•Most products mentioned here are linked to Most products mentioned here are linked to this page.this page.
http://www.cdc.noaa.gov/coads/products.html
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 3030
Plans, Web Status and Plans, Web Status and DevelopmentsDevelopments• ““Related Data and Resources”Related Data and Resources”
– Links to Projects and OrganizationsLinks to Projects and Organizations•VOSCLIMVOSCLIM•CLIWOCCLIWOC• JCOMMJCOMM
– Links to Associated DataLinks to Associated Data• JISAO Climatologies (Mitchell UW)JISAO Climatologies (Mitchell UW)
– 11° global ° global •Extended Reconstructed Analyses (NCDC)Extended Reconstructed Analyses (NCDC)
http://www.cdc.noaa.gov/coads/related.html
Grow this area, link more projects and Grow this area, link more projects and datadata
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 3131
ConclusionsConclusions
I-COADS is international
1998-2002 update available now
Many access points and formats
New analyses are available
New format (IMMA) will be future standard
I-COADS real-time observations – coming next year
Brussels, Belgium 17-22 Nov. 2003 CLIMAR II, I-COADS Data and Products 3232
ThanksThanks!!