Upload
debra-hunt
View
226
Download
0
Tags:
Embed Size (px)
Citation preview
LBA-DIS Working Group Report
LBA Science Steering Committee Meeting
-- LBA-DIS Report --
Ubatuba, SP
May 13-15, 2004
LBA DIS WG Topics
• Data Registration / Archive Status
• Recommendations from 14th SSC Meeting, Fortaleza
• Interactions with other Projects
• Long-term Archive-Related Activities
• System Update
• Software Development Plan
• LBA DIS Institutional Support
• LBA DIS Composition
LBA Data Registration Status
LBA Overall Status:Metadata Registered in Beija-flor
260
40
0
62
8
118
63
133
83
152
80
231
92
325
100
379
207
413
211
418
211
469
215
0
50
100
150
200
250
300
350
400
450
500
06/99 01/00 06/00 10/00 02/01 05/01 11/01 05/02 11/02 05/03 11/03 05/04
# of data sets registered
# of posters registered
N=684
Increased 8.9%
LBA Overall Status:Data Volume Archived at LBA-DIS / CPTEC
4.00.50.6
14.88.7
1.1
69.0
8.91.2
79.6
9.21.3
104.8
9.31.3
0
20
40
60
80
100
120
Gig
ab
yte
s
11/01 11/02 05/03 11/03 05/04
Unrestricted DataRestricted DataPosters
28.1% increase includes:
CLAIRE DataEUSTACH Flux DataORNL Regional Data
Hyperion DataNepstad Risque Data
SPOT NDVI DataMelack Wetlands Data
LBA Component Status
122
45
23
50
10
3
0
0
341
22
12
0 100 200 300 400 500 600
US-BR
EU-BR
BR
Data Sets with data available 341 22 12
Data Sets w/ broken links ? 0 0
Data Sets w/o data 50 10 3
Posters 122 45 23
US-BR EU-BR BR
xx
Con
N=38 (9 of 17 Phase 1 teams contributing)
N=77 (13 of 16 Phase 1 teams contributing)
N=513 (58 of 60 Phase 1 teams contributing)
*
* Increase of 12 data sets (LBA Ra-CCI)** Decrease of 10 broken links (Data now at CPTEC)
**
SSC Recommendationsfrom 14th SSC Meeting, Fortaleza
• Improve data registration/archive performance record for European and Brazilian science teams
• Report on data archived by LBA DIS but not registered on Beija-Flor
• On-line tutorial for Beija-Flor / Metadata Editor
• Each LBA team should name a primary and secondary data contact
• Make LBA tower flux data available to Flux networks: AmeriFlux & FluxNet (12th SSC)
Data Registration/Archive Performance of EU and BR teams
• Broken links on all of the LBA data on Alterra Web Site (~10 data sets)
• LBA tower flux data – (FluxNet & Ameriflux)
• Plan for next 6 months:
• Contact each of the non-contributing Brazilian and European/BR teams to acquire data
• Complete analysis of data in archive but not registered -- theory is that much of the “missing” EU/BR & BR data is archived but not registered
Data Archived but Not Registered
Huge task!!
1. Develop a method to cross-reference [hundreds of] data directories in archive with entries in Beija-flor
2. Record these findings in a “data tracking system”
• Link the B-f registered data sets with corresponding archive directories
3. Identify the archive directories that aren’t represented in Beija-flor – who? what? when? where? …
4. Register the “found” data
On-line Tutorial and Help Files • LBA Metadata Editor (LME):
• FAQs updated
• Glossary updated
• Tutorial updated and improved
• Beija-flor
• Tutorial updated
• Beija-flor User’s Guide
• Contains all of the above [updated] information
• Available for download as .pdf (in Portuguese & English) on Beija-flor site
Data Contact for each Science Team
• Data Contact for Brazilian-led teams will be PI
• LBA-ECO requested in Annual Reports for Phase 2 teams that Data Contact be designated
• 25 teams complied; PI will be default Data Contact if not specified
• LBA-ECO Phase 1 Teams: PI is the default Data Contact
Interactions with Other Projects
• CEOP (<= LBA) Flux, Surface met, Soil data
• HIBAM (=> LBA) Hydrology data
• FluxNet / Ameriflux networks (<=LBA)
• CODATA Workshop, Campinas (March ’04)
• SIVAM < = > LBA, Manaus, AM (April ’04)
• TEAM Workshop, Belém-PA (March ’04)
• PPBio Infrastructure Plan, Belém-PA (April ’04)
• GEOMA Seminar, Ilhabela, SP (May ’04)
PantanalReserva Jaru
RondoniaFazenda NS Aparecida
Manaus-K34 Flona-Santarem
Brasilia-Cerrado
Fig. 1. LBA-CEOP Reference sites
Caxiuana
LBA is a CSE (Continental Scale Experiment) for GEWEXCEOP (Coordinated Enhanced Observation Period) Phase 3 in progressOther CSEs: BALTEX, GAPP, etc.
NOT operational
Data N/A
Data N/A
CEOP 3 Phase I
Study Area Data Contact
Data Coverage (Oct01,2002 to Mar31,2003)
Registered In Beija-flor?
Tower still operational?
Surface Metereological and Radiation Data Archived
in LBA-DIS
Flux Data Archived in
LBA-DIS
Soil temp and
moisture Data
Archived in LBA-DIS
Meteorological Tower Data Archived in
LBA-DIS Brasilia-Cerrado Heloisa S. Miranda n n n n n
Caxiuana Julia Cohen y y y y y
Flona-SantaremHumberto Rocha / Scott Miller y y y y y
Manaus-K34 Hermes Braga Xavier y y n n yPantanal Carlos Cesar de Oliveira n y n n yRondonia Fazenda NS Aparecida Fernando Luiz Cardoso y y y y y
CEOP 3 Phase II
Study Area Data Contact
Data Coverage (Apr01,2003 to Sep30,2003)
Registered In Beija-flor?
Tower still operational?
Surface Metereological and Radiation Data Archived
in LBA-DIS
Flux Data Archived in
LBA-DIS
Soil temp and
moisture Data
Archived in LBA-DIS
Meteorological Tower Data Archived in
LBA-DIS Brasilia-Cerrado n n n n n
Caxiuana y n n n nFlona-Santarem y n n n nManaus-K34 y n n n nPantanal n n n n nRondonia Fazenda NS Aparecida y n n n n
HIBAM Data Status
• Metadata has been registered in Beija-flor– Data currently accessible from relational database on 3
servers• ANA – Agencia Nacional de Aguas - Brasilia
• IRD – Institute de Recherche por le Development – France
• Golden-Teledetection - France
• Data being copied to LBA DIS / CPTEC for distribution from there as well
• Waiting for authority to distribute from CPTEC
Flux Data StatusLBA Investigator/Site
In
Beija-flor?
Data Available
Data Archived
in LBA-DIS
In AmeriFlux
In
FluxNet
Wofsy/Tapajos Yes Yes Yes Yes --
Goulden/Tapajos Yes Yes Yes Yes --
Fitzjarrald/Santarem Yes Yes, but restricted
Yes No --
Manzi / Rondonia Yes Yes Yes (Agreed)* --
A. Nobre / Manaus Yes Yes Yes -- (Agreed)*
Malhi / Braganca Mangrove characterization but no flux data
Char. data but no flux data
Char. Data only (still no
flux data)
-- Yes
Da Rocha?
Others?
* Requirements TBDColor indicates progress since 11/03
Long-term Archive-related Activities
• LBA-ECO is attempting to link pubs to data sets
• LBA-ECO Data Archive-Related Processes, Roles, & Responsibilities Document
• LBA-ECO Documentation Standards / Doc Tool Status
• Implications for LBA DIS / CPTEC archive
• Documentation standards, completeness?, e.g. fields labeled properly, units of measurement identified?
• Data quality assessment? e.g. calibrations, algorithms used?
• Should we try to differentiate wide range of data/documentation quality within archived data collection? e.g. a rating or “score”?
Submit Data Packages to ORNL for Archive
Work w/ PI to Prepare Data Package
Submit to Archive Panel for Evaluation
Send Data Package “as is” to LBA DIS for archive
ORNL ingests data package into DAAC
NASA ORNL DAAC
Archive
Send Data Package to LBA DIS for archive
Review Metadata
LBA DISArchive
Send data to LBA DIS
Metadata
Register Metadata
Raw Data
Update
Mark as “Ready for Archive”
Preliminary
Data
Final Data
Data SetApproved to be Archived?
NoYes
Send data to LBA DIS
Legend -- RolesPI / Team’s Data ContactData CoordinatorArchive Review PanelLBA DISORNL
LBA-ECO Data Archive Process: Overview
Ancillary FilesData Set
Doc DataMetadata
File
24-February-2004
Legend -- RolesPI / Team’s Data ContactData CoordinatorArchive Review PanelLBA DISORNL
LBA Data Archive Process:Data Archived at LBA DIS / CPTEC
LBA-ECO Data (Preliminary & Final)-- Not Permanently Archived at ORNL
Restricted Data – All Sources
Available to the public
Restricted Access
LBA-ECO Final Documented Data-- Also Permanently Archived at ORNL
European & South American Teams’ Data-- Preliminary… Final Data
Historic Data / Pre-LBA Data
LBA DIS Archive
24-February-2004
System Updates
• Status of new LBA Home Page
• LBA server almost out of space for new data – Additional disk space requested in 9/03– Purchased and should arrive soon
• New database for Palm users created: contacts, groups, sites
LBA-Server Hard DrivesWHY are they so expensive?
• Industrial STRENGTH• Faster seek time: 0.55 ms • Faster access time• Faster Transfer Rate : 320 MB/s • Faster Rotational Speed: 10,000 rpm • Faster interface: Ultra320 SCSI • Raid-5 (fault tolerance: ability of a
system to respond gracefully to an unexpected hardware failure)
• About US$ 300 / 36.9 Giga bytes ($550 if bought in Brazil)
LBA-DIS server array of SCSI drives
Perspectives for post-December 2005
• LBA DIS will receive data long after field work is over (as LBA-ECO data is archived at the DAAC)
• LBA-ECO will use Beija-flor / LME after field work is over and data are QA’d & documented, i.e. until data archive is complete
• LBA-ECO computer systems will be migrated to LBA • LBA will plan to adopt freeware platform for data
management in the long-term• Long-term LBA data management issues will be
addressed & post-LBA plans implemented
Software Development/Migration Plan
• Post-LBA non-proprietary Beija-Flor replacement:LBA DIS will propose a plan to implement an enhanced Beija-Flor system and LME based on freeware platform for long-term data/metadata management after LBA data
• LBA-ECO STI system to be migrated to LBA Central Office (beginning Aug-Sept/2004)
LBA DIS Institutional Support
• LBA Publication Management – INPA’s Central Library on the Web (Done)• Database configuration/migration at INPA – Client/Server Platforms (Done)• LBA Offices Manaus, Santarém, & CPTEC - Systems and computer infrastructure documentation (Done)• Regional Offices – Systems requirements and plans for computer and network enhancements (Done)• 1st LBA Informatics Week at Santarém – June’04
LBA DIS Distribution of Tasks
• LBA Central Office– Responsible for computing infrastructure for DIS– Technical training for staff
• LBA DIS / CPTEC– Data/Metadata archive– Interactions w/ LBA & LBA affiliate science teams re: data– Coordination of updates for Beija-flor & LME
• LBA-ECO Project Office– Coordination with:
• LBA DIS at Central Office & CPTEC • LBA-ECO science team • ORNL DAAC