USGS/EDC

Archive Preservation

StatusAn Update

Stuart Doescher, USGS

(Cheryl Greenhagen)WGISS-19March 2005

Cordoba, Argentina

TM and MSS Archive Conversion TM and MSS Archive Conversion System (TMACS)System (TMACS)

• Operations: 1992 to 2002.• Number of High Density Tapes (HDT)

transcribed: 32,300– Landsat MSS, TM

• Number of DCRSi Cassette Tapes (DCT)s output: 3,200

• Mostly in Telemetry stream format (Frame sync required to read)

WBVT summaryWBVT summary

• Operated: April 1999 - May 2002• Number of HDTs converted: 21,300• Number of Digital Linear Tape (DLT 7000)

generated: 143 and an additional 143 as backup • Format: BSQ ( MSS-X)• "orphan" data (no DIAT / SLAT) save on 172 DLTs• Bad input tapes: 476 (even with cleaning &

baking).

SPOT summarySPOT summary

• Operated: late April 2000 - February 2002

• Number of HDTs converted: 6,100

• Number of DLT 7000 tapes generated: 901

• Format: MDA's FRED

Landsat Archive Conversion System (LACS)

• Planned Operations: June 2004 – June 2006

• Input: About 6000 HDTs, 4000 DCTs, plus current Landsat 5 data

• Output: ~ 1000 tapes of "formatted" data and ~ 1100 tapes of "raw" data on 9940B. (200G/tape)

• Format: BSQ

LACS DescriptionLACS Description

• Ingest current Landsat 5 data from disk files• Ingest data from DCRSi Cassette Tapes (DCTs)

– Multispectral Scanner Archive format (MSS-A) data– Multispectral Scanner Product format (MSS-P) data– Thematic Mapper Archive format (TM-A) data– Thematic Mapper raw (TM-R) data

• Ingest TM-R data from High Density Tapes (HDTs)• Archive the data to a high-density computer-compatible

digital tape: 9940B • Generate and archive browse imagery• Generate inventory information for the new archive.

Data to be converted by LACSData to be converted by LACS

Data SetNumber of

ScenesNumber of

Tapes Data Volume

MSS-P 65,128 118 DCTs 3.2 terabytes

MSS-A 262,088 277 DCTs 9.5 terabytes

TM-A 13,733 108 DCTs 3.6 terabytes

TM-R (1982 – 1994) 593,563 3,431 DCTs 147.4 terabytes

TM-R (1995 +) 106,300 6,000 HDTs 30.0 terabytes

TM-R (new) ~ 8 terabytes

 1,040,812  Total 201.7 terabytes

Current LACS ConfigurationLACS1

LTS 1: D isk file IngestL IS , LA C S D B

LACS3LTS 3: H D T Ingest

LACS2LTS 2: D C T Ingest

LAM

DCS

LAC S1Local D isk

SAN NA3

LAC S3Local D isk

LAC S2Local D isk

SILO

Loca l

D B linkS A Nnfs

LACS Status

• Started Operations: June 2004

• Progress to Date: (January 2005)

– Current Landsat 5 Data: 949 Intervals (22,061 scenes)

– DCTs: 318 Tapes (2,809 Intervals, 53,697 scenes)

– HDTs: 2624 Tapes (2,846 Intervals, 52,203 scenes)

• This is 105,900 of 1,040,812 scenes in 6 ½ months

(Implies total time greater than 5 years with 4.5 years

remaining - that would imply a July 2009 completion)

LACS StatusLACS Status

• LTS (LACS Transcription Subsystem)

3 Strings:

– LTS1: Current Landsat 5 data only (acquired at EDC, or received on DLT from Australia.)

– LTS2: DCT Input ( 2 to 3 shifts per day)

– LTS3: HDT Input. (1 to 2 shifts per day)Tapes (HDTs) & tape drives (HDTRs) are cleaned before use Many of the HDTs are bakedSome HDTs are tried many times before successHDTRs may require adjustment for specific sets of tapes

LACS StatusLACS Status

• LIS (LACS Integration Subsystem)

1 String:

– Performs Automated Cloud Cover Assessment,

Browse Generation, Quality Assignment

– Provides Interactive Assessment Capability

– Gives priority to current Landsat 5 data

– This system is currently the bottleneck

LACS Status

• Issues / Problems:– Some tapes in very poor condition– LIS performance needs improvement

• Plans:– Continue to improve handling of poor-quality data– Performance Improvements:

• Run LIS on 3 Linux systems• Run DCT Ingest on 2 LTS systems• Run DCT Ingest on 3 LTS systems when HDTs done

– Anticipate 3 time improvement (1.5 years remaining that would imply a July 2006 completion)

Planned LACS Configuration

LACS1LTS 1: R C C Ingest

LACS3LTS 3: H D T Ingest

LACS2LTS 2: D C T Ingest

LAM

DCS

LAC S1D isk

SAN

LAC S3D isk

LIS1LIS 1: D B /B /A

LIS2LIS 2: B /A

LIS3LIS 3: B /A

LCDSLACS DBDCS DB

SILO

Local

D B linkSANnfs