Upload
brad-houston
View
3.134
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Presentation on reformatting options for active and inactive records. Originally presented at the 2009 Annual Conference of the International Institute of Municipal Clerks, May 20, 2009
Citation preview
Microform or Digitize: Which is Right For You?
Brad HoustonRecords Manager, University of
Wisconsin-Milwaukee
Reformatting
• v. 1. To create a copy with a format or structure different from the original, especially for preservation or access. 2. To migrate information from one carrier to another.• Source: Richard Pearce-Moses, A Glossary of Archival and Records Terminology,
Society of American Archivists, 2005
– Encompasses microfilming, digitization, photocopying, redevelopment, you name it!
Why reformat your records?
• Save space in the office or records center• Improve access to frequently-requested
records• Move records to format more suitable for
long-term preservation– Alternately: Provide duplication for
redundancy/off-site storage
Goals of this presentation
• Discuss general issues to consider before embarking on reformatting project
• Examine the two main forms of reformatting practical for most municipalities – Digital and Microfilming, of course!
• Describe scenarios in which either, both, or neither type of reformatting is appropriate
PART I: GENERAL CONSIDERATIONS
LegalOperationalAdministrative
Before you start reformatting…
Why is my office thinking about this in the first place?– Space concerns?– Vital Records Protection?– Increased Public Access?– Better Discoverability?– Improved Security?– Do these records NEED to be reformatted?
How are these records used?
Preservation Concerns• Vital records
– Deeds, Birth Certificates, etc.
• Private-use records• Archival Documents
– Policies, reports, minutes
• Infrequently accessed• Unlikely to be
subpoenaed/discovered
Access Concerns• Administrative records
– Notarization, budgets, etc.
• Public-use records• Short-retention documents
– Procedures, expense reports
• Frequently accessed• Increased likelihood of
subpoena/discovery
What are the startup costs?
• Time/Labor needed to reformat– Specialized labor/skills needed for project
• Equipment outlays• Public notices (as needed)• Other buy-in costs (appeals to officials?)
What are the maintenance costs?
• Cost of physical equipment maintenance• Time/labor needed to maintain• Time/labor needed to provide access• Media stability offsets• Is the maintenance cost offset by return on
investment?
What are the security implications?
• Does the information in the reformatted records require protection?
• If so:– Reformatting a net increase or decrease in
security?– Initial security measure outlays– Staff information security training– Secure access to public (as applicable)
What are the legal implications?
• Federal Rules of Civil Procedure Sec. 34 (December 2006)– Discovery applies equally to reformatted records!
• Public Records Laws in various states– Example: Wis. Stat. 19.32: Open Records Act
• Other federal/state disclosure laws– Sarbanes-Oxley, Gramm-Leach-Bliley, etc.– May apply if working with private contractors
Should I reformat at all?
• Do reformatting candidates have high retention periods (>10 years)?
• Is my estimated ROI (in either time or cost) more than the time or cost to reformat?
• Do I have the infrastructure (or can I build it) to adequately support and maintain system?
If no to any of the above, consider postponing reformatting program!
PART II: MICROFILM AND MICROFICHE
An Oldie But a Goodie
The least you need to know: Microfilm
• Preservation: Well-suited• Access: Requires time investment• Initial outlay: Medium equipment/low
training costs• Continued Maintenance: Low Cost• Security Impact: Neutral• Legal impact: Potentially high
First steps: Needs planning
• Active vs. Inactive records for reformatting?• Storage and use copies? One or both?• Need for long-term preservation?• Microfilming from source documents or data
from a computer?• Large or small amount of documents to be
filmed?
Film Selection: Physical Medium
• Diazo Film– Best for use copies; less stable for permanent
preservation• Vesicular Film– Flips master image polarity; good for non-printing
films. Non-ideal for preservation• Silver Halide Film– Best for long-term preservation; easily damaged
with repeated use
Film Selection: Format
• Microfilm roll (16mm or 35mm)– Inexpensive and compact, but cumbersome to
access– Best format for storage of vital or archival
documents
• Microfiche (105mm, rows/columns)– Easier to organize/access, but more expensive to
produce and store– Best format for case files, semi-active records
Initial Outlay: Equipment
• Microfilm Camera– Rotary—best for initial
reformatting project– Planetary—better image
quality, diverse sizes
• Microform reader– Different designs for film
and fiche– some do both, for a price
– Built in printer?
Planetary microfilm camera
Initial Outlay: Document Prep
• Remove documents from folders, remove fasteners, etc.
• William Saffady: 1000 pages/hour (2.5 drawers/hour) as a benchmark– This figure increases for non-standard file sizes
• Do you need a target? Find out before you film!
Reduction Parameters
• Reduction: Number of times a page is reduced through microphotography– 24x reduction for standard-sized documents– 30x reduction for large documents– 48x for COM reduction
• Remember to make sure your microform reader can read the specified reduction!– Variable-zoom lens, as appropriate
Quality Control
• Should be done for each slide to check for:– Legibility (naked eye overview)– Resolution: ability of reader to distinguish fine
detail• Specialized resolution chart
– Density: Contrast between info/non-info areas• Densitometer
• For standards, see: ANSI/AIIM MS23-1998
Systems Maintenance
• Low cost of providing access, training staff/public on use
• Cameras may need to be replaced periodically• Readers may last decades, depending on
complexity– If all else fails, records readable with a light table
and magnifying glass!
Environmental/Security
• On-site records=your responsibility to maintain good preservation environment!– Windowless room if possible– No more than 50% relative humidity– No more than 70 degrees Fahrenheit– Acid-free/chemically stable sleeves (microfiche)
• Same security implications as with paper records– Maintain secure room, locked cabinets, etc. as
needed
Access concerns
• Microfiche may be indexed/organized same way as “regular” case files
• Microfilm: roll format means special index should be created– Consider primary access point of records– Entries should include: reel, frame, secondary access– Provide cross-references as needed– Indicate in index when filming took place, and by
whom the records were filmed
Legal Issues and Records Retention
• Will you be able to produce the documents in a timely manner?
• Uniform Photographic Copies Act (28 USC 1732)
• If records have 10 year retention period or less, off-site paper storage may be more cost-effective
Final Verdict:
Microfilm remains useful for its relative lack of expense, easy reproduction, and extreme longevity (>500 years!). However, the issue of accessibility, particularly with microfilm rolls, must be taken into account when deciding to film records for preservation or access.
PART III: DIGITIZATIONNew Paradigms, New Problems
The Least You Need to Know: Digitization
• Preservation: Potentially problematic• Access: Very good, with the right tools• Initial Outlay: Low equipment/medium
software/high staff training• Continued Maintenance: high level of staff
time/effort• Security Impact: Potentially high• Legal Impact: High, but manageable
Digitization is NOT a panacea!
• Potentially very powerful productivity tool and space saver
• At minimum, the following are required for an effective program:– Content Management System– Digital Preservation Plan– Disaster Recovery Plan
Needs Planning: Digitization
• Active vs. Inactive records– For digital records, this is a function of retrieval
speed
• Public vs. Private access to the records• What are your preservation needs?• File-sharing needs?
Initial Outlay: Hardware
• Scanner, preferably with a document feed– Resolution required depends on type of scan
• Digital Camera/Overhead Scanner• Importing/editing computer• Public Access computer (as needed)• Storage Media
Digital Storage Media
• Good: Removable storage media (CD-ROMs, DVD-ROMs, External Hard Drive)– Good for quick access; low longevity
• Better: Disc Array or Server Array– Medium longevity and access speed; better for
redundancy of data
• Best: Distributed Storage Space– Maximum redundancy; slowest retrieval time
Initial Outlay: Software
• Scanning/Import software– Usually comes with scanner
• Image Manipulation software– For document reformatting, include Optical
Character Recognition (OCR) capability
• Content Management System (CMS)– The most important (and most expensive)
component of digitization management
Content Management Systems
• Should include robust metadata at point of entry:– Subject– Author– Recipient– Date– Other keywords/tags
• Some CMSs extract this automatically– ask your vendor!
Content Management, Records Management
• Consider a CMS with integrated Records Management:– Facilitated indexing– Automated or semi-automated disposition– Improved information security– Legal protection under “Safe Harbor” rule (FRCP
37)
DoD 5015.2 RM standard
• Standards for ERMS– often advertised!• A compliant system will include:– Metadata entry and file plans (including retention
scheduling)– Records declaration and classification– Records protection and/or change auditing– Designation of official ERMS administrator(s)
Imaging Parameters
• Pixels per inch/dots per inch (ppi/dpi)• Capture mode• Bit Depth• File Format• Compression• n.b. Higher values increase image quality
AND file size/storage requirements
PPI/Resolution
• Should be as high as possible, but consider display and space needs!
• To adjust resolution of already-imported image, adjust image size (smaller=sharper)Document Type PPI (master) PPI (access)
Text 400 150
Photographs 400-800 200-300
Maps 600 (400 if oversized) 150-300
Graphics (including arch. Drawings)
600-800 200-400
Capture Mode/Bit Depth
• A measure of the amount of colors able to be displayed in the image
• USUALLY an easy call to make– with some exceptions
Document Type Capture Mode Bit Depth
Text Bitonal 1 bit
B/W photos/drawings Grayscale 8 bit
Color photos/drawings RGB 24-48 bit (access v. master)
Maps Grayscale/RGB 24-48 bit (access v. master)
File Format/Compression
• Higher compression=greater degradation of data over time
• Lossy vs. Lossless compression• Proprietary vs. Non-Proprietary formats
Document Type File Format (Preservation) File Format (Access)
Text TIFF or PDF/A JPEG or PDF
Photographs TIFF or JPEG 2000 JPEG
Graphics TIFF or JPEG 2000 JPEG
Maps TIFF or JPEG 2000 JPEG
What’s in a name?
• Don’t rely on the CMS to find files for you!• File names should be easily-browsable:– Date of imaging or use– Type of file (e.g. dept. acronym?)– Unique identifier (e.g. project name, serial #)
• File naming not a substitute for proper metadata in the CMS
Quality Control: Physical Aspects
• Looking for similar defects as with microfilm QC– Legible images– Correct orientation/resolution– Density, color balance, etc.• Targets and color bars assist with this check
• Some QC defects are digitization-specific:– Correct size, name, file format, etc.– Digital artifacts (lines on screen, etc.)
Quality Control: Content Aspects
• Has the OCR read the content properly?– Usually a cursor drag determines this– Most OCR programs allow for manual corrections
• Is the metadata complete and appropriate?– Dublin Core: search and use metadata– PREMIS: Preservation metadata– Administrative metadata (rights, statutes, etc.)– Structural metadata (how records tie together)
Systems Maintenance
• Maintenance of a digitization system often comprises up to two-thirds of total budget!– Cataloging, description, indexing– Ongoing staff training– Physical data storage– Data migration
• Consider cost-sharing with neighboring municipalities
Access to Digitized Records
• The key upside to digitization! • Public Access Terminal/Web Access/Admin
use only?• Rights management?– Will you allow printing?– Watermarks
• Electronic file structure• Searchability?
Preservation Concerns
• IT department usually keeps data storage secure, environmentally stable
• Data itself may have stability problems!– Software obsolescence– Hardware obsolescence– Data integrity loss– File format obsolescence
Preservation Solutions?
• Non-proprietary formats, lossless compression– Usually implies access copy creation
• REGULAR (5 years/3 version) reformatting:– Migration to new systems/file formats– Emulation of programs to read files on old systems– Encapsulation in XML “wrappers”
• Work with IT to see which is right for your organization
Information Security
• Risk of security breach is MUCH higher because of ease of reproducibility
• Potential Solutions:– Versioning/Logging of imaged documents– Declaration of imaged documents as “records” in
DoD 5015.2 system– Visible or metadata-based watermarks– Embargo on portable storage for imaged
documents
Digitization Legal Issues
• Zubulake Case (2004): If you have the records, you must produce them or pay penalties
• Federal Rules of Civil Procedure– Electronic records are discoverable– Safe Harbor rule DOES NOT APPLY if there is
reasonable expectation of litigation (Broccoli)– Plaintiff may request format for production
Legal Issues: Responses
• Make sure your CMS has a robust search feature– Semantic web /fuzzy logic?
• Be prepared to convert formats as needed• Talk to legal team about Litigation Hold
procedures• Formalize records retention/destruction
schedules and procedures
Final Verdict
Digitization offers organizations excellent options for saving space and making reformatted records accessible to staff and to the public. However, great care must be taken to ensure that preservation, security, and legal issues are addressed before a digitization solution is implemented.
PART IV: CONCLUSIONSummation, scenarios, resources, questions
In Summary:
Microfilm and Microfiche• Excellent preservation
medium• Low-to-medium startup
costs• Low maintenance and staff
training requirements• Potential accessibility issues• High security, reduced legal
response ability
Digitization• Superb accessibility if
managed within CMS• Increased ability to respond
to legal requirements• Medium-to-high startup
costs• High maintenance and staff
training requirements• Security and preservation
issues
Scenarios: Office A
• Reformatting target: Birth Records– Considered a “vital record” by statute– Medium-to-high usage– High security needs– Medium-to-low subpoena likelihood
• What would you suggest?– My answer: Microfilm
Scenarios: Office B
• Reformatting Target: Financial Records– Records Schedule states 7-year retention time– Subject to public access under state records law– Very high use– referred to on daily basis– High likelihood of audit/subpoena– Medium security requirements
• What would you suggest?– My answer: Do not reformat OR digitize as
requested
Scenarios: Office C
• Reformatting Target: Child Welfare Case Files– Records Schedule states 20-year retention time– High use while clients active; low use thereafter– Very high security requirements– Medium-to-high potential for subpoena
• What would you suggest?– My answer: Microfiche case files and/or digitize
Scenarios: Office D
• Reformatting Target: Committee Minutes– Permanent Retention/Archival– Subject to public access under state records law– High use by both public and staff users– Low security requirements– Medium likelihood of audit/subpoena
• What would you suggest?– My answer: Digitize with web access