Data Storage Considerations for the Tactical Field Collection of Digital Imagery - 2010

Research Paper

Data Storage Considerations for the Tactical Field Collection of Digital Imagery

INTL625 Imagery Intelligence

20 April 2010

Robert L. Watson

2

Table of Contents

Introduction ................................................................................................................................................. 3

Chapter 1 Introduction to Storage Media ................................................................................................ 5

Magnetic Tape ........................................................................................................................................... 6

Optical Disks .............................................................................................................................................. 7

Magnetic Hard Disks ............................................................................................................................... 11

Solid State Drives .................................................................................................................................... 12

Redundant Array of Inexpensive/Independent Disks (RAID) ................................................................. 17

Chapter 2 Introduction to Video Compression ...................................................................................... 25

Video Compression Standards ................................................................................................................. 28

Metadata ................................................................................................................................................... 30

Multimedia Players and Container Formats ............................................................................................. 34

Chapter 3 Recommendations ................................................................................................................... 37

Hardware Recommendations ................................................................................................................... 37

Software Recommendations .................................................................................................................... 39

Video Retention and Deletion Schemes ................................................................................................... 41

The Future ................................................................................................................................................ 44

Chapter 4 Conclusion ............................................................................................................................... 46

Bibliography .............................................................................................................................................. 53

3

INTRODUCTION

As a result of the Global War on Terror, the need for timely intelligence, both on a global

as well as local scale has facilitated the need for more sophisticated imagery intelligence

gathering systems that provide useful information from satellites, unmanned aerial vehicles and

stationary surveillance systems, to name but a few. This large data gathering effort is

unprecedented in history and presents new challenges to our country and its government.

A recent article in the New York Times stated: "Military deluged in intelligence from

drones: Remote-controlled planes produce about 24 years' worth of video in 2009."1 Other

recent news has described the U.S. Intelligence community’s inability to preempt the Christmas

Day 2009 terrorist attempt as a byproduct of "information overload."2

Out of necessity, the need for timely imagery intelligence that provides support to both

our military and government has created the requirement for large groups of skilled analysts to

evaluate the enormous amount of imagery data that is being collected to determine what is

significant to the intelligence effort. In light of this, large-scale digital storage systems are

necessary to provide data storage and retrieval.

However, not all of the imagery that is collected is useable or of significant intelligence

value. In some instances, the time or resources may not exist to have the imagery reviewed and

analyzed in a timely manner, so its value as an intelligence source may diminish.

1 Christopher Drew, “NYT: Military deluged in intelligence from drones - Remote-controlled planes produced

about 24 years’ worth of video in 2009,” New York Times, January 10, 2010, Sunday;

http://www.msnbc.msn.com/id/34798080/ns/world_news-he_new_york_times/ (accessed January 11, 2010).

2 Dan De Luce, “US spy agencies face information overload: experts,” AFP (via Yahoo News), January 7, 2010,

Thursday; http://news.yahoo.com/s/afp/20100107/pl_afp/ usattacksintelligence (accessed January 25, 2010).

http://www.msnbc.msn.com/id/34798080/ns/world_news-he_new_york_times/

http://news.yahoo.com/s/afp/20100107/pl_afp/%20usattacksintelligence

4

The paper will explore the current storage technologies and methodologies in use, their

advantages and shortcomings, and provide some insights and possible solutions to this ever-

growing issue.

The research will focus primarily on solutions for real-time digital video gathering

systems, specifically stand-alone, tactical systems currently used in the field environment. Since

a system’s storage capacity and the means of increasing this capability may be limited due to

system design and mission constraints, newer storage technologies and methods will be

suggested to include specific metrics, both user-initiated and software-based, that could be used

as criteria in determining imagery retention and disposal.

5

CHAPTER 1

INTRODUCTION TO STORAGE MEDIA

Digital storage devices are the building blocks of storage in disk subsystems and

standalone server systems. Functioning in the microscopic realm, they perform the reading and

writing functions necessary for the storage of data on nonvolatile media. Digital image

processing of remote sensor data and its associated geographic information system (GIS) data

requires significant storage resources, which are also necessary for the collection of large

imagery databases.3

Table 1 illustrates some of the more common digital mass storage devices and their

average time to physical obsolescence – the point at which the media begins to deteriorate and

data loss can occur.

Table 1. LONGEVITY OF DIGITAL STORAGE MEDIA.

AVERAGE LONGEVITY OF DIGITAL STORAGE MEDIA

MEDIA TYPE AVERAGE OBSOLESCENCE

(IN YEARS) REMARKS

Optical Disk >100 Very cost efficient, presently best means of long-term, digital

storage

Magnetic Disk

(Hard Disk Drive)

20 Main system storage medium, but storage longevity normally

less than 20 years.

Magnetic tape 10-15 Cost efficient, but may become unreadable if not rewound and

properly stored in a cool, dry place.

Flash-Solid State

Disk*

10 Expensive, but provides faster seek time than magnetic hard

disks. Data retention for up to 10 years without power applied. Jensen, John R., Introductory Digital Image Processing: A remote Sensing Perspective, New Jersey: Prentice-Hall, 2005, 118-119.

*Tudor, Marius, “Are Flash Solid-State Disk Drives Ready for the Enterprise?,” Bit Micro Networks, Inc., 2009, 1 (accessed March

2, 2010).

The storage of remote sensor data can significantly impact system reliability and costs.

Substantial resources are required for even the most simple of systems. In light of this, mass

3 Marc Farley, Storage Networking Fundamentals: An Introduction to Storage Devices, Subsystems,

Applications, Management, and File Systems (Indiana: Cisco Press, 2007), 69.

6

storage media should permit rapid retrieval of required imagery data, provide longevity, and be

cost effective.4

MAGNETIC TAPE

Tapes are low cost and find frequent use as storage and backup media. They are linear

access devices as all data is written to or read from them sequentially.5 They come in several

types varying in size, storage capacity, density, length, thickness, number of tracks and reels, and

speed. They are composed of magnetic tape with single or dual reels contained in a plastic

enclosure.6

Data in older tape drive technology was written by multiple heads in a parallel track

across the entire tape with some drives using a helical scanning method which wrote the data

diagonally. A linear serpentine recording method is used for modern tape drives, which requires

more tracks and fewer tape drive heads. Data is written using the linear method except that data

is continually written with the head being adjusted and reversed once the end of the tape is

arrived at.13

Tape media is not without its drawbacks. Since access to data stored on magnetic tape is

linear or sequential, it is not considered efficient for random data access. The linear nature of

tape also makes recovery and back up operations time consuming. This is the reason why tape is

not considered for use as primary system storage, but is used mainly for offline data storage and

vaulting.7, 8

4 John R. Jensen, Introductory Digital Image Processing: A Remote Sensing Perspective, (New Jersey:

Prentice-Hall, 2005), 117.

5Shrivastava. Op. Cit. 269.

6Ibid. 33.

7Ibid.

7

Additionally, magnetic tapes slowly deteriorate over time developing surface cracks,

tears, and corrosion of the metal oxide coatings. Whether they are being used or stored for future

use, they should be maintained in environments having moderate temperatures and low

humidity.9 Improper maintenance (e.g., not rewinding the tape) and storage can cause magnetic

tape media to become unreadable within ten to fifteen years.10

OPTICAL DISKS

An optical disk is another recordable storage media which comes in a variety of types to

include compact disks (CDs), digital video/versatile disks (DVDs), magneto-optical (MO) disks,

and Blu-Ray disks (BDs).11 Optical disks comprise three broad categories, which determine their

usability. These categories are as follows12:

1. Read only optical disks, which are recorded when they are manufactured and cannot

be altered or erased. They include Compact Disk (CD), Compact Disk – Read Only

Memory (CD-ROM), Digital Versatile/Video Disk – Read Only Memory (DVD-

ROM), Digital Versatile/Video Disk – Video (DVD-Video), and Blu-Ray Disk (BD).

2. Write Once Read Many (WORM) optical disks can be recorded once and cannot be

erased. These include Compact Disk – Recordable (CD-R), Digital Versatile/Video

Disk – Recordable (DVD-R), and Blu-Ray Disk – Recordable (BD-R).

3. Rewriteable/Magneto optical disks, which can be written, erased and read from any

number of times. These include Compact Disk – Rewriteable (CD-RW), Digital

Versatile/Video Disk – Rewriteable, Blu-Ray, and magneto-optical (MO) disks.

Regardless of type, they are manufactured using similar technologies which incorporate a

thin polycarbonate disk which is impressed with microscopic bumps arranged in a continuous,

8Josh Judd, Principles of SAN Design: Design Build and Manage SANS, (Pennsylvania: Infinity Publishing,

2007), 18-19.

9Marc Farley, Storage Networking Fundamentals: An Introduction to Storage Devices, Subsystems,


10

John R. Jensen, Introductory Digital Image Processing: A remote Sensing Perspective, (New Jersey:


11Encyclopedia.com, "Optical Disk," The Columbia Encyclopedia, Sixth Edition, 2008,

http://www.encyclopedia.com/doc/1E1-optidisk.html (accessed March 4, 2010).

12Park, Oskar, “What is an Optical Disk?," Self SEO (Search Engine Optimization), September 10, 2006,

http://www.selfseo.com/story-18894.php (accessed March 11, 2010).

http://www.encyclopedia.com/doc/1E1-optidisk.html

http://www.selfseo.com/story-18894.php

8

spiral track of data. Following this, the disk is coated with a thin, reflective layer of aluminum

(barium ferrite for magneto-optical disks) to cover the bumps. A thin acrylic layer is then

sprayed over the metallic coating for protection.13, 14 Common optical disk media and their

typical storage capacities are shown in Table 2.

Table 2. COMMON OPTICAL STORAGE MEDIA.

OPTICAL MEDIA TYPICAL STORAGE

CAPACITY

REMARKS

CD/CD-ROM 700Mb Recorded at time of manufacture and cannot be

altered/erased

CD-R 650Mb Write Once Read Many (WORM) media. Once

written, can only be read from.

CD-RW 650Mb Data can be written/erased/read from the disk

any number of times.

DVD/DVD-ROM 9.4Gb

(4.7Gb per side)

Recorded at time of manufacture and cannot be

altered/erased

DVD-R 9.4Gb

(4.7Gb per side)

Write Once Read Many (WORM) media. Once


DVD-RW 9.4Gb

(4.7Gb per side)

Data can be written/erased/read from the disk


BD* 25Gb (single layer)

50Gb (double layer)

Recorded at time of manufacture and cannot be

altered/erased

BD-R* 25Gb (single layer)

50Gb (double layer)

Write Once Read Many (WORM) media. Once


BD-RE* 25Gb (single layer)

50Gb (double layer)



MO** 128Mb to 9.2Gb

depending on disk size



Park, Oskar, “What is an Optical Disk?," Self SEO (Search Engine Optimization), September 10, 2006,


*Wikipedia contributors, "Blu-ray Disc recordable," Wikipedia, The Free Encyclopedia,

http://en.wikipedia.org/w/index.php?title=Blu-ray_Disc_recordable&oldid=346269932 (accessed March 12, 2010).

**Wikipedia contributors, "Magneto-optical drive," Wikipedia, The Free Encyclopedia,

http://en.wikipedia.org/w/index.php?title=Magneto-optical_drive&oldid=338693032 (accessed March 12, 2010).

Optical disk systems write the data to the media using a low-power laser that etches

binary bits into the reflective layer. In this technique, the bits are heated to 150 degrees

centigrade, which are then realigned when a magnetic field is applied, creating a binary bit one.

13Encyclopedia.com, "Optical Disk," The Columbia Encyclopedia, Sixth Edition, 2008,


14Mediatechnics Systems, Inc, "How a CD is Made," Mediatechnics Systems, Inc., FAQ,

https://www.mediatechnics.com/cdfaqs.htm (accessed March 5, 2010).


http://en.wikipedia.org/w/index.php?title=Blu-ray_Disc_recordable&oldid=346269932

http://en.wikipedia.org/w/index.php?title=Magneto-optical_drive&oldid=338693032


https://www.mediatechnics.com/cdfaqs.htm

9

The recording of new data requires that existing bits be reset to a binary bit zero.15 To read

information from the disk, polarized light from a low-power laser is rotated according to the

direction of the magnetic field and the original binary signal is reproduced.16

Together with an optical disk drive, optical disks function much the same way as hard

disk drives. Access time components (of the optical disk drives laser seeking a target track and

acquiring the target sector ) are similar to the seek and rotational latencies present in hard disk

drives.17

The use of optical disks over magnetic storage media offers many advantages to include

higher storage capacity, lower cost, high data stability, environmental tolerance, and long shelf

life.18 The long-term storage potential exceeds 100 years and provides the means of storing large

amounts of data on a relatively small media footprint.19

Optical disks have a constant linear velocity which provides a constant read/write

bandwidth when access is sequential. This makes the media appropriate for multimedia backup

and restore operations, which are generally sequential in nature.20 As such, rewriteable CD-RWs

15R. Jensen, Introductory Digital Image Processing: A Remote Sensing Perspective, (New Jersey: Prentice-

Hall, 2005), 118.

16Daintith, John, "Magneto-optic Storage," A Dictionary of Computing. 2004, Encyclopedia.com,

http://www.encyclopedia.com/doc/1O11-magnetoopticstorage.html (accessed March 5, 2010).

17

Huseyin Simitci, Storage Network Performance Analysis (Indiana: Wiley Publishing, Inc., 2003), 123.

18

Park, Oskar, “What is an Optical Disk?," Self SEO (Search Engine Optimization), September 10, 2006,


19

R. Jensen, Introductory Digital Image Processing: A Remote Sensing Perspective, (New Jersey:


20Huseyin Simitci, Storage Network Performance Analysis (Indiana: Wiley Publishing, Inc., 2003).

http://www.encyclopedia.com/doc/1O11-magnetoopticstorage.html


10

and DVD-RWs have replaced tapes as the primary backup system in most remote sensing

laboratories.21

However, the recent advent of Blu-Ray, deemed as a future replacement for the DVD

format, offers even greater storage capacity. Having the same physical dimensions as CDs and

DVDs, a standard, double layer Blu-Ray disk can store up to 50GB of data. The Blu-Ray

standard is open-ended with theoretical storage limits left unspecified. Larger disk capacities of

100 and 200 GB are currently available.22

The use of optical disks is not without its drawbacks. In relation to hard disk drives, the

data access latencies are much higher in optical drives. The typical seek time for hard disk

drives is 10msec compared to 100-300msec for optical drives.23, 24

Constant improvements in hard disk drive technology (i.e., price, capacity, and speed)

make the decision to use optical disks less inviting. The development of DVDs did provide

some improvement in the storage capability, but applications are limited with many

organizations still relying on tapes and hard disk drives for business and scientific data storage.

However, the Blu-Ray disk may provide the impetus necessary to facilitate a greater move

towards the use of optical media for large database storage.25

21Jensen. Op Cit.

22

Wikipedia contributors, "Blu-ray Disc," Wikipedia, The Free Encyclopedia,

http://en.wikipedia.org/w/index.php?title=Blu-ray_Disc&oldid=349002833 (accessed March 12, 2010).

23

Huseyin Simitci, Storage Network Performance Analysis (Indiana: Wiley Publishing, Inc., 2003), 123.

24The Free On-line Dictionary of Computing, "Average Seek Time,"

http://encyclopedia2.thefreedictionary.com/average+seek+time (accessed March 12 2010).

25 Simitci. Op Cit.

http://en.wikipedia.org/w/index.php?title=Blu-ray_Disc&oldid=349002833

http://encyclopedia2.thefreedictionary.com/average+seek+time

11

MAGNETIC HARD DISKS

The Magnetic Hard Disk, also referred to as a Hard Disk Drive (HDD), is an

electromechanical device which controls the performance of the storage system environment,

providing the primary function of reading and writing the data that is stored on the media. They

are the primary storage medium used on computers for the storage and access of data and

software applications.26, 27

A magnetic hard disk consists of round, magnetic platters which encode digital data with

magnetically charged media. Each platter is arranged in cylinders/tracks with sectors on which

respective data is stored. The platters spin at a high rate of speed, up to 15,000 revolutions per

minute (RPM). Several platters, a read/write head, and a controller constitute the main

components of a HDD. The servo-controlled read/write head is attached to a rapidly moving arm

and is positioned over specific sectors to provide access to data. Since the HDD is a read/write

mechanism, data may be consistently written to and removed from the media.28, 29

HDDs provide quick and simultaneous access to arbitrary data locations and have large

capacities. Multiple disks may be configured into storage arrays providing increased capacity

and improved performance.30 In relation to other storage media like optical disks and tape,

26

Peter Alok Shrivastava and G. Somasundaram, eds., Information Storage and Management: Storing,

Managing, and Protecting Digital Information (Indiana: Wiley Publishing, Inc., 2009), 33.

27Marc Farley, Storage Networking Fundamentals: An Introduction to Storage Devices, Subsystems,


28Dane Nelson and Sam Siewart, PhD, “Solid State Drive Applications in Storage and Embedded

Systems,” Intel Technology Journal, Volume 13, Issue 1, 2009, 30,

http://download.intel.com/design/flash/nand/extreme/ITJ_03_SSDs_for_Embedded.pdf (accessed March 2, 2010).

29Peter Alok Shrivastava and G. Somasundaram, eds., Information Storage and Management: Storing,


30Ibid.

http://download.intel.com/design/flash/nand/extreme/ITJ_03_SSDs_for_Embedded.pdf

12

HDDs provide high data throughput and low latency data retrieval.31 However, as degradation

may occur in less than twenty years, HDDs have a comparatively short storage life. The

continual use of a HDD will reduce its life expectancy significantly.32

SOLID STATE DRIVES

Solid state drives (SSDs) use semiconductor flash memory chips for data storage and

retrieval. Unlike mechanical HDDs, they have no moving parts. All processes are operated via a

controller assembly that is also composed of semiconductor materials. SSDs have improved in

recent years and are used when high performance is required for mission-critical

applications.33,34

They use nonvolatile memory which supports persistent data storage and are constructed

in either single-level cell (SLC) or multi-level cell (MLC) configurations, which are used to store

data bits on respective memory cells. SLC storage is used in high performance memory and

stores one bit per cell. MLC memory stores multiple bits per cell providing slower data transfer

rates, but is cheaper to manufacture than SLC memory.35

SSDs using SLC technology and high reliability controllers imitate HDDs via a

traditional storage interface and provide ultra-fast read/write performance, high reliability, high

31Lawrence D. Bergman, and Vittorio Castelli, Image Databases: Search and Retrieval of Digital Imagery

(New York: John Wiley & Sons, Inc., 2002), 144.

32Jensen, John R. Introductory Digital Image Processing: A remote Sensing Perspective. (New Jersey:


33Peter Alok Shrivastava and G. Somasundaram, eds., Information Storage and Management: Storing,


34Dane Nelson and Sam Siewart, PhD., “Solid State Drive Applications in Storage and Embedded Systems,”

Intel Technology Journal, Volume 13, Issue 1, 2009, 30,


(accessed 03-02-10).

35




13

data integrity, and minimized power requirements. This makes them ideal for supporting

applications that require the rapid processing of large amounts of information such as real-time

data feeds.36

HHDs, on the other hand, use rotating media and servo-actuated read/write heads, which

introduce seek and rotation latencies. The device is also prone to mechanical failure. Their ever-

increasing capacity due to advances in the areal density of the recording surfaces and their low

cost are their main advantages, but fast random access limitations have always been an issue.37

As a rule of thumb, central processing units (CPUs) are able to process data much faster

than mechanical HDDs can transfer it. HDDs have been the mainstay of storage for decades

with a storage capacity that has increased over 200,000 times that of the first HDDs developed in

the 1950’s. During that same time period, decreases in price have also made the media more

affordable.38

However, HDD performance has not managed to keep up with processor development.

Within the last 30 years, processors have seen exponential increases in speed with only marginal

increases in the read/write response time of HHDs. This has resulted in a major break between

CPU access capabilities and those of HDDs.39

The performance differences between HDDs and SSDs concern the number of random

reads/writes per second that the device can perform and the time necessary to resume full

36 Ibid.

37

Dane Nelson and Sam Siewart, PhD., “Solid State Drive Applications in Storage and Embedded

Systems,” Intel Technology Journal, Volume 13, Issue 1, 2009,

http://download.intel.com/design/flash/nand/extreme/ITJ_03_SSDs_for_Embedded.pdf , 30 (accessed 03-02-10).

38Tom Coughlin, Neal Ekker, and Jim Handy, “Solid State Storage 101: An introduction to Solid State

Storage,” Storage Networking Industry Association (SNIA) Solid State Storage Initiative, January 2009,

http://www.snia.org/forums/sssi/knowledge/education/SSSI_Wht_Paper_Final.pdf , 1 (accessed March 13, 2010).

39 Ibid, 2.


http://www.snia.org/forums/sssi/knowledge/education/SSSI_Wht_Paper_Final.pdf

14

operational power from a low power state.40 The number of read/write operations that can be

completed in one second is called IOPs, or inputs/outputs per second. This metric is used to

measure the random read/write performance of different storage media. High performance

HDDs (15K RPM) normally perform 300 IOPs of random 4-kilobyte (KB) data.41 On the other

hand, SSDs are presently capable of processing 25,000+ 4KB IOPs, which is a tremendous

increase.42

Performance in HDDs is also affected adversely by the resumption of operation following

inactivity. When a HDD is inactive for a certain period of time, it will assume a low power state,

moving the read/write head off to the side and stopping the spinning platters. Upon the next

read/write request, the read/write head must be moved back in place and the platters must be

reactivated. This normally takes on the order of a few seconds to occur. However, when an SSD

is inactive and goes into a low power state, reactivation takes only a few milliseconds.43 The

following characteristics are indicative of solid state storage devices44:

1. Lowest possible access times: 100-1000 times faster than mechanical drives.

2. High bandwidth: provides multiple gigabytes (GB) per second of random data

throughput.

3. High IOPS: provides extremely high random input/output (I/O) performance due to

low access times and high bandwidth.

40



http://download.intel.com/design/flash/nand/extreme/ITJ_03_SSDs_for_Embedded.pdf, 30, (accessed 03-02-10).

41

Ibid.

42

Brook Crothers, “Seagate Enters Solid-State Drive Market,” CNet News, December 7, 2009,

http://news.cnet.com/8301-13924_3-10411097-64.html (accessed March 13, 2010).

43Nelson. Op. Cit.

44

Tom Coughlin, Neal Ekker, and Jim Handy, “Solid State Storage 101: An introduction to Solid State


http://www.snia.org/forums/sssi/knowledge/education/SSSI_Wht_Paper_Final.pdf, 10 (accessed March 13, 2010).


http://news.cnet.com/8301-13924_3-10411097-64.html


15

4. Low price for performance: provides the best possible price/performance of all

storage devices.

5. High reliability: provides the same levels of data integrity/endurance as other

semiconductor devices.

6. Provides more consistent I/O response times.

7. Provides predictable wear and lifespan characteristics.

These characteristics certainly favor solid state drives over mechanical hard disk drives.

However, barriers to the large-scale adoption of SSDs do exist. First, the cost of SSDs is more

than HDDs. These costs are decreasing with the increased demand by organizations that require

the performance, reliability, lower power, and/or resistance to shock and vibration that they

provide.45 Second, the capacity of SDDs is usually much smaller than that of mechanical HDDs.

However, flash density is improving rapidly to meet user demand for increased capacity with

200GB SSDs already commercially available.46,47 Finally, SSDs have a limited number of write

cycles per storage cell. After many program/erase operations, flash memory loses the capability

to retain data. However, increased density and improved write wear-leveling algorithms have

greatly improved the longevity of SSDs, with enterprise-grade SSDs providing as many as 1

million program/erase cycles.48,49

45



http://download.intel.com/design/flash/nand/extreme/ITJ_03_SSDs_for_Embedded.pdf, 31(accessed 03-02-10).

46Ibid.

47

Brook Crothers, “Seagate Enters Solid-State Drive Market,” CNet News, December 7, 2009,


48 Nelson. Op. Cit.

49

Jonathan Thatcher, “NAND Flash Solid State Storage Reliability and Data Integrity: An In-Depth Look,”

Storage Networking Industry Association (SNIA) Solid State Storage Initiative, 2009,

http://www.snia.org/forums/sssi/knowledge/education/Solid_State_Storage_Reliability_and_Data_Integrity--An_In-

depth_Look.pdf, 26 (accessed March 13, 2010).



http://www.snia.org/forums/sssi/knowledge/education/Solid_State_Storage_Reliability_and_Data_Integrity--An_In-depth_Look.pdf


16

Notwithstanding the shortcomings in SSDs, they are finding increased usage in

environments requiring enhanced response times to include high performance computing,

military applications, and data storage.50 The improving performance of SSD technology

permits them to complement or replace HDDs in computer, communications, and consumer

electronics. They have become the storage media of choice in avionics, industrial, medical, and

military equipment, which requires higher reliability under adverse mechanical and

environmental conditions.51

Direct attached storage (DAS) is typically used in aerospace, industrial, government, and

military applications, which require small form factor, durability, reliability, performance, and

the ability to endure extreme field environments. Currently, SSDs are the only practical solution

for these requirements, finding use in aircraft, weather balloons, spacecraft, missiles, ships,

submarines, trains, tanks and other armored vehicles, and portable military computers to name

but a few.52

The increasing requirements for reliability and performance in military, industrial, and

business applications is driving standards for data storage to extraordinary levels with faster

access, mobility, and extreme reliability becoming more imperative. As the performance

limitations of HDDs are reached and the costs of SSDs continue to decrease, SSDs will play a

more critical role in an environment that demands faster, more reliable data storage.53

50Tom Coughlin, Neal Ekker, and Jim Handy, “Solid State Storage 101: An introduction to Solid State


http://www.snia.org/forums/sssi/knowledge/education/SSSI_Wht_Paper_Final.pdf , 10 (accessed March 13, 2010).

51Marius Tudor, “Are Flash Solid-State Disk Drives Ready for the Enterprise?,” BitMicro, 2009,

http://www.bitmicro.com/press_resources_flash_ssd_enterprise.php, 1 (accessed March 2, 2010).

52 Ibid. 3.

53Ibid.


http://www.bitmicro.com/press_resources_flash_ssd_enterprise.php

17

At the present, the mechanical HDD is the main obstacle to storage performance today,

due to rotational and seek actuation latencies when accessing data. This problem is complicated

when data access is random and of small, distributed I/O. Most hard drives can deliver from a

few hundred to around 100 MB/sec of IOPs when accessing sequential large blocks of data.54

However, this issue can be alleviated via two methods. The first method involves the

direct replacement of HDDs with SSDs, which can provide a tenfold increase in performance.55

The second method involves the use of a Redundant Array of Inexpensive/Independent Disks

(RAID), which can improve HDD performance by allowing concurrent access to data over an

entire array of disks, thereby increasing I/O throughput.56 As such, it is expected that HDDs will

see continued usage where high capacity storage is necessary and SSDs will be used where

higher performance capability is required.57

REDUNDANT ARRAY OF INEXPENSIVE/INDEPENDENT DISKS (RAID)

The term “RAID” stands for “Redundant Array of Inexpensive (or Independent) Disks,”

and it is one of the most important technologies in storage networking. RAID is often associated

with hardware such as disk subsystems and RAID adapters, but it is actually a set of software

algorithms that combine storage input/output (I/O) operations across multiple storage address

locations. RAID is usually applied with disk drives in disk subsystems, but it can also be applied

54Dane Nelson and Sam Siewart, PhD., “Solid State Drive Applications in Storage and Embedded Systems,”

Intel Technology Journal, Volume 13, Issue 1, 2009,

http://download.intel.com/design/flash/nand/extreme/ITJ_03_SSDs_for_Embedded.pdf, 38 (accessed 03-02-10).

55

Ibid.

56Ibid.

57

Tom Coughlin, Neal Ekker, and Jim Handy, “Solid State Storage 101: An introduction to Solid State


http://www.snia.org/forums/sssi/knowledge/education/SSSI_Wht_Paper_Final.pdf, 10 (accessed March 13, 2010).



18

across multiple disk subsystems in a storage network. It should be noted that these algorithms

are autonomous and apply to any storage media as long as the storage capacities are the same.58

Original research defined five levels of RAID with varying characteristics of

performance, capacity, and redundancy. Of these, levels one and five are the most common.

RAID Level 1 (RAID-1) encompasses simple disk mirroring without the use of parity or striping.

This level does not provide the benefits of performance and scalability related to other RAID

levels. RAID Level 5 (RAID-5) calculates parity values for stored data resulting in more

efficient data redundancy than mirroring.59 Present day has seen the development of other RAID

levels. Table 3 describes the current RAID levels and their main characteristics.60

Table 3. RAID LEVELS.

RAID Type Characteristics

RAID 0 Striping, no redundancy

RAID 1 Mirroring, full redundancy

RAID 10 Mirrored stripes, full redundancy

RAID 2 ECC protection, not used

RAID 3 Byte-interleaved parity, single parity disk

RAID 4 Block-interleaved parity, single parity disk

RAID 5 Block-interleaved parity, rotated parity blocks

RAID 5DP RAID 5 with double parity blocks per stripe Huseyin Simitci, Storage Network Performance Analysis (Indiana: Wiley Publishing, Inc., 2003), 117.

RAID provides four main benefits for data storage. These are as follows61:

1. Data redundancy.

2. Large capacity storage.

3. Management consolidation of devices and subsystems.

4. Parallel processing for enhanced performance.

58

Marc Farley, Storage Networking Fundamentals: An Introduction to Storage Devices, Subsystems,


59Ibid. 196.

60Huseyin Simitci, Storage Network Performance Analysis (Indiana: Wiley Publishing, Inc., 2003), 117.

61

Farley. Op. Cit. 193.

19

The main building block of the RAID system is the array. RAID architecture allows

combining multiple storage entities, called array members, into a single array that functions as a

single, virtual storage device. A RAID can have two or more arrays assembled from member

disk partitions as shown in Figure 1.62

3 U

RAID

CONTROLLER

MEMBER

MEMBER

MEMBER

MEMBER

MEMBER

ARRAY MEMBER

DISK PARTITIONS

Figure 1. RAID ARRAY WITH DISK PARTITION MEMBERS.

Marc Farley, Storage Networking Fundamentals: An Introduction to Storage Devices, Subsystems,


The use of RAID in image and video servers provides fault tolerance, which is the

primary concern for imagery/video databases. These systems attain fault tolerance by either disk

mirroring or parity encoding. Disk mirroring provides fault tolerance by the duplication of data

62Ibid. 192.

20

onto separate disks. Parity encoding techniques use error-correcting codes to reduce storage

overhead.63

In disk mirroring, data is stored on two separate disks creating two copies of the data – a

mirrored pair, as shown in Figure 2. If one disk fails, the data remains intact on the other

(redundant) disk and the controller continues to provide requested data from the surviving disk.

Replacement of the failed disk automatically instructs the controller to copy data from the

remaining disk to the new one – an operation transparent to the host system.64

4

2 2

1

4

3

1

3

DATA 1

DATA 2

DATA 3

DATA 4

MIRRORING

Figure 2. MIRRORING.



Mirroring provides complete data redundancy and enables faster recovery time following

disk failure. On the other hand, it only provides data protection and is not to be confused as a

63Lawrence D. Bergman, and Vittorio Castelli, eds., Image Databases: Search and Retrieval of Digital

Imagery (New York: John Wiley & Sons, Inc., 2002), 145.

64



21

substitute for data backup. In this instance, mirroring continuously captures data changes, as

opposed to backups, which capture point-in-time images of the stored data.65

The amount of required storage is twice the amount of data needing to be stored, so

mirroring is considered expensive, being used on mission-critical applications which cannot

suffer data loss. The technique does provide improved read performance since read requests can

be read from both disks. However, there is an obvious deterioration in write performance since

each write request manifests itself as two writes on the disks.66

To understand the parity method of fault tolerance, the concept of “striping” must be

explained. In a RAID array, each disk has a predefined number of contiguously addressable

blocks called “strips.” The series of aligned strips spanning across all of the disks in the array is

called a “stripe.”67 This division and distribution of data across multiple disks is called

striping.68

RAID using this technique does not provide data protection unless mirroring or parity is

used in the process. However, use of striping can considerably increase I/O performance if the

RAID controller is configured to simultaneously access data across multiple disks.69

Parity is a mathematical technique used to re-create missing data. Parity RAID protects

striped data from disk failure through the addition of an extra disk. This disk is added to the

65Ibid. 55.

66Ibid. 56.

67Ibid. 54.

68Ibid. 433.

69Ibid. 54.

22

stripe width and is used to hold parity, which provides a redundancy check that ensures

protection of data without having to maintain a full set of duplicate data.70

As opposed to mirroring, the use of parity significantly reduces data protection costs. For

example, in a RAID configuration using five disks (Figure 3), one disk retains the parity

information while the other four retain data. In this case, parity would require only 25 percent

extra disk usage compared to mirroring, which would need 100 percent extra disk usage.71

However, the use of parity is not without its disadvantages. Since parity information is

created by the data currently residing in the disk, it is recalculated for every change in data. This

process is time consuming and negatively affects the performance of the RAID controller.72

3

13

1

3

1

11 2

2

3

1

2

1

1

2

7

5

7

9

PARITY DISK

DATA DISKS

Figure 3. PARITY RAID.



Additionally, in the case of a disk failure, the load increase on the surviving disks results

in deadline violations during video stream playback operations. To prevent this occurrence using

70Ibid. 56.

71Ibid. 56-57.

72Ibid.

23

conventional fault-tolerance methods, disk use must be decreased during the fault-free state.

This reduction may be accomplished prior to imagery compression by using partitioning

techniques that take advantage of the characteristics of imagery through the use of two general

methods.73

First, the sequential nature of video access may be used to reduce the system overhead of

on-line recovery in a RAID. This may be accomplished by the computation of parity over a

video stream block sequence, ensuring that retrieved data used to recover a block that was stored

on the failed disk would be requested by the system in the future. The blocks are temporarily

stored in and serviced from the buffer, minimizing the time necessary for the failure recovery

process.74

Second, since human perception is tolerant to minor image distortion, the inherent

redundancies in imagery may be used to reconstruct lost imagery data to a fairly accurate degree.

This method uses error-correcting codes to accomplish this. The imagery is divided into sub-

images that are stored across the array members. The failure of a single disk will result in

fractional losses for several images.75

However, if the sub-images are generated using pixels, and none of the neighboring

pixels belong to the same sub-image, all elements of the lost pixels will be accessible in the event

of a single disk failure. The relationship between the neighboring pixels will allow for an

73

Lawrence D. Bergman, and Vittorio Castelli, eds. Image Databases: Search and Retrieval of Digital


74Ibid.

75Ibid.

24

approximate reconstruction of the original imagery without the need for additional information

from any of the other surviving disks members.76

It should be noted that use of the aforementioned techniques will adversely affect the

compression of the imagery due to the reduction of the correlation between pixels allocated to

the same sub-images. This will increase the bit rate, imposing greater load requirements on all

disks in the RAID and reduce the number of video streams that may be retrieved

simultaneously.77 Post-compression partitioning algorithms have been developed to address this

limitation and are discussed in the following section.

76Ibid.

77Ibid.

25

CHAPTER 2

INTRODUCTION TO VIDEO COMPRESSION

Present day imagery technology has provided an endless amount of photographic and

video data, which presents a major challenge in its capture, processing and storage. As this

technology has evolved, the need to improve the quality and to reduce the scale of the imagery

has become of primary importance. This is particularly true since the development of high

resolution photography and video has created more complex files, which can be of extremely

large size, requiring more resources to use. This ever-growing database has required the

development of techniques to control the size and complexity of the imagery and to allow ease of

use. The science of digital compression technology was born out of this need.

Compression is the science of reducing the amount of data used to convey information

about an object. Because information has order and patterns, these can be extracted and

reconstructed to provide the fundamental nature of the original information using less data for

transmission and reception.78 Certain restrictions are required for the compression of

multimedia. The encoded/decoded data should provide the best possible quality. Complexity

should be minimal to provide limited data link delay and cost-effective implementation. Modern

compression techniques must compromise between these requirements.79

The goal of compression technology is the elimination of redundant data while retaining

only the information that is essential for the effective reproduction of the original data. In other

words, the number of bits represented by a signal is reduced to provide a smaller data file that

78Peter Symes, Digital Video Compression (New York: McGraw-Hill, 2004), 2.

79Wolfgang Effelsberg and Ralf Steinmetz, Video Compression Techniques, (Netherlands: KoninKlijke

Wöhrmann B.V., 1998), 45.

26

still represents the fundamental nature of the original data. This technique is accomplished using

either “lossless” compression, “lossy” compression, or a combination of both methods.80

Lossless data compression algorithms permit exact reconstruction of original data

following compression and allow for compression ratios of about four to one.81, 82 It is used

when it is critical that the original data be identical following decompression. This is important

for executable files, source code, and some image file formats like portable network graphics

(PNG) and graphics interchange format (GIF). It has many applications to include ZIP utilities,

and is often used in conjunction with lossy compression methods.83

As opposed to lossless compression, lossy compression does not allow for complete

recovery of the original signal. Only a near approximation is available due to the tradeoff

required to provide reasonable compression of the signal.84 Compression ratios ranging from

eight to one and twenty to one can provide images that are visually comparable to the original. It

is possible to produce higher ratios for lossy compression, but a noticeable difference will exist

between the original and the compressed image.85 The most common use of lossy compression

80Lawrence D. Bergman, and Vittorio Castelli, eds. Image Databases: Search and Retrieval of Digital


81Ibid.

82Wikipedia contributors, "Lossless data compression," Wikipedia, The Free Encyclopedia,

http://en.wikipedia.org/w/index.php?title=Lossless_data_compression&oldid=346709055 (accessed March 10,

2010).

83Ibid.

84Ibid.



27

is for the compression of multimedia data like audio, video, and still imagery. So, it finds greater

use in streaming media and internet telephony applications.86

A standard image compression method consists of three components: pixel-level

redundancy reduction, data discarding, and bit-level redundancy reduction. The normal image

compression process is shown in Figure 4. Note that lossless image compression systems do not

use the data discarding process as compression is achieved solely from the redundancy reduction

processes. Lossy image compression generally uses all three processes, but may omit bit-level

redundancy.87

Figure 4. IMAGE COMPRESSION COMPONENTS.

Lawrence D. Bergman, and Vittorio Castelli, eds. Image Databases: Search and Retrieval of Digital Imagery (New

York: John Wiley & Sons, Inc., 2002), 212.

The pixel-level redundancy process reverses the mapping of the image, disassociating the

original pixels from the output. This process enumerates the image pixilation (converts it to data

bits) in a way that more corresponds to the human visual system frequency response. This data

is then transmitted to the data discarding process.88

The data discarding process removes insignificant data from the original data stream that

has been divided into separate data bits during the pixel-level redundancy process. It then

discards the data bits that are deemed unnecessary, and retains those that are minimally required

86Wikipedia contributors, "Lossy compression," Wikipedia, The Free Encyclopedia,

http://en.wikipedia.org/w/index.php?title=Lossy_compression&oldid=347940008 (accessed March 10, 2010).

87Op Cit. 212.

88Ibid.

28

to produce a near-perfect rendition of the original, without loss of image fidelity. Statistical

image properties and human visual characteristics are considered in this approximation. This

data is then sent for bit-level redundancy reduction for final processing.89

The bit-level redundancy reduction process is considered a lossless process. It is used to

remove and/or reduce any unnecessary dependencies left over from the data discarding process.90

Following compression of the imagery by one or more compression standards such as those

developed by the Motion Pictures Experts Group (MPEG), the imagery is then sent to the

applicable storage location.

VIDEO COMPRESSION STANDARDS

Several video compression techniques exist that use different techniques to reduce data

size.91 The most commonly used family of digital video compression techniques is the Motion

Pictures Experts Group (MPEG) standards, which are briefly described in Table 4.

Table 4. MPEG STANDARDS.

STANDARD OBJECTIVE CHARACTERISTICS APPLICATIONS

MPEG-1 To provide a standard for encoding motion video at bit rates transportable over T1 data circuits and for

replay on CD-ROM.

Provided audio and video recording at the same

data rate.

Broadcasting in any form and

large-distribution CD-ROMs.

MPEG-2 To provide a standard that included broadcast

quality video.

Provides video interlacing, scalable syntax, a

system layer to handle multiple program

streams.

Most popular standard presently

in use. Used for HDTV and for compression of other

video/audio data.

MPEG-3 To provide a compression system suitable for high-

definition television (HDTV) broadcasting. Abandoned when determined that MPEG-2 was

able to accommodate the need for HDTV. N/A

MPEG-4 To provide for encoding of video and audio at very

low bit rates.

Provides for the combining of multiple video streams. Increased efficiency and error

forgiveness over MPEG-2.

Provides end-user interaction

with specific applications like

games, interactive TV, and educational systems.

MPEG-7 To provide a standard for applying metadata

technology to recorded video and audio.

Uses metadata technology to catalog/ index

video/audio files to accommodate search engine retrieval.

In work, but will provide ease of

retrieval of data in large-scale databases.

MPEG-21 To provide a complete structure for the management

and use of digital assets.

Provides all infrastructure support for

commercial transactions and rights

management.

In work, but will provide

intellectual property

management and protection.

Peter Symes, Digital Video Compression (New York: McGraw-Hill, 2004), 152,172, 196, 268, 276.

89Ibid.

90Ibid.

91

Peter Symes, Digital Video Compression (New York: McGraw-Hill, 2004), 2.

29

MPEG is a committee that was formed in 1988 under the Joint Technical Committee of

the International Standards Organization (ISO) and the International Electro-technical

Commission (IEC). Their formation was facilitated to derive a standard for encoding full motion

video at rates suitable for transport over T1 data circuits and for replay from CD-ROM at about

1.5Mbits/sec. This was the birth of the MPEG-1 standard, which ultimate goal was to record

sampled audio at 48 kHz together with video, at the same data rate.92

Following the development of MPEG-1, a standard was required that allowed for the

compression, storage, and digital transmission of television broadcast quality video. This project

became MPEG-2, which extended and improved MPEG-1.93 Among other things, MPEG-2

included the provision of video interlacing, a scalable syntax, and the addition of a system layer

to handle multiple program streams. In light of this, MPEG-2 has become the worldwide

standard for both standard- and high-definition transmissions.94

After development of MPEG-2, a standard that would support compression suitable for

high-definition television (HDTV) was deemed necessary. This effort became MPEG-3.

However, it was discarded very early into the program when it became clear that the new

compression algorithms and schemes used in MPEG-2 were capable of handling the HDTV

requirement.95

With the demise of MPEG-3, MPEG-4 was started to accommodate the encoding of

video/audio at very low data rates. Later amendments also provided applications that allowed

92Ibid. 152.



94Symes. Op. Cit. 172-173.

95Effelsberg. Op. Cit. 45.

30

end-user interaction with the video. This was particularly useful for gaming, interactive

television, educational systems, and high-end data retrieval systems. However, the standard has

not seen wide acceptance due to the popularity of MPEG-2.96

The current amount of digital video and audio information presently available in the

public domain is enormous and continues to grow. To sort, maintain, and access required

information in a timely manner, MPEG-7 has been proposed and may provide the future means

by which digital video is cataloged, stored, and retrieved. The MPEG-7 standard uses metadata

technology to both catalog and index audio-video files using methods that complement data

retrieval.97

The latest effort in video compression by the MPEG organization is MPEG-21, which

also accommodates metadata technology. The standard was undertaken to provide a complete

framework for the use and management of digital resources. Infrastructure support for

commercial transactions and rights management is also an important part of the standard, which

focus is “to enable transparent and augmented use of multimedia resources across a wide range

of networks and devices.”98

METADATA

To better understand what the MPEG-7 and MPEG-21 standards can provide to the

organization and retrieval of multimedia data, the concept of metadata requires further

explanation. Metadata is the “data about the data” or the “information about the information.” It

96Op. Cit. 196, 222.

97Ibid. 268-269.

98Ibid. 276.

31

describes the various attributes of a given piece of information by providing meaning and

context. It also aids in the location, retrieval, use and management of the information.99, 100

Metadata provides many benefits to include101:

1. Aids in the search for required information.

2. Aids in organization of digital resources.

3. Facilitates interoperability and legacy resource integration.

4. Provides digital identification.

5. Supports data archiving and preservation.

Typical remote sensing systems have enormous amounts of imagery that must be

maintained. To keep track of this data, information about the images is necessary. This

metadata can be stored with the imagery or in a separate database using pointers to link the

applicable imagery with its respective data set. Various metadata may be used in search and

retrieval operations while other metadata may provide user information.102

Metadata, as used in remote sensing systems, consists of several characteristics with

some elements associated with a collection of images and others that specify smaller subsets

called “granules,” which can be a single image or a small set of images. Examples of these

characteristics are as follows103:

1. General Description, which includes the name, topic, collection version, etc.

2. Data Origin, which describes the platform, instrument, sensor used, etc

3. Spatial Coverage, which specifies the geographic area covered by the imagery in the

collection (e.g., location coordinates of the corners of a bounding rectangle).

99

Cornell University Library, “Moving Theory into Practice: Digital Imaging Tutorial, Chapter 5,

Metadata: Definition, Types, and Functions,”

http://www.library.cornell.edu/preservation/tutorial/metadata/metadata-01.html (accessed January 14, 2010).

100National Information Standards Organization, Understanding Metadata (Maryland: NISO Press, 2004),

http://www.niso.org/publications/press/UnderstandingMetadata.pdf (accessed January 20, 2010), 1.

101Ibid.



103Ibid.

http://www.library.cornell.edu/preservation/tutorial/metadata/metadata-01.html

http://www.niso.org/publications/press/UnderstandingMetadata.pdf

32

4. Temporal Coverage, which specifies the time range during which the imagery was

obtained (e.g., start/end date/time).

Standards for the collection of geographic data have been established by the U.S. Federal

Geographic Data Committee (FGDC). All geographic data that is managed by federal

government agencies must comply with this standard. The standards serve as a guideline for the

use of metadata in remote sensing applications.104 The ISO has also established a working group

on metadata for the purpose of developing standards for the recording of metadata on a separate

track than the audio-video. This would provide a powerful means of retrieval for later access to

the digitized data and illustrates the importance of the MPEG-7 and MPEG-21 standards.105 The

rest of this section will discuss metadata in regards to the MPEG-7 standard.

The scope of the MPEG-7 standard is illustrated in Figure 5. From the figure, it is clear

that only the syntax (Standard Description) is standardized. Devices that might generate

metadata and how that data is represented are specified by the standard. Generation of metadata

(i.e., Feature Extraction) and the applications that might use it (i.e., Search Engines) are

unspecified and left to commercial developers to provide.106 Several groups are presently

developing video content analysis algorithms to automatically extract semantic information from

video data with the hopes of partially automating the metadata creation process.107

104Ibid. 63.



106

Peter Symes, Digital Video Compression (New York: McGraw-Hill, 2004), 270.

107Effelsberg. Op. Cit.

33

FEATURE EXTRACTION STANDARD DESCRIPTION SEARCH ENGINE

MPEG-7 SCOPE

Figure 5. SCOPE OF THE MPEG-7 STANDARD.

Wolfgang Effelsberg and Ralf Steinmetz, Video Compression Techniques, (Netherlands: KoninKlijke Wöhrmann

B.V., 1998), 58.

Assuming metadata is recorded simultaneously with the audio-visual data and timing

information, search engines can be developed that can access specific audio or video data from

large digital archives or databases.108 This timely access to required material can provide

immediate use – an important consideration especially for mission critical applications.

The generation of metadata poses two main issues. The first is the amount of data that is

generated. Human interaction in the indexing process will result in only a fraction of the total

created content being indexed. In this case, the automated analysis of digital records is critical

and presents many technical challenges. The second issue regards the standardization of

metadata descriptions. The situation has improved with the widespread availability of

information.109

Conversely, practical application of metadata has been made problematic by the creation

of different standardization methods. The MPEG-7 standard calls for extensible markup

language (XML) schemas as opposed to the Society of Motion Picture and Television Engineers

(SMPTE) which uses key-length-value (KLV) coding. Other organizations have created their

108Ibid.

109Peter Symes, Digital Video Compression (New York: McGraw-Hill, 2004), 270.

34

own metadata sets that are based on their own needs. This proliferation of different metadata

methodologies creates interoperability issues, which was never the intent of MPEG-7. Efforts

are being led by MPEG and SMPTE to synchronize the different methodologies to allow

portability between them.110

MULTIMEDIA PLAYERS AND CONTAINER FORMATS

This discussion is not complete without providing a clear distinction between multimedia

players and multimedia container formats. Multimedia players are the software applications that

are used to playback video and audio data files. There are many of these applications available

to include Windows Media Player®, Real Player®, and QuickTime®, to name but a few.

Multimedia container formats provide for the actual formatting of the recorded video and audio.

Different methods exist and are signified by the actual data file extensions. These include, but

are not limited to the following format types: AVI, ASF, and MOV.

Many of the currently fielded video collection systems use digital video recorder utilities

that apply the audio video interleave (AVI) container format for playback operations. Playback

of applicable video is then performed using one of the many available multimedia players like

Windows Media Player®. The AVI format is proven over some years of usage and has continued

to be supported by many multimedia players. For purposes of this paper, only the AVI container

format will be discussed.

The AVI format has its origins in the Resource Interchange File Format (RIFF), which

divides the data into “blocks” that are each identified by a tag. The AVI file is a single block,

RIFF formatted file that is further divided into two mandatory sub-blocks and one optional sub-

block. The first sub-block is the file header, which contains video metadata such as frame width,

110Ibid. 275-276.

35

height and rate. The second sub-block contains the actual audio-visual data that is the AVI

movie file. The third, optional, sub-block is used to index data block offsets within the file.111

Via the RIFF format, the sub-block containing the actual audio-visual data is encoded

and/or decoded by a software algorithm called a “codec,” an abbreviation for (en) coder/decoder.

Upon file creation, the codec interprets between raw data and the compressed data format used

within the sub-block. AVI files can hold audio-visual information inside the data blocks in

almost any compression scheme to include full frame (uncompressed), Motion JPEG, editable

MPEG, and MPEG-4 to name a few.112

AVI was not originally intended to contain compressed video, which requires access to

video frame data beyond the currently used frame. Methods do exist that support modern video

compression use within the AVI framework (MPEG-4, et. al.), even though this is outside the

scope of the original specification. Problems can occur during playback with utilities that do not

anticipate any issues like not having the correct codec required to play the video file.113

However, most audio-video files can be reviewed for playback provided the necessary

codec files are installed on the computing machinery that is being utilized for this purpose. As

such, the AVI format continues to see major use in both commercial as well as military

applications.

The previous sections have described some of the concepts related to data storage. In

particular, storage media, data redundancy, and video compression technologies were discussed

111Wikipedia contributors, "Audio Video Interleave," Wikipedia, The Free Encyclopedia,

http://en.wikipedia.org/w/index.php?title=Audio_Video_Interleave&oldid=347331467 (accessed March 10, 2010).

112Ibid.

113Ibid.

36

in relation to the large scale storage of recorded audio-video data files. The discussion will now

turn to possible solutions to the storage of this data in relation to tactical video acquisition.

37

CHAPTER 3

RECOMMENDATIONS

HARDWARE CONSIDERATIONS

Although somewhat outside the original scope of this paper, it is important that the

system developer/user consider the computing machinery to be used in the tactical video

collection system. The personal computer (PC) has advanced to the point that powerful, robust

systems are available commercially at a fraction of the cost of high performance systems from

years past. Most PCs available for commercial-off-the-shelf (COTS) purchase are quite suitable

for digital video recording functions. Notwithstanding the obvious need for robust computing

machinery, given the critical nature of tactically fielded video collection systems, some minimal

hardware recommendations are as follows:

1. Robust PC-based system using a dual-processor, 64-bit bus motherboard with large

random access memory (RAM) capacity.

2. Best commercially available video graphics adapter using the most video RAM.

3. Main internal storage consisting of solid state hard drives (SSDs).

4. Redundant backup using an external (or internal) RAID-based system.

5. CD-RW/DVD-RW/Blu-Ray-RW for archiving purposes.

Following these deliberations, the primary, internal system storage should consist of solid

state drives (SSDs) of sufficient size to accommodate the operating system (OS), applicable

system and recorder applications, and onboard storage. The use of SSDs will allow for enhanced

system performance during mission critical operations. External (or internal) RAID storage is

recommended using standard hard disk drives, which provide more efficient mass storage

properties and should be sufficient for normal backup operations.

Regarding archival storage, the use of Blu-Ray optical disks is recommended given both

the large capacity and the storage longevity of optical disks. Compared to other optical storage

mediums as well as magnetic tape, Blu-Ray best accommodates the requirements necessary for

38

the storage of large amounts of captured video data, and it will also reduce hard storage

requirements. A simple hierarchy depicting these guidelines is shown in Figure 6 followed by

brief explanations for each level.

LEVEL 3

ANCILLARY STORAGE

(STORED ON-SITE)

LEVEL 2

SECONDARY STORAGE

(EXTERNAL/INTERNAL RAID)

LEVEL 1

PRIMARY STORAGE

(INTERNAL)

Figure 6. SIMPLE STORAGE HIERARCHY.

LEVEL 1:

Primary storage, internal to the computing machinery and using flash Solid State Drives (SSDs)

for their enhanced performance and data retrieval.

LEVEL 2:

Secondary storage for database back up (automated or manual) using RAID-based architecture

and consisting of mechanical Hard Disk Drives (HDDs) for their mass storage benefits.

LEVEL 3:

Ancillary storage using Blu-Ray optical disks for their large storage capacity, environmental

durability, and long-term storage properties.

39

SOFTWARE CONSIDERATIONS

Once the computing machinery is in place, the system developer/user must next consider

the software operating system (OS) and the required application software to be used with the

system. The Windows® OS is recommended for its ready availability and widespread use. The

digital video recorder (DVR) application may be selected from open or proprietary sources. It

should be robust, and its selection based on system considerations. In addition to its recording

function it must also provide post-mission playback and video editing utilities.

For backup operations, an automated backup and restore system utility is recommended.

It should have a built in scheduling capability and operate in the background, transparent to the

host system. It should provide the capability for different types of backup operations, the most

common of which are explained in Table 5. For mission critical operations, it is recommended

that Full backups be performed to provide complete restoration of all data in case of corruption

or catastrophic system failure. The utility should also provide the ability to automatically

overwrite older video data files when necessary, in the interest of freeing up hard drive space.

This means that any video that is intended to be kept for planning or historical purposes must be

manually copied and archived on separate media, preferably Blu-Ray optical disk.

Table 5. BACKUP OPERATIONS.

BACKUP

TYPE OPERATION CHARACTERISTICS

Full Provides full backup of all data. Time intensive, but provides all data for complete

restoration.

Incremental Only copies files that have changed since last

full backup.

Assuming weekly backups, subsequent backups

may be larger.

Differential Only copies data that has changed since last

differential backup.

Shortens backup time, but may increase

restoration time.

Snapshot Point-in-time copy of the data, also called

pointer-based backup. Copies pointers and file

metadata.

Very fast backup - copies only metadata and

pointers that point to the data blocks. Does not

protect against disk loss.

Schulz, Greg, Resilient Storage Networks: Designing Flexible Scalable Data Infrastructures (Massachusetts:

Elsevier Digital Press, 2004), 324.

40

A suggested backup scheme is shown in Figure 7. It is recommended that a Master

System disk be produced on Blu-Ray optical media following final system setup. This should be

maintained on site and will provide a backup in case the magnetic hard disk drive needs to be

rebuilt due to file corruption or catastrophic system events.

Additionally, a Full system backup that is stored in the external RAID storage assembly

should be scheduled as required to intermittently capture the present system configuration. This

will allow the user to recover system operation quickly by using a previously known working

system configuration. This is also useful in the event of system file corruption or other events

that may affect the system operation.

Recorded video should also be captured in the external RAID storage area to keep the

internal, primary storage medium free of unnecessary files that may adversely affect the overall

system performance. The RAID storage capacity should be in the terabyte range to

accommodate the large amount of data that will be generated by the video collection system. At

a minimum, RAID-1 architecture providing mirroring and full redundancy is recommended.

41

Figure 7. BACKUP SCHEME.

Given the nature of real-time, tactical video collection systems, large amounts of data

are produced within a short period of time. In the interest of freeing up storage space for the

recording of newer video, it is recommended that any video that is desired to be retained for

historical reference should be archived on optical media and stored on-site to facilitate its later

usage. A master system copy stored on optical disk should also be maintained for recovery

operations due to system corruption and/or catastrophic system failure. This should also be

maintained on-site and be updated as required.

VIDEO RETENTION AND DELETION SCHEMES

Tactical operations are defined as “military operations conducted on the battlefield,

generally in direct contact with the enemy.”114 Tactical data collection is indigenous to the area

of operations in which the tactical collection asset is fielded. Given the nature of tactical

114Tactical Operations. Answers.com. The Oxford Essential Dictionary of the U.S. Military,

Oxford University Press, 2001, 2002. (accessed March 07, 2010).

42

intelligence data such as real-time video, the usefulness of the data may diminish quickly within

a very short period of time.

However, this data is still retained in storage although it may not provide any useful

intelligence value. This is generally true for most video that is collected over time and the

burden on storage assets can be significant. This section outlines a selection process that may be

used to determine automatic retention and/or deletion of tactically collected video.

Figure 8 is an example of a simple, software driven selection process, which could use

criteria such as the number of times a file has been accessed within a given period of time to

determine retention or deletion. It might also be based on guidelines established by the

command to determine the long-term or short-term tactical intelligence value of specific video.

RAW VIDEOVIDEO

PROCESSING

VIDEO

DATABASEVIDEO FILE

VIDEO

SCREENING

PROCESS

RETENSION

DELETION

NO

TACTICAL

VALUE

TACTICAL

VALUE

Figure 8. A SIMPLE VIDEO RETENTION/DELETION SCHEME.

A more in depth selection process is shown in Figure 9. The displayed retention times

shown are arbitrary examples. The automatic deletion of stored video imagery would rely on

specific, established criteria to determine whether a file is a candidate for deletion. At a

minimum, criteria might include the date/time stamp (from metadata) and the number of times

the specific files have been accessed, which denotes level of importance. However, user

43

intervention might be required to determine the retention of specific video that contains

significant events such as:

1. Reconnaissance operations.

2. Surveillance operations.

3. Intelligence gathering operations.

4. Mission operations support.

5. IED incidents.

6. Counter-IED operations.

7. Area/terrain observations in support of missions.

8. Areas of interest (AI).

9. Any video showing direct/indirect contact with an enemy.

10. Other significant events that are deemed to provide intelligence value of a tactical

nature, as determined by the command.

VIDEO

DATABASEPROCESSED VIDEO VIDEO FILES

RETENTION

SCREENING

ALGORITHM

CRITICAL

VALUE

HIGH VALUE

MEDIUM

VALUE

LOW VALUE

NO VALUE

TAGGED VIDEO

RETAIN INDEFINITELY

RETAIN 6-12 MONTHS

RETAIN 3-6 MONTHS

RETAIN 1-3 MONTHS

RETAIN 30 DAYS

TAGGED

VIDEO

3-6MOS6-12MOS

1-3MOS

AUTOMATED

DELETION

ALGORITHM

30 DAYS

DELETED VIDEO

INDEFINITE

RETENTION

Figure 9. A MORE ROBUST VIDEO RETENSION/DELETION SCHEME.

Additionally, unless it is used for planning purposes or as an historical reference in

support of future operations, most tactically captured video may have a short shelf life for

44

providing up to date intelligence information. Of course, this is dependent upon the intelligence

requirements and the value placed on specific video by the command.

FUTURE CONSIDERATIONS

Current digital video recorder software applications use a date/time stamp to sort

recorded video. This does allow for automated sorting as the video data files are sent to a

predetermined storage location and are sorted according to the date and time they were recorded.

However, this convention does not allow for the sorting of video based on its content. The

search and retrieval of multiple related records is also not supported. In normal operations, the

user must make note of the specific time and date of an event and access the file manually,

usually having to manually parse through different file locations and numerous video files to

locate the one in question. The use of metadata can alleviate this issue.

The U.S. government’s Federal Geographic Data Committee (FGDC) describes metadata

as the data about the content, quality, condition, and other characteristics of the data.115 At this

time, the FGDC is the organization that has been tasked with developing the guidelines and

standards for digital metadata that is to be used in geospatial imagery. Although metadata is

being developed primarily for use in the support of geographic information system (GIS)

mapping data, it would serve equally useful from the perspective of tactical video collection.

Some minimal considerations for items that might be included as part of a data element set that

should be included with the metadata might be as follows:

1. Time/date stamp.

2. Geographic locations of known points.

3. Location Coordinate System (Military Grid Reference System (MGRS)).

4. Distance/altitude units (feet/meters).

115Federal Geographic Data Committee, Content Standard for Digital Geospatial Metadata,

http://www.fgdc.gov/standards/projects/FGDC-standards-projects/metadata/base-metadata/v2_0698.pdf, 64

(accessed January 20, 2010).

http://www.fgdc.gov/standards/projects/FGDC-standards-projects/metadata/base-metadata/v2_0698.pdf

45

5. Altitude/elevation data in reference to location.

6. Populated areas: cities, towns, and villages.

7. Rural areas: farms, ranches, etc.

8. Infrastructure: schools, public buildings, places of worship, power stations, military

installations, police stations, telecommunications (antenna towers), water treatment

facilities, transportation (roads/highways both improved and unimproved,

intersections, traffic circles), airfields/airports/airstrips, dams, bridges, waterways

(lakes, oceans, seas, rivers, streams, man-made), etc.

Many of these items already have a place in the metadata guidelines developed by the

FGDC. Other important metadata considerations have also been outlined by the FGDC, which

include mission specific data items. These may easily be adapted in the metadata that is used in

support of tactical video collection efforts. Some of this mission specific data is as follows116:

1. Mission data/time.

2. Mission name.

3. Mission significant events.

4. Mission Platform and Instrumentation.

At the present, the use of metadata in many video applications has been limited. Its usage

may also necessitate user intervention for the manual input of some data items. However, the

advent of the MPEG-7 and MPEG-21 video compression standards may provide a path forward

for the intelligent organization of video data files. The adoption of these compression methods

in future video collection systems will allow video files to be “tagged” and automatically sorted,

catalogued, and organized according to specific attributes that are captured as part of the

metadata architecture. This will aid in the timely search and retrieval of related video data

applicable to specific software queries.

116Federal Geographic Data Committee, Content Standard for Digital Geospatial Metadata: Extensions for

Remote Sensing Metadata, http://www.fgdc.gov/standards/projects/FGDC-standards-

projects/csdgm_rs_ex/MetadataRemoteSensingExtens.pdf, 67-71 (accessed January 20, 2010).

http://www.fgdc.gov/standards/projects/FGDC-standards-projects/csdgm_rs_ex/MetadataRemoteSensingExtens.pdf

http://www.fgdc.gov/standards/projects/FGDC-standards-projects/csdgm_rs_ex/MetadataRemoteSensingExtens.pdf

46

CHAPTER 4

CONCLUSION

The purpose of this paper was to evaluate the issue of database storage for the tactical

video collection process. This is a major concern for end users of these systems given the large

amount of video that is collected on a daily basis. The previous discussion has attempted to

explain the necessary hardware that is available and the software related techniques that are used

in this process. Possible recommendations were also provided that may assist in easing the

concerns related to this issue.

First, large capacity storage media were discussed to include magnetic tape, optical disks,

magnetic disk drives, and finally solid state drives. These media types are of particular

importance since they provide the best large-scale storage capacity. Each was evaluated to

determine which of them were best suited to be used in a tactical environment. Redundant

storage in the form of the RAID architecture was also examined.

Magnetic tape media has been a mainstay for the backup of digital video and imagery as

it provides a low cost method of backing up data for archival purposes. However, the sequential

nature of magnetic tape media makes it very inefficient for random data access, and

recovery/backup operations. This is particularly important when fielding a tactical collection

system since the command cannot afford to be without the asset for long periods of time

following, for example, a catastrophic failure. The system must be recoverable very quickly.

Magnetic tape media also deteriorate over time and must be stored in moderate environments

with low humidity. The conditions in many of the current locations in which these assets are

fielded make the use of magnetic tape media problematic.

47

Optical disks have many advantages over magnetic tape to include lower cost, greater

storage capacity, data stability, environmental durability, and a long shelf life potential of over

100 years. Their constant linear velocity provides unvarying read/write bandwidth during

sequential access operations, which makes them ideal for multimedia backup and restore

operations. These reasons have made optical disk technology the primary backup media in many

remote sensing operations.

Magnetic hard disks, also known as hard disk drives, are the primary storage technology

used in most personal computer systems and many video collection systems. They provide fast,

concurrent access to data and have very large capacities. Because of this, they are still useful as

a storage medium in redundant systems such as RAID assemblies. However, as they are electro-

mechanical devices with moving parts, extreme environments together with continual use can

dramatically increase their mortality rate.

Solid state drives, on the other hand consist of high reliability flash memory technology

and have no moving parts, which makes them more durable in severe environments. They have

much faster response times, lower access times, and much higher throughput bandwidth than

magnetic hard disks. Lifespan characteristics are comparable to magnetic hard disks with on-

the-shelf data retention of up to ten years.

However, solid state drives are more expensive than magnetic hard disks and capacities

are usually much smaller. In spite of these shortcomings, solid state drives are fast becoming the

primary storage medium for applications that require quicker response time and proven

durability in extreme environments.

The discussion on RAID described its importance as a backup storage medium and some

of the methods used in the different RAID architectures. Two of the most prevalent methods

48

were examined and included RAID Level 1 (RAID-1) and RAID Level 5 (RAID-5). Both

methods provide a means of data redundancy and recovery in case of system failures and/or hard

disk drive loss.

RAID-1 architecture performs only disk mirroring, which provides full redundancy. It

also permits quicker recovery in the case of disk failure in the array, and offers increased read

performance as data is read across all disks in the array. However, it lacks in overall

performance as it is necessary to have twice the storage capacity to accommodate data storage,

which can add significantly to costs.

In comparison, RAID-5 architecture uses parity checking methods and provides more

efficient data redundancy than simple mirroring. It is also less expensive in regards to data

protection costs. However, every change in data facilitates a recalculation of parity, which

adversely affects system performance.

Following the discussion on storage media, the subject of video compression was

investigated. Video compression deals with the reduction of the amount of data required to

convey a useable facsimile of a specific data object. This is particularly important since most

present day imagery can produce extremely large files, which facilitates the need for larger

storage capacities. Both “lossless” and “lossy” compression techniques were examined and are

used to compress data according to the different standards that have been developed to include

those of the Motion Pictures Experts Group (MPEG), which are the most common.

Lossless compression techniques allow for complete reconstruction of the original data

signal and provide a slight amount of compression. Lossy compression techniques provide only

a near approximation of the original data signal since some tradeoff is required to provide

satisfactory compression. Imagery can be produced that is fundamentally and visually similar to

49

the original using ratios much greater than those of lossless compression. Lossy compression is

most commonly used for compressing multimedia data such as audio, video, and still images.

These compression algorithms are used in conjunction with different standards, with the

most prevalent being developed by the MPEG. The standards outline the basic requirements for

the compression of audio, video, and imagery data. Short discussions of the different MPEG

standards were presented with particular attention given to the MPEG-7 and MPEG-21

standards, which are still in their infancy.

These standards do provide for the necessary compression of multimedia data, which is

the primary consideration. However, it is their use of metadata techniques that is of particular

importance. Since metadata considers the peripheral, supporting information about the actual

data, its use for the indexing and cataloging of collected data is significant to the video collection

process.

Multimedia players and multimedia container formats were briefly discussed to provide a

distinction between the two. Multimedia players, like Windows Media Player®, are the

applications that are used to replay the recorded audio-video data files. Container formats like

audio video interleave (AVI) provide the actual formatting process for the video. The AVI

container format was explained solely because of its widespread usage and to provide an

elementary understanding of this technology. No recommendation has been made for these

items since there are many different types which work equally well.

Following discussion of the different storage media and software techniques used in the

storage process, recommendations were made that might assist in alleviating storage capacity

issues in tactical video collection systems. These recommendations were divided into three

50

groups: hardware, software, and future considerations. Recommendations were provided for

each of these categories.

In developing the collection system, hardware considerations were taken into account to

include the motherboard architecture, system memory, processing power, the graphics adapter

and internal storage. As such, these suggestions were offered mainly as guidelines with the

system storage being the primary focus. In this case, it was recommended that solid state drives

be used as the primary data storage media in the computing machinery for tactical video

collection systems due to their higher performance capability and durability. Magnetic hard

disks should still be used in secondary data storage and backup operations as part of a RAID

assembly since they provide higher capacity storage than solid state drives.

Redundant storage was also discussed in the form of a RAID assembly, which provides

for system backup and large external storage capacity. Magnetic drives should be used for this

assembly as they are more efficient than solid state drives for mass storage and should suffice for

normal backup operations. The tactical video collection system is a mission-critical platform

that cannot tolerate any loss in data, if at all possible. In light of this, it was recommended that

RAID-1, providing complete data redundancy, be used to accommodate quicker system recovery

in case of system corruption or total system failure.

The archiving of data was also examined using optical disks as the storage medium.

Their large capacity, data retention characteristics, and durability made them the best choice for

this issue. Blu-Ray optical disks were recommended because they presently provide the most

storage capacity of all optical disk types.

Software considerations included the operating system, the digital video recorder (DVR)

application, and a backup and restore utility. In these cases, the Windows® OS was

51

recommended due to its widespread usage. No DVR application was suggested as this should be

selected based on system requirements. At a minimum, it should provide the necessary

recording function as well as post-mission playback and video editing utilities. With respect to

the system backup operations, a reliable backup and restore application was also examined. Its

operation should be transparent to the host system and provide automated, scheduled backups as

deemed necessary. Full backups were recommended to provide complete repair of the system in

case of file corruption or system failure. These scheduled backups should be placed in the

secondary storage area (RAID assembly) to provide expedited system repair or rebuild as

required.

Following system setup, it was recommended that a master system disk be created on

Blu-Ray optical disk to facilitate system repair or rebuild in case of catastrophic system failure or

internal disk issues. Tactical video intended to be kept for historical purposes should also be

copied to optical media to facilitate its later usage. These items should be kept in the on-site

archive. It was also suggested that all recorded video be captured and stored on the external

RAID assembly in the interest of keeping the primary storage area free of any extraneous files

that might affect system operation.

Proposals were made for software metrics that could provide for the automated retention

or deletion of stored video in the interest of freeing up storage space. These metrics could use

items such as the time/date stamp and the number of times a file has been accessed to determine

retention or deletion of files from the video database. Other versions of this method might base

retention and deletion on the actual nature of the video and its importance to the command.

Different grades or levels of importance might be assigned to provide a clear distinction between

video that is considered of tactical value and worth retaining and video that is deemed as

52

unimportant and requiring deletion. Some user intervention may still be required for these

methods, but they could provide a basis for determining the importance of the collected video

and its subsequent tactical value.

Finally, future considerations were alluded to, in particular the use of metadata as an

automated cataloging and indexing tool. The metadata technology is used to describe the

background information attributed to the collected data. The use of this technology is

accommodated in the MPEG-7 and MPEG-21 compression standards, which are currently in

work by the Motion Pictures Experts Group (MPEG) organization.

The Federal Geographic Data Committee (FGDC) provides the oversight regarding the

use of metadata and is the current working body responsible for the standardization of the

metadata technology. The guidelines developed by the FGDC are currently used in conjunction

with the development of GIS applications.

However, these same guidelines might be easily adapted for use on tactical video

collection systems and may provide the means for the expeditious retrieval of video data sets that

correspond to specific events, times, and locations. This makes it a potentially powerful

intelligence tool and may provide a foundation for content-based video data retrieval in the

future.

53

BIBLIOGRAPHY

Bergman, Lawrence D., and Castelli, Vittorio. Image Databases: Search and Retrieval of

Digital Imagery. New York: John Wiley & Sons, Inc., 2002.

Christopher Drew. “NYT: Military deluged in intelligence from drones - Remote-controlled

planes produced about 24 years’ worth of video in 2009.” New York Times, January 10,

2010, Sunday; http://www.msnbc.msn.com/id/34798080/ns/world_news-

the_new_york_times/ (accessed January 11, 2010).

Cornell University Library. “Moving Theory Into Practice: Digital Imaging Tutorial, Chapter 5,

Metadata: Definition, Types, and Functions.”

http://www.library.cornell.edu/preservation/tutorial/metadata/metadata-01.html (accessed

January 14, 2010).

Coughlin, Tom, Ekker, Neal, and Handy, Jim. “Solid State Storage 101: An introduction to

Solid State Storage.” Storage Networking Industry Association (SNIA) Solid State

Storage Initiative, January 2009,


(accessed March 13, 2010).

Crothers, Brook. “Seagate Enters Solid-State Drive Market.” CNet News, December 7, 2009,


Daintith, John. "Magneto-Optic Storage." A Dictionary of Computing. 2004. Encyclopedia.com.

http://www.encyclopedia.com/doc/1O11-magnetoopticstorage.html (accessed March 5,

2010).

Effelsberg, Wolfgang and Steinmetz, Ralf. Video Compression Techniques. Netherlands:

KoninKlijke Wöhrmann B.V., 1998.

Encyclopedia.com. "Compact Disk." The Columbia Encyclopedia, Sixth Edition, 2008,

http://www.encyclopedia.com/doc/1E1-compactd.html (accessed March 4, 2010).

Encyclopedia.com. "Digital Versatile Disc." The Columbia Encyclopedia, Sixth Edition,

2008, http://www.encyclopedia.com/doc/1E1-DVD.html (accessed March 4, 2010).

Encyclopedia.com. "Optical Disk." The Columbia Encyclopedia, Sixth Edition, 2008,


Farley, Marc. Storage Networking Fundamentals: An Introduction to Storage Devices,

Subsystems, Applications, Management, and File Systems. Indiana: Cisco Press, 2007.

Jensen, John R. Introductory Digital Image Processing: A remote Sensing Perspective. New

Jersey: Prentice-Hall, 2005.

http://www.msnbc.msn.com/id/34798080/ns/world_news-the_new_york_times/

http://www.msnbc.msn.com/id/34798080/ns/world_news-the_new_york_times/

http://www.library.cornell.edu/preservation/tutorial/metadata/metadata-01.html



http://www.encyclopedia.com/doc/1O11-magnetoopticstorage.html

http://www.encyclopedia.com/doc/1E1-compactd.html

http://www.encyclopedia.com/doc/1E1-DVD.html


54

Judd, Josh. Principles of SAN Design: Design Build and Manage SANS. Pennsylvania: Infinity

Publishing, 2007.

Judd, Josh and O’Conner, Michael. Introducing File Area Networks Advance Edition: Your

First Look at FAN Technology. Pennsylvania: Infinity Publishing, 2007.

Luff, John. “Technology in Transition: Video storage.” Broadcast Engineering,

http://broadcastengineering.com/mag/broadcasting_video_storage/index.html (accessed

January 18, 2010).

Luff, John. “Video Storage.” Broadcast Engineering, April 1, 2003,


January 17, 2010).

Luff, John. “Video Storage.” Broadcast Engineering, July 1, 2004,


January 17, 2010).

Mediatechnics Systems, Inc. "How a CD is Made." Mediatechnics Systems, Inc., FAQ,

https://www.mediatechnics.com/cdfaqs.htm (accessed March 5, 2010).

Mediatechnics Systems, Inc. "How are DVDs Made for use in Copiers?" Mediatechnics

Systems, Inc., FAQ, https://www.mediatechnics.com/dvdcopiers.htm (accessed March 5,

2010).

National Information Standards Organization. “Understanding Metadata.” (Maryland: NISO

Press, 2004), http://www.niso.org/publications/press/UnderstandingMetadata.pdf

(accessed January 20, 2010).

Nelson, Dane and Siewart, PhD, Sam. “Solid State Drive Applications in Storage and Embedded

Systems.” Intel Technology Journal, Volume 13, Issue 1, 2009,



Park, Oskar. “What is an Optical Disk?." Self SEO (Search Engine Optimization), September

10, 2006, http://www.selfseo.com/story-18894.php (accessed March 11, 2010).

Schulz, Greg. Resilient Storage Networks: Designing Flexible Scalable Data Infrastructures.

Massachusetts: Elsevier Digital Press, 2004.

Shrivastava, Alok, and Somasundaram, G., eds. Information Storage and Management: Storing,

Managing, and Protecting Digital Information. Indiana: Wiley Publishing Incorporated,

2009.

http://broadcastengineering.com/mag/broadcasting_video_storage/index.html



https://www.mediatechnics.com/cdfaqs.htm

https://www.mediatechnics.com/dvdcopiers.htm

http://www.niso.org/publications/press/UnderstandingMetadata.pdf



55

Simitci, Huseyin. Storage Network Performance Analysis. Indiana: Wiley Publishing, Inc.,

2003.

Symes, Peter. Digital Video Compression. New York: McGraw-Hill, 2004.

Thatcher, Jonathan. “NAND Flash Solid State Storage Reliability and Data Integrity: An In-

Depth Look.” Storage Networking Industry Association (SNIA) Solid State Storage

Initiative, 2009,

http://www.snia.org/forums/sssi/knowledge/education/Solid_State_Storage_Reliability_a

nd_Data_Integrity--An_In-depth_Look.pdf (accessed March 13, 2010).

The Free Dictionary. “Flash Memory.”

http://encyclopedia2.thefreedictionary.com/flash+memory (accessed March 2, 2010).

The Free On-line Dictionary of Computing. "Average Seek Time."

http://encyclopedia2.thefreedictionary.com/average+seek+time (accessed March 12

2010).

The Oxford Essential Dictionary of the U.S. Military. “Tactical Operations.” Answers.com.

The Oxford Essential Dictionary of the U.S. Military, Oxford University Press, 2001,

2002, http://www.answers.com/topic/tactical-operations (accessed March 07, 2010).

Tudor, Marius. “Are Flash Solid-State Disk Drives Ready for the Enterprise?” Bit Micro

Networks, Incorporated, 2009,

http://www.bitmicro.com/press_resources_flash_ssd_enterprise.php (accessed March 2,

2010).

Wikipedia contributors. "Audio Video Interleave." Wikipedia, The Free Encyclopedia,

http://en.wikipedia.org/w/index.php?title=Audio_Video_Interleave&oldid=347331467


Wikipedia contributors. "Blu-ray Disc.” Wikipedia, The Free Encyclopedia,

http://en.wikipedia.org/w/index.php?title=Blu-ray_Disc&oldid=349002833 (accessed

March 12, 2010).

Wikipedia contributors. "Blu-ray Disc recordable." Wikipedia, The Free Encyclopedia,



Wikipedia contributors. "Lossless data compression.” Wikipedia, The Free Encyclopedia,

http://en.wikipedia.org/w/index.php?title=Lossless_data_compression&oldid=346709055


Wikipedia contributors. "Lossy compression.” Wikipedia, The Free Encyclopedia,

http://en.wikipedia.org/w/index.php?title=Lossy_compression&oldid=347940008




http://encyclopedia2.thefreedictionary.com/flash+memory

http://encyclopedia2.thefreedictionary.com/average+seek+time

http://www.answers.com/topic/tactical-operations

http://www.bitmicro.com/press_resources_flash_ssd_enterprise.php

http://en.wikipedia.org/w/index.php?title=Audio_Video_Interleave&oldid=347331467

http://en.wikipedia.org/w/index.php?title=Blu-ray_Disc&oldid=349002833


http://en.wikipedia.org/w/index.php?title=Lossless_data_compression&oldid=346709055

http://en.wikipedia.org/w/index.php?title=Lossy_compression&oldid=347940008

56

Wikipedia contributors. "Magneto-optical drive." Wikipedia, The Free Encyclopedia,




Documents

Data Storage Considerations for the Tactical Field Collection of Digital Imagery - 2010