Bringing Digital Technology Back to Earth

Bringing Digital Technology Back to Earth

Presented by

Ken Dozier

USC Viterbi School of EngineeringNASA Far West Technology Transfer Center

Technology TransferNASA Mission: Transfer

technology developed in the space program to the private sector,Across all 50 StatesSpecial focus on SME

Technology in Space Industry should be a technological windfall to the U.S. economy

Windfall - unexpected gain, blessing, unsolicited advantage, serendipity, stroke of luck, pleasant surprise, godsend, boon, bonanza

Digital Content Resolutions

480p (DVD Quality) 300Kp

720p (Release Print Quality) 1Mp

1080p (HD Cinema Quality) 2Mp

High Definition Imaging

Original Data was 1920 x 1080 (60i)Two levels of compressionThen converted to 1280 x 720p Then data was compressed

using Microsoft’s Media Player

The projector is being driven by a PC, not a high definition video deck

Shuttle/Earth footage

Combinations of Compression

“311”“322”

+4.4+

6.7

422

+4.7

444

+2.1

444

+0

Cam

Tape

A New Paradigm - Economical Technologies

Digital Camera for AcquisitionImmediate feedback for everyoneImmediate editing, color correction, immediate post

Computer(s) StorageServer FarmsOn line conform, color correction, effects, format conversionMastering

Digital Projection Screening RoomNo tape or film

Cinema Master Workflow

Source: “A Data-Centric Approach to Cinema Mastering” Thomas J. True, SGI

Acquisition

Image courtesy of Thompson Grass Valley

Camera: Thompson Viper Camera 1080p Output: 4:4:4 10-bit log dpx format (1920 x 1080) Capture: Directors Friend

Camera: Sony F950Output: 4:4:4 10-bit Linear RGB CIF format (1920 x 1080) Capture: Dual HD-SDI tapeless system

Image courtesy of Sony

Post Production

Source: “Digital Infrastructure Solution for Production” J. Farney, SGI

Imagica Film Scanners

Visual EffectsOptical HouseTitling House

CXFS SAN52 TB TP4900

Brocade Switches

Laser Film Output

ELab Real-timeColoring Timing and

Dust-bustingSGI Onyx 3400

MultiversionRendering

SGI Origin 300

DLP ProjectionScreening Theater

Edit RenderingSGI Origin 300

Backup ServerSGI Origin 300

ELab Real-timeFilm Stock Emulation

SGI Onyx 3400

eFilm Configuration

Source: 1999 Fall Meeting, Community Development Council, Chuck MatthewsINFOWORLD, Sept. 2000

Distribution Media Bandwidth

B01-017

DSL/ Cable

IEEE 1394 / Firewire

Gigabit Ethernet

LASER / Fast Ethernet

10BaseT / CAT 5

Microwave / Ethernet

G2 Wireless

G3 / Wireless LAN

Async. Transfer Mode (ATM)

G1 Wireless

• Satellite

• Land-based/ Wired broadband

• Physical digital media

Editing/Post

• PC-based servers with massive storage

• Real-time editing in native HD resolution

Projection

• Digital Light Processing (DLP)Developed at Texas Instruments max resolution (2048 x 1080)

• Direct Drive Image Light Amplifier (D-ILA)Developed at JVCmax resolution (2048 x 1536)

Image courtesy of JVC

Image courtesy of TI

Old vs. New“Roadwarrior” connected to Internet and to a

projector is a cinema

Today’s setup is a DLP Projector (2K) and Surround Sound system

• Boom mounted Sony F900 camera allowed close moves.

• Shot @ 1920 x 1080

• One day of shooting (4 hours of material)

• Cameraman used HD Monitor not Eyepiece.

• Cinema Projector @ 1920 x1080

Pasadena-clip-1

HDTV Title (acquition)“Pasadena”

Adobe Premiere using proxy images

Down sampled to 480 x 240 images for editing on ordinary PC

Time Code was not used or needed

HDTV Title(post production)“Pasadena”

Full 1920 x 1080 resolution HD conformed on a PC

Finished HD playback 1280 x 720. (720P)

Pasadena-clip

HDTV Title (distribution) Pasadena”

NASA Technology

• Millions $$ in Government funded R&D• Special programs for SMEs• Available for Licensing and

Commercial Development• Help in accessing information

is available through USC’s Engineering Technology Transfer Center

• See the web page at http://www.usc.edu/go/TTC

PROBLEM ADDRESSED Evaluation of visual quality of digital video (HDTV)

TECHNICAL APPROACH A computationally efficient metric based on human visual processing: (1) Comparison of one or more selected features (image frames, color channels, rows of blocks, columns of blocks, horizontal spatial frequencies, or vertical spatial frequencies) of processed digital sequences of the image under evaluation and a reference image: (2) Processing consisting of sampling, cropping, and color transformations, blocking, acceleration of spatial filtering by taking discrete cosine transforms, temporal filtering and normalization to visual thresholds

POTENTIAL APPLICATIONS Any digital display, e.g. HDTV

BENEFITS (1) Quantitative assessment of display quality that incorporates properties or human visual processing, (i.e., dynamic adaptation to changing brightness, luminance, and chromatic channels, spatial and temporal filtering, spatial frequency channels, dynamic contrast masking, and summation of probabilities); (2) Requires only modest computational resources to compute metrics in real time

Sampling, Cropping andColor Transformations

BlockingDiscrete Cosine

TransformTransformation to

Local Contrast

TemporalFiltering

Transformation To Contrast Sensitivity Functions

ContrastMasking

PoolingConversion

To a Measure of Visual Quality

Reference Image SequenceTest Image Sequence

DIGITAL VIDEO QUALITY

Technology Opportunities

PROBLEM ADDRESSED: VISAR is a computer algorithm that corrects for zoom, tilt, and jitter

TECHNICAL APPROACH: By combining several video images together, noise can be averaged out among the frames and the video frames can be centered on the screen.

BENEFITS:(1) Stabilize, sharpen, and brighten image sequences and stills(2) Steadying and reducing the noise in the images, brings out a wealth of information, revealing new, previously obscured details

POTENTIAL APPLICATIONS: Surveillance, crime scene footage, sting operations, and dash-mounted video cameras, Security, video feed from aircraft, target identification and confirmation, training, and event reconstruction

Images courtesy of NASA Ames Vision group

VISARImage Stabilization and Registration


VIDEO DISPLAY ENGINEERING AND OPTIMIZATION SYSTEM

PROBLEM ADDRESSED: Computer-aided design (CAD) of visual displays

TECHNICAL APPROACH: (1) Programmable computer with a number of discrete software modules for modeling display system components; (2) Integration of models of circuit, electro-optics, surface geometry, and physiological optics components to simulate performance of display

POTENTIAL APPLICATIONS: Design of displays and control interfaces in a variety of applications: e.g., avionics and automotive displays, computer system displays, telecommunications systems, data display systems, entertainment systems

BENEFITS: (1) Avoids costly and time consuming iterative conventional design process; (2) All system elements included in simulation, with ability to modify each individually ; (3) Objective performance measurement with comparisons to a a standard system design; (4) Capability of combining empirical data with theoretical models in simulation


DISTRIBUTED INTELLIGENT AGENTS FOR INFORMATIONMANAGEMENT AND SHARING (DIAMS)

PROBLEM ADDRESSED: Management and sharing of information distributed over the Internet

TECHNICAL APPROACH: Gathering, organizing, and sharing of relevant information on the Internet by integrating knowledge-based, neural network and genetic algorithm technologies in distributed and collaborative information agents: (1) Information access based on dynamically organized views of personal information repositories using knowledge-based and neural network representation and indexing; (2) Organization of information based on observing users’ preferences for organizing information; (3) Knowledge exchange protocol for collaboration using distributed automated agents incorporating multi-attribute network objects, added semantic knowledge, and symbolic processing capabilities

POTENTIAL APPLICATIONS: Widely usable for efficient information gathering on the WWW by individuals and for sharing of information for collaboration; e.g., for distributed manufacturing, joint research projects, military deployments, etc.

BENEFITS: (1) Rapid and effective access to information for both individuals and work groups; (2) Dynamically adaptive, avoiding static, hierarchical and monolithic shortcomings of some current Internet information aids; (3) Flexible interface protocols, permitting use of existing tools and capabilities; (4) Reduction in amount of irrelevant information in WWW searches


AUTOMATION OF INFORMATION ACQUISITION AND MANIPULATION (AIMS)

PROBLEM ADDRESSED: Improvement in data search, retrieval, discovery, and processing

TECHNICAL APPROACH: Automatic manipulation of digital information with Java-based software that (1) automates the collection and formatting of different data types, followed by (2) complete integration of the data for processing, analysis, and extraction of resultsPOTENTIAL APPLICATIONS: Any area in which large amounts of disparate data must be collected and manipulated, e.g. medical imaging; monitoring and control of industrial processes; processing of data in geosciences, biotechnology, pharmaceuticals; development of maps of terrain and demographic information; etc.BENEFITS: (1) A powerful, flexible system that is broadly applicable; (2) Reduction in manual labor; (3) Performance of tasks in real and near-real time; (4) A comprehensive system that not only gathers data, but also identifies, processes and tracks desired information


ADAPTIVE RELEVANCE-LEARNING SYSTEM (ARNIE)

PROBLEM ADDRESSED: Adaptive information indexing and retrieval agent TECHNICAL APPROACH: A computerized personal librarian assistant using a neural network that is dynamically configured with a genetic algorithm: (1) Learns users’ interests based on how they organize information, enabling the encoding of information according to relevance and structure; (2) Permits user feedback on resource utility which is then used to dynamically update ranking of resources

POTENTIAL APPLICATIONS: Widely usable for information gathering under various contexts: e.g., already used in a Boeing Portable Maintenance Aid, NASA/JSC Adaptive Hyperman Electronic Documentation system in mission control, and WebTagger – a personal bookmarking service for Web-based information resources.

BENEFITS: (1) Rapid and effective access to relevant information, with minimization of irrelevant information; (2) Efficacy proven (e.g., 94.8% relevance on retrieved materials when user profile known vs. 10% relevance for unlearned user profile); (3) System accessible from Internet


ADVANCES IN PARALLEL COMPUTING ON ADAPTIVE GRIDS

PROBLEM ADDRESSED: Optimization of parallel computing on unstructured grids

TECHNICAL APPROACH: Five approaches, comprising: (1) Improvements in rates and efficiency through multithreading; (2) Partitioning of computational loads among the parallel processors by use of self-avoiding walk algorithm; (3) A load balancing strategy for integrating all major components, including interfaces between a parallel mesh-adapting code and a data remapping module; (4) Balancing loads using a global view that is updated each time the grid is adapted; (5) Load balancing based on symmetric broadcast networks

POTENTIAL APPLICATIONS: Any adaptive grid tying together parallel processors, e.g. in networks of supercomputers for rapid computation of complex problems, for weather and climate change forecasting, for scientific computing in biology, chemistry, and physics, etc.

BENEFITS: (1) Increased computing rates and efficiencies; (2) Reduction of clashes in use of computing resources; (3) More equitable use of computing resources, with particular computing assignments matched to capabilities; (4) Real-time optimization of grid


SCALABLE HIERARCHICAL NETWORK MANAGEMENT SYSTEMFOR DISPLAYING NETWORK INFORMATION IN THREE DIMENSIONS

PROBLEM ADDRESSED: Hierarchical network management and control

TECHNICAL APPROACH: A three tier network management system comprising: (1) Simple network management protocol (SNMP) agents distributed at one or more sites; (2) Server module configured exclusively for hierarchical network management protocol (HNMP) communications, connected with input-output modules that are configured for both SNMP and HNMP; (3) Management aided by informationally complete and non-iconified views provided of network elements, with network information displayed in three dimensions

POTENTIAL APPLICATIONS: Generally applicable to any local or wide area network of computers; particularly useful for management of large high speed networks

BENEFITS: (1) A distributed architecture system based on open, published, standard; (2) Capability of displaying network information at multiple levels without hiding data behind other data; (3) No editing of displays required of user; (4) Capability of monitoring networks having multiple managerial points of control; (5) Not limited in scalability (6) Not subject to data overload; (7) Does not provide false information due to data flow latency; (8) Substantial user involvement not required


AUTOMATED TRAFFIC MANAGEMENT SYSTEM AND METHOD, AND REAL-TIME SURFACE TRAFFIC ADVISER

PROBLEM ADDRESSED: Real-time data management system for data generated at different rates by many incompatible data sources

TECHNICAL APPROACH: (1) A client-server expert system for fusing data from a variety of sources, with means of establishing, predicting, and updating reference data values; (2) electronic communications system for facilitating information sharing between different domains of control; (3) Easy-to-understand electronic display of data

POTENTIAL APPLICATIONS: Scheduling of movement of multiple vehicles, such as aircraft at airports, marine vessels in harbors and ports, trucks or railroad cars in shipping and switching yards: Useful also for managing containers at shipping docks and stock in factories or warehouses; Simulation and training tool for operators

BENEFITS: (1) Control and management in a broad system-wide context; (2) Optimization of resource allocation; (3) Enhanced safety through improved situational awareness; (4) Automatic archiving in an information system database; (5) Efficacy demonstrated in a large airport hub (Atlanta)


• PROBLEM ADDRESSED Numerical measure of the perceptual intensity and difference of an image

• POTENTIAL APPLICATIONS Photometric instruments, copiers, video codecs, displays, graphics software, Optical Character Recognition (OCR) systems

• BENEFITS (1) Allows quantitative assessment of the goodness of a single image or comparison of two images

Spatial Standard Algorithm

Difference?

SPATIAL STANDARD OBSERVER


• PROBLEM ADDRESSED: Improvement of digital image compression

• TECHNICAL APPROACH: Discrete cosine transform (DCT) compression of image; customized quantization matrix to image, using:(1) Luminance masking(2) Contrast masking (3) Error pooling(4) Entropy coding

• POTENTIAL APPLICATIONS Internet multimedia, cable TV, HDTV, motion picture transmission & archival, still or motion picture editing, digital copiers & scanners, digital facsimile machines, digital still-and video-cameras

• BENEFITS Minimum perceptual error for any bit rate, or minimum bit rate for a given perceptual error can be used as add-on SW module to existing imaging workstations

Images courtesy of NASA Ames Vision group

Original Optimized

Technology OpportunitiesDCTUNE

IMAGE COMPRESSION TECHNOLOGY

Landmark Connection

• Jim Steele and Microsoft furnished first digital content to the Far West RTTC to allow testing of the “roadwarrior”

• Jim Steele and Landmark are furnishing HD content for testing.

• Exploring possible innovative partnerships for education content in digital cinema venues.

Landmark Theatres Overview

• The largest independent film theatre chain in the U.S.– 209 screens in 22 top markets nationwide– 12+ million guests per year– Landmark controls 40% of the core audience of independent film

goers

• Atlanta• Austin• Boulder• Boston• Chicago• Dallas• Denver• Detroit• Houston• Los Angeles• Madison• Milwaukee• Minneapolis• New Orleans• New York• Palo Alto• St. Louis• San Diego• San Francisco• Seattle• Washington, D.C.

Landmark Independent Film Audience

• Highly educated (beyond four-year college degree)• Over 79% are over 25+ • 50% HHI over $50,000; 12% over $150,000• Career professionals• Cultured, urbane, savvy, dual-income• Likely to reside in A-counties and

East/West coasts

Digital CINEMA NETWORK

• IN 2006 Landmark will be installing an HD digital• projection network throughout the chain. Top

theatres• in the network will feature:

• UNPARALLELED 4K PROJECTION

Programming for Landmark’s Digital Theatre Network will include: • Digital films• Educational Seminars• Interactive Forums• Live concerts • TV Specials• Original partner content

Digital On-screen Programs

• As the cornerstone to its Educational Initiatives, Landmark invites NASA to use its Digital Network and big screen experience to communicate and educate their millions of fans around the country.

• FY06 Congress is urging NASA to increase utilization of NASA TV.

Windows Media HD examples

Step Into Liquid- Multiple image capture formats- 720p Windows Media Player

Coral Reef -IMAX footage, Stereo Mix-1080p Windows Media Player

Mars Rover Animation- Rendered in Pixar RenderMan- 1080p Windows Media Player

Brought to you by

The NASA Far West Regional Technology Transfer Center

http://www.usc.edu/go/TTC

Documents

Bringing Digital Technology Back to Earth