MOMA Display Screens

MOMA Display Screens

K u r t R a l s k e

QuickTime™ and aPhoto - JPEG decompressor

are needed to see this picture.

Technologies used in the project

C - for graphics processing codeJava - for XML parsing and show schedulerXML - for storing show templates and show definitionsMax - for image display and talking to QuickTimeOSC - network protocol

Motion Graphics is extremely data-intensive

One second of video =

(720 (width) x 480 (height)) pixels x 30 frames per second =

10,368,000 pixels per second

…this is a lot of data for a hard disk to read, consistently

10,368,000 pixels per second

CPU uses four elements per pixel (alpha, red, green, blue) , so

10,368,000 pixels x 4 =

41,472,000 CPU operations per secper image layer

…plus, many operations may need to be done to each element

Motion Graphics is extremely CPU intensive

10,368,000 pixels per second

CPU uses four elements per pixel (alpha, red, green, blue) , so

10,368,000 pixels x 4 =

41,472,000 CPU operations per secper image layer

…plus, many operations may need to be done to each element

Motion Graphics is extremely CPU intensive

___Standard video format -- 720 x 480

(720 x 480) pixels x 30 fps x 4 =41,472,000 CPU operations per sec per image layer

___MoMA video format -- 1280 x 768

(1280 x 768) pixels x 30 fps x 4 =117,964,800 CPU operations per sec per image layer

(1280 x 768) / (720 x 480) = 2.84

MoMA spec requires high resolution video

Problem: How to get huge amounts of video data off hard drive, quickly?

Problem / Solution

Problem: How to get huge amounts of video data off hard drive, quickly?

Solution: Don’t do it. Instead, render video in real-time.

Technical hurdle: Compressed / uncompressed video? How to create specified video actions in real-time, using many layers of images?

Problem / Solution

Problem: How to render video in real-time, at high resolution?

Problem / Solution

Problem: How to render video in real-time, at high resolution?

Solution: Work from still images. Create visual motion by moving, transforming, and layering the images.

Technical hurdle: How to access hundreds of still images?

Problem / Solution

Problem: How to access hundreds of large still images from the hard drive, with no latency?

Problem / Solution

Problem: How to access hundreds of large still images from the hard drive, with no latency?

Solution: Don’t do it. Store images in RAM.

Technical hurdle: How many images will fit in RAM?How much RAM is available?

Problem / Solution

Problem: Each machine needs a different set of images. How to catalog and retrieve hundreds of still images from 9 different machines?

Problem / Solution

Problem: Each machine needs a different set of images. How to catalog and retrieve hundreds of still images from 9 different machines?

Solution: Don’t do it. Make sure all the machines store all the images that any machine might need.

Technical hurdle: How many images will fit in RAM?How much RAM is available?

Problem / Solution

Problem: How to deal with network latency, from the master controller machine to the 9 player machines?

Problem / Solution

Problem: How to deal with network latency, from the master controller machine to the 9 player machines?

Solution: Structure commands so only very few messages need to be sent.

Technical hurdle: How few messages can be sent, with still keeping time-sync between multiple machines?

Problem / Solution

Problem: How to keep performance high, so each machine can create layers of video, without slowing down or dropping frames?

Problem / Solution

Problem: How to keep performance high, so each machine can create layers of video, without slowing down or dropping frames?

Solution: 1970’s-style optimizations: integers instead of floating point, fixed-point math, no divisions, bit shifting, keeping math outside of inner loop.

Technical hurdle: How to keep code human-readable!

Problem / Solution

1) A dream

2) Researching the limits of the technology

3) Working with (or working around) the limits

The development process