Research and Pedagogy of Research and Pedagogy of Low-Latency Distributed Low-Latency Distributed Interactive Multimedia Interactive Multimedia Collaboration SystemsCollaboration Systems

Dylan McNameeDylan McNamee

Department of Computer Department of Computer Science and EngineeringScience and Engineering

Oregon Graduate InstituteOregon Graduate Institute

Using Video Games to Motivate Using Video Games to Motivate Computer Science Education and Computer Science Education and ResearchResearch

Taxonomy of video gamesTaxonomy of video games Computer Science of Video GamesComputer Science of Video Games Research AreasResearch Areas

Taxonomy of video gamesTaxonomy of video games

The Classic [Crawford, 1980]:The Classic [Crawford, 1980]:• ““genre” genre”

– maze gamesmaze games– shoot-em-upsshoot-em-ups– puzzlespuzzles– simulationssimulations

Worked for a while, but is now brokenWorked for a while, but is now broken• All 3-D games are simulations, most are All 3-D games are simulations, most are

puzzles, many involve shooting in mazespuzzles, many involve shooting in mazes

An implementation-based An implementation-based taxonomytaxonomy

Measure attributes of “model” and Measure attributes of “model” and “view” orthogonally, fit games to “view” orthogonally, fit games to matrix:matrix:

Text 2D 2.5D 3D

1D Books Cartoons,Movies

Plays, “linear”games

2D Adventure,“goto” Books

PacMan,SimCity, …

Myst, Riven,Mazewar

2.5D SimCity2K,Zaxxon

Doom

3D QuakeFlightSims

View

Mod

el

Teaching computer Teaching computer science with gamesscience with games

PerfectPerfect capstone course capstone course• Games apply almost every aspect of CSGames apply almost every aspect of CS

– graphics, linear algebra, ...graphics, linear algebra, ...– operating systems - scheduling, networking, ...operating systems - scheduling, networking, ...– OO programming (and general programming)OO programming (and general programming)– compilers, interpreters, language designcompilers, interpreters, language design– architecture, ISA, memory hierarchies, ...architecture, ISA, memory hierarchies, ...– algorithms and data structuresalgorithms and data structures– artificial intelligence, artificial life, …artificial intelligence, artificial life, …

• Did I leave anything out?Did I leave anything out?

Example applicationsExample applications

Algorithm Design and Analysis:Algorithm Design and Analysis:• performance of algorithms in games is performance of algorithms in games is

criticalcritical• a game’s goal is at least 25 frames per a game’s goal is at least 25 frames per

secondsecond– serious action games strive for 60 fpsserious action games strive for 60 fps

• modest machines can perform 50 MIPSmodest machines can perform 50 MIPS– ~2 million instructions per frame~2 million instructions per frame

• asymptotic analysis, O(n), O(nasymptotic analysis, O(n), O(n22), …), …

Asymptotic performanceAsymptotic performance

How big a problem can be solved How big a problem can be solved per frame?per frame?

Complexity Problem Size/Frame

O(logN) ~infinite

O(N) 2,000,000

O(NlogN) 118,650

O(N2) 1,414

O(2N) 21

Effects of scaling Effects of scaling processor speedprocessor speed

Assume we use a faster machine Assume we use a faster machine or language - 500 MIPSor language - 500 MIPS

Complexity 2M / frame 20M / frame

O(logN) ~infinite ~infinite10

O(N) 2,000,000 20,000,000

O(NlogN) 118,650 1,003,201

O(N2) 1,414 4,472

O(2N) 21 24

Another angle at scalingAnother angle at scaling

The evolution of Intel’s The evolution of Intel’s microprocessors:microprocessors:• Four orders of magnitude in 28 yearsFour orders of magnitude in 28 years

4004(1971)

8088(1979)

486(1989)

PentiumIII(1999)

Hardware vs. algorithms Hardware vs. algorithms (a simulated comparison)(a simulated comparison)

Sort performance Pentium III vs 4004

0

2000000000

4000000000

6000000000

8000000000

10000000000

12000000000

0 20000 40000 60000 80000 100000 120000

Input size

Pentium III, N^2 sort

4004, NlogN sort

Impact on games Impact on games programmersprogrammers

this gets their attention!this gets their attention! traditionally have been “hit the traditionally have been “hit the

metal” kind of low-level hackers, but:metal” kind of low-level hackers, but: assembly code buys at most a assembly code buys at most a

constant factor speed improvementconstant factor speed improvement• asymptotic complexity dominates any asymptotic complexity dominates any

constantconstant time to market matters time to market matters a lota lot

Recent developmentsRecent developments

classic algorithm analysis analyzes classic algorithm analysis analyzes instruction counts, instruction counts, but is this but is this thethe bottleneck?bottleneck?• today, memory accesses often dominate today, memory accesses often dominate

performance (cache misses)performance (cache misses) analyze cache effects of algorithm designanalyze cache effects of algorithm design some surprising resultssome surprising results

• Mergesort, O(N log N) can outperform radix Mergesort, O(N log N) can outperform radix sort O(N), due to radix sort’s poor localitysort O(N), due to radix sort’s poor locality

Simple DS & A application: Simple DS & A application: mazesmazes

Data structure for a maze?Data structure for a maze?• Simple: 2D matrix of cells, but walls?Simple: 2D matrix of cells, but walls?• Forget cells, just store wallsForget cells, just store walls

Random maze, using Disjoint Sets Random maze, using Disjoint Sets ADT:ADT:• Pick a random wallPick a random wall

– If cells on either side are in a different setIf cells on either side are in a different set– Remove wall, and Union the two cellsRemove wall, and Union the two cells

Java demoJava demo

Aside: union-find for Aside: union-find for equivalence relationsequivalence relations

A set of elements and a list of known A set of elements and a list of known equivalence relations equivalence relations (e.g., reachable, not (e.g., reachable, not <)<)• Query whether two elements are related Query whether two elements are related

(find)(find)– find(a) == find(b)find(a) == find(b)

• Establish equivalence (union)Establish equivalence (union) Brute force: an array a[Brute force: an array a[ii] = # of set ] = # of set ii is in is in

• Find is O(1), but Union is O(N)Find is O(1), but Union is O(N)

Union-findUnion-find

Elegant: nodes point to elements in Elegant: nodes point to elements in setset• In find, collapse nodes to point to root In find, collapse nodes to point to root

Complexity becomes quite good: Complexity becomes quite good: • Worst-case Worst-case (M (M(M,N)), where(M,N)), where

= 1/Ackerman(M,N) , where = 1/Ackerman(M,N) , where A(1,j) = 2j for j>=1A(1,j) = 2j for j>=1A(i,1) = A(i-1, 2) for i >=1A(i,1) = A(i-1, 2) for i >=1A(i,j) = A(i-1, A(i,j-1)) for i,j >=2A(i,j) = A(i-1, A(i,j-1)) for i,j >=2

ProjectsProjects

Goals - incorporate a broad range of Goals - incorporate a broad range of issuesissues• algorithms, OS issues, OO programming, …algorithms, OS issues, OO programming, …

Suitable framework: Suitable framework: interactive interactive simulationssimulations• original motivation for OO programmingoriginal motivation for OO programming• natural platform for multithreadingnatural platform for multithreading• simulation scale simulation scale stressesstresses algorithms and algorithms and

data structuresdata structures

Bridge to research issuesBridge to research issues

Games provide a natural transition to Games provide a natural transition to studying current researchstudying current research• an abundance of distribution issues an abundance of distribution issues • games on consoles games on consoles areare operating systems operating systems

– managing shared resourcesmanaging shared resources– developing abstractionsdeveloping abstractions– soft real-time issuessoft real-time issues– synchronizationsynchronization

• Embedded extension languages are common Embedded extension languages are common (e.g., Quake-C, Crobots), safety, speed.(e.g., Quake-C, Crobots), safety, speed.

Distribution issue: Distribution issue: interaction latencyinteraction latency

Goal: users interact with Goal: users interact with perceivedperceived zero latencyzero latency

Environment: one server, many Environment: one server, many clients, with star topologyclients, with star topology

Interaction mode: users deliver Interaction mode: users deliver projectiles at each otherprojectiles at each other

Problem: high and/or variable Problem: high and/or variable latency to serverlatency to server

Solution: client cachingSolution: client caching

Clients cache players’ motion Clients cache players’ motion vectorsvectors• Allows piecewise prediction of future Allows piecewise prediction of future

positionposition• Server sends position/vector updatesServer sends position/vector updates• Client display accumulates increasing Client display accumulates increasing

errorerror• Collision detection performed at clientCollision detection performed at client

Client-cached motion Client-cached motion vectorvector

observati

on predictions

Problem: cache Problem: cache consistencyconsistency

Sharp turns or changes in velocity Sharp turns or changes in velocity invalidate cached stateinvalidate cached state

Client collision detections are not finalClient collision detections are not final Server arbitrates disputes adaptivelyServer arbitrates disputes adaptively

• field-weapons: decreased damagefield-weapons: decreased damage• spot weapons: high coherence, thus are less spot weapons: high coherence, thus are less

responsiveresponsive Dynamic, Adaptive Relaxed Consistency Dynamic, Adaptive Relaxed Consistency

Policy (!)Policy (!)

Another research issue: Another research issue: timetime

Lots of software doesn’t care about Lots of software doesn’t care about when things happen - faster is betterwhen things happen - faster is better• Excel, Word, Netscape, …Excel, Word, Netscape, …

Other applications really do care Other applications really do care about time - faster is about time - faster is notnot better for: better for:• Video game logic, streaming media, ...Video game logic, streaming media, ...• Airplanes, cars, medical equipmentAirplanes, cars, medical equipment

Soft vs. hard “real time”Soft vs. hard “real time”

Real timeReal time software software

Specifies to the operating system when Specifies to the operating system when things should run, and for how longthings should run, and for how long• Video frame decode, physics model Video frame decode, physics model

computationcomputation Guarantees vs. best effortGuarantees vs. best effort

• hard vs. soft real-timehard vs. soft real-time

The interface is low level, and The interface is low level, and hard to usehard to use How to get the system to make it easier?How to get the system to make it easier?

Real-rateReal-rate — a new — a new abstraction for systems abstraction for systems softwaresoftware

Applications are “pipelines” of processesApplications are “pipelines” of processes One pipeline element is a One pipeline element is a real-timereal-time

driverdriver The system allocates resources to match The system allocates resources to match

the application’s the application’s rate requirementsrate requirements

server

network client

Display–30 fps

Real rate & software Real rate & software feedbackfeedback

System uses feedback to monitor System uses feedback to monitor and allocate resourcesand allocate resources• Monitors progress of each connectionMonitors progress of each connection• Allocates more or less of each Allocates more or less of each

resource to keep up with rate resource to keep up with rate requirementrequirement

servernetwork client

Display–30 fps

OGI’s Quasar research OGI’s Quasar research applied to gamesapplied to games

More axes of adaptivityMore axes of adaptivity• sound, richer video adaptivity, consistency as sound, richer video adaptivity, consistency as

a QoS parametera QoS parameter Enhancing scalabilityEnhancing scalability

• Distributed filters embedded in active networkDistributed filters embedded in active network Hybrid media handlingHybrid media handling

• synthetized and real-time audio/video synthetized and real-time audio/video combinedcombined

• adaptive mixingadaptive mixing

Questions?Questions?

Other ideas?Other ideas?