CSCE 552 Spring 2009

Network/Multiplayer and Audio

By Jijun Tang

Announcements - 1

April 20th (Monday): third presentation Expected: working demo from start to finish, with

at least one level finished You can send me your current code so I can

play it a bit Homework 4

Evaluate the best and worst games with AI, strength and weakness

6 points total Due April 15th

Announcements - 2

Public Demo 2:00pm, May 5th, Tuesday Location: TBA

Set up Show the design (ppt) Show the game (demo) Allow others to play

PathPlannerApp Demo

Characteristics of Grids

Fast look-up Easy access to neighboring cells Complete representation of the level

Waypoint Graph

A waypoint graph specifies lines/routes that are “safe” for traversing

Each line (or link) connects exactly two waypoints

Navigation Meshes (continued)

Random Trace (continued)

How will Random Trace do on the following maps?

Overall Structure of the Algorithms

1. Create start point node – push onto open list2. While open list is not empty

A. Pop node from open list (call it currentNode)B. If currentNode corresponds to goal, break from step 2C. Create new nodes (successors nodes) for cells around currentNode and push them onto open listD. Put currentNode onto closed list

Breadth-First

Finds a path from the start to the goal by examining the search space ply-by-ply

Best-First (continued)

A*

Uses both heuristic cost and given cost to order the open list

Final Cost = Given Cost + (Heuristic Cost * Heuristic Weight)

Start Node and Costs

F=G+H

Cost Map

Multiplayer Modes:Event Timing

Turn-Based Easy to implement Any connection type

Real-Time Difficult to implement Latency sensitive

Multiplayer Modes:Shared I/O

Input Devices Shared keyboard layout Multiple device mapping

Display Full Screen and Screen Swap Split Screen

Split Screen

Multiplayer Modes:Connectivity

Non Real-Time Floppy disk net Email Database

Direct Link Serial, USB, IrD, … (no hops)

Circuit Switched (phones) Dedicated line with consistent latency

Packet Switched Internet Shared pipe

Protocols:Protocol Design

Packet Length Conveyance Acknowledgement Methodology Error Checking / Correcting Compression Encryption Packet Control

Protocols:Packets

Packets Header = Protocol Manifest Payload

Gottchas Pointers Large/Variable Size Arrays ADTs Integer Alignment Endian Order Processor dependant Intrinsic Types (int and long) Unicode vs. ASCII Strings

Protocols: Request for Comments

RFC web site http://www.rfc-editor.org/

Protocol Specifications Definitive Resource Public Criticism Future Protocols

Example Packets

IP Packet TCP Packet

Protocol Stack: Open System Interconnect

R outer

Sender R eceiver

Ne two rk

D a ta L in k

Physica l

Ne two rk

D a ta L in k

Physica l

App lica tio n

Pre se n ta tio n

Session

T ra nspo rt

Ne two rk

D a ta L in k

Physica l

App lica tio n

Pre se n ta tio n

Session

T ra nspo rt

Ne two rk

D a ta L in k

Physica l

G am e E vents

G am e P ac k et iz at ion

Connec t ion & Data E x c hange

Input UpdatesS tate Updates

S eria liz at ionB uffering

S oc k ets

TCPUDP

IP

E thernet (M A C)

W ired (C5, Cable)F iber O pt ic sW ire les s

Another View

Protocol Stack:Physical Layer

Bandwidth Width of data pipe Measured in bps = bits per second

Latency Travel time from point A to B Measured in Milliseconds You cannot beat the physics

The Medium Fiber, FireWire, IrD , CDMA & other cell

SerialUSB1&2

ISDN DSL CableLAN

10/100/1GBaseT

Wireless802.11a/b/g

Power Line

T1

Speed(bps)

20K12M

480M128k

1.5M down896K up

3M down256K up

10M100M

1G

b=11Ma,g=54M

14M 1.5M

Table: Max Bandwidth Specifications

Protocol Stack:Data Link Layer

Serializes data to/from physical layer Network Interface Card (NIC)

Ethernet MAC Address

Protocol Stack:Network Layer

Packet Routing Hops Routers, Hubs, Switches

Internet Protocol (IP) Contains Source & Destination IP Address IPv4

Widespread Infrastructure

IPv6 Larger IP address

Protocol Stack:Network Layer: IP Address

Unicast Static DHCP

Multicast Requires multicast capable router

Broadcast Local Directed

Loop Back Send to self

Multicast and Broadcast

Protocol Stack:Network Layer: DNS

Domain Name Service Converts text name to IP address Must contact one or more DNS servers to resolve Local cache resolution possible

Game Tips Store local game cache to use when DNS out of

order. DNS resolution often slow, use cache for same

day resolution.

DNS

Protocol Stack:Transport Layer Manage data deliver between endpoints

Error recovery Data flow

TCP and UDP used with IP Contains Source and Destination Port TCP is connection-oriented, UDP is

connectionless Port + IP = Net Address

Port Range = 0-64k Well known Ports 0-1k

Protocol Stack:Transport Layer: TCP

Guaranteed Correct In Order Delivery Acknowledgement system: Ack, Nack, Resend Checksum Out of Band

Connection Required Packet Window Packet Coalescence (united) Keep Alive

Streamed Data User must serialize data

Protocol Stack:Transport Layer: UDP

Non Guaranteed Delivery No Acknowledgement system May arrive out of order Checksum

Not Connected Source not verified Hop Count Limit = TTL (time to live) Required for Broadcasting

Datagram Sent in packets exactly as user sends them

TCP/UDP Comparison

Protocol Stack:Session Layer

Manages Connections between Apps Connect Terminate Data Exchange

Socket API live at this layer Cross platform Cross language

Protocol Stack:Session Layer: Sockets

Based on File I/O File Descriptors Open/Close Read/Write

Winsock Provides standard specification implementation

plus more Extension to spec prefixed with “WSA” Requires call to WSAStartup() before use Cleanup with WSAShutdown()

Protocol Stack:Session Layer: Socket Design

Modes Blocking Non-Blocking

Standard Models Standard Select

Extended Models Windows

WSAEventSelect I/O Completion Ports

Unix Poll Kernel Queues

Socket Programming

Socket Programming-2

Socket Programming -3

Socket Programming -4

Protocol Stack:Presentation Layer

Prepares App Data for Transmission Compression

Pascal Strings String Tables Float to Fixed Matrix to Quaternion

Encryption Endian Order

When used cross platform or cross language Serialize

Pointers Variable Length Arrays

Endian Order

Protocol Stack:Presentation Layer: Buffering

Packet Coalescence

Induced Latency

Dead Data

Large Packets

Protocol Stack:Application Layer

Handles Game Logic Update Models

Input Reflection State Reflection

Synchronization Dead Reckoning AI Assist Arbitration

Real-Time Communications:Connection Models

Broadcast Good for player discovery on LANs

Peer to Peer Good for 2 player games

Client / Server Good for 2+ player games Dedicated lobby server great for player

discovery

Real-Time Communications:Peer to Peer vs. Client/Server

Broadcast Peer/Peer Client/Server

Connections 0Client = 1Server = N

N = Number of players

Broadcast Peer/Peer Client/Server

Send 1 N-1Client = 1 Server = N

Receive N-1 N-1Client = 1Server = N

1

1

N

x

x

P 1 P 2

P 1

P 2 P 3

P 1 P 4

P 2 P 3

P 1

P 2 P 5

P 3 P 4

2 p lay er s1 c o n n ec tio n

3 p lay er s3 c o n n ec tio n s

4 p lay er s6 c o n n ec tio n s

5 p lay er s1 0 c o n n ec tio n s

Security:Encryption Goals

Authentication

Privacy

Integrity

Security:Encryption Methods

Keyed Public Key Private Key Ciphers

Message Digest Certificates IPSec (Internet Protocol Security)

IPSec

Security:Copy Protection

Disk Copy Protection Costly Mastering, delay copies to ensure

first several months’ sale Invalid/Special Sector Read

Code Sheets Ask code from a line in a large manual

Watermarking

Security:Execution Cryptography

Code Obfuscation Strip Symbols: int numOfPlayer;int a1;

Heap Hopper Move sensitive data around in the heap to avoid

snapshots Stack Overrun Execution

Check for buffer overflow Timer Hacking

Do not allow time to change backward DLL Shims

Privacy

Critical data should be kept secret and strong encrypted: Real name Password Address/phone/email Billing Age (especially for minors)

Using public key for transforming user name and password

Security:Firewalls

Packet Filter Allow a packet based on its headers

Proxies Check contents

Circuit Gateways Setup secure sessions

Security:Firewalls: Network Address Translation

LAN Add re ss W AN Add re ss

1 92 .1 6 8 .1 .1 :2 0 0 2 4 .1 5 .1 .1 1 8 :2 0 0

1 92 .1 6 8 .1 .1 :2 0 1 2 4 .1 5 .1 .1 1 8 :2 0 1

1 92 .1 6 8 .1 .2 :1 9 9 2 4 .1 5 .1 .1 1 8 :1 9 9

1 92 .1 6 8 .1 .2 :2 0 0 2 4 .1 5 .1 .1 1 8 :4 0 0 0*

I S P

W AN I P2 4 .1 5 .1 .1 1 8

L AN I P1 9 2 .1 6 8 .1 .1

L AN I P1 9 2 .1 6 8 .1 .2

N A TR eq u es ted P o r ts2 0 02 0 1

R eq u es ted P o r ts1 9 92 0 0

R o u ter

Security:Firewalls: NAT Traversal

Port Forwarding

Port Triggering

DMZ

Determining WAN IP

Summary:Further Study

Socket Programming Serial Communication Server Design Network Gear & Infrastructure VOIP Tools of the Trade Unit & Public Beta Testing Middleware Databases Web Development Asynchronous Programming

Audio Programming

Audio Programming

Audio in games is more important than ever before

Programming Basic Audio

Most gaming hardware has similar capabilities (on similar platforms) Mostly programming interfaces differ

Learning fundamental concepts of audio programming is important

API Choices

DirectSound (part of DirectX API) Only available on Windows platforms XACT for XNA

OpenAL Newer API Available on multiple platforms

Proprietary APIs Typically available on consoles

3rd Party Licensable APIs Can offer broad cross-platform solutions

Analog Sound Wave

Basic Audio Terminology and Physics

Amplitude Measurement of a sound wave’s pressure

Frequency Measurement of the interval between wave cycles,

typically measured in Hertz Pitch

The perception of frequency Tuning

Musical distribution of frequencies over keys Decibel

Measures sound amplitude

Digital Representation of a Sound Wave

Sampling

Most common technique Sampling involves measuring the amplitude

of the analog wave file at discrete intervals The frequency of sampling is known as sampling

rate Each sample is typically stored in a value ranging

from 4 to 24 bits in size The size of the sample value in bits is known as

the ‘bit depth’ Music CDs have a sample rate and bit depth of

44.1 kHz (samples/sec) and 16 bits (sample size)

Quantization Error in Sampling

Bit Depth and Signal Noise

Bit depth of sample data affects signal noise Signal to noise ratio = number of available bits /

1 For example, 8-bit samples have a 256:1 SNR

(~48 dB), and 16-bit samples have a 65,536:1 SNR (~96 dB)

Decibel ratio is calculated using 10 x log10 (ratio) or 8.685890 x log e (ratio)

Sampling Frequency and Frequency Reproduction

Sampling frequency affects range and quality of high-frequency reproduction

Nyquist Limit Frequencies up to one-half the sampling

rate can be reproduced Audio quality degrades as frequency

approaches this limit

Sampling Errors vs. Sampling Frequency

Modern Audio Hardware

Samples are piped into sound “channels” Often a hardware pipeline from this point

Various operations, such as volume, pan, and pitch may be applied

3D sounds may apply HRTF algorithms and/or mix the sound into final output buffers.

Sound Playback Techniques

Two basic playback methods:1. Play sample entirely from memory buffer2. Stream data in real-time from storage medium

Streaming is more memory efficient for very large audio files, such as music tracks, dialogue, etc

Streaming systems use either a circular buffer with read-write pointers, or a double-buffering algorithm

Playback

Sample Playback and Manipulation

Three basic operations you should know Panning is the attenuation of left and right

channels of a mixed sound, results in spatial positioning within the aural stereo field

Pitch allows the adjustment of a sample’s playback frequency in real-time

Volume control typically attenuates the volume of a sound

Compressed Audio Format

Compressed audio formats allow sound and music to be stored more compactly

Bit reduction codecs generally are lightweight ADPCM compression is implemented in hardware on all th

e major current video game console systems Psycho-acoustic codecs often have better compres

sion Discard sounds our ears would not typically be able to hea

r Require substantially more computational horsepower to d

ecode

MP3, Ogg Vorbis,Licensing & Patent Issues

The MP3 format is patented Any commercial game is subject to licensing term

s as determined by Fraunhofer & Thompson Multimedia, the holders of the patents

Ogg Vorbis is similar to MP3 in many ways Open source and patent-free (royalty-free)

Be aware of patent and license restrictions when using 3rd party software

3D Audio

Two sets of data required when working in world coordinates: Listener Data

Composed of world position and orientation (virtual microphone in the world)

Source Data Composed of sound position, orientation,

velocity, etc (virtual sound source in the world)

Listener/Source

Environmental Effects

Environmental effects nearly always implemented in hardware

Sound transmission is categorized in three ways Direct transmission Early reflections (echo) Late reflections (reverberation)

Sound Transmission Categories

EnvironmentalEffects Standards

EAX 2.0 and beyond EAX 2.0 developed by Creative Labs and

released as an open standard EAX 3.0 and 4.0 remain proprietary

Creative Labs standards I3DL2

Open standard developed by IA-SIG, similar to EAX 2.0 in functionality

Programming Music Systems

Two common music systems MIDI-based systems

(Musical Instrument Digital Interface) Digital audio streaming systems

(CD audio, MP3 playback, etc)

Advantages and Disadvantages of MIDI

Actual music data size is negligible Easy to control, alter, and even

generate in real-time High quality music is more difficult to

compose and program Only effective if you can guarantee

playback of a common instrument set

Example

Other MIDI-based technologies to be aware of

DLS (DownLoadable Sound) Format A standardized format for instrument defi

nition files iXMF (Interactive eXtensible Music For

mat) New proposed standard for a container fo

rmat for interactive music

Advantages / Disadvantages of Digital Audio Streams

Superb musical reproduction is guaranteed

Allows composers to work with any compositional techniques

Some potential interactivity is sacrificed for expediency and musical quality

Generally high storage requirements

A Conceptual Interactive Music Playback System

Divide music into small two to eight-bar chunks that we’ll call segments.

A network of transitions from segment to segment (including loops and branches) is called a theme.

Playing music is now as simple as choosing a theme to play. The transition map tracks the details.

Advanced Audio Programming

3D Audio Environmental Effects Integration

Audio Scripting and Engine Integration Lip-sync Technology Advanced Voice Playback Voice Recognition

3D Audio Environmental Effects Integration

Environmental effects should be driven by a room’s shape and material composition. Can determining the optimal effect

settings be done automatically? This may be important as game worlds

become larger and more complex

3D Audio Environmental Effects Integration (cont)

Sound occlusion and damping is a particularly difficult problem to solve This is essentially a pathfinding problem

for audio. Doors can dynamically affect a sound’s

properties Very few titles have even attempted a

robust, general-purpose, and automated solution to these problems.

Room Acoustics

Dynamic Occlusion

Audio Scripting and Engine Integration

Very little audio programming should be done by general game programmers

Game Engine should offer robust support for audio triggers and scripts

Engine should deal with audio scripts, not “sound files”

Why is this so important?

Audio Scripting

Many situations require much more information than can be embedded in a linear audio file Sound Variation Sound Repetition Complex Sound Looping Background Ambience

Lip-sync Technology

Lip-sync technology is a blending of audio and visual techniques to create realistic-looking speech by in-game actors. Simple techniques such as waveform amplitude

measurement has worked previously, but… In future titles, it will be considered inadequate. Much work can still be done in this field.

Advanced Voice Playback

Real-time spoken feedback is especially important in sports titles (simulated announcers)

Game are reaching the limits of what current techniques (canned, prerecorded phrases combined in series) can provide.

Again, this is an opportunity for future groundbreaking audio work.

Voice Recognition

Spoken commands are much easier to deliver in certain situations.

A great example of this? Squad-based tactical shooters.

Current generation systems are still very error prone. A great opportunity for breakout audio technology.