UC)SantaCruz Computer)Science)–Game)Design) …€“ Sales)of)many)videogames)larger)than)mostmusic)album) sales) – For)many)musicians,)mighthave)larger)audience)for)video) game)soundtrack)than)for)tradiOonal)album)

Computer Science – Game Design UC Santa Cruz

Adapted from Jim Whitehead’s slides

Sound in games

•  Think about truly memorable games – They almost always have excellent background music and sound effects

– Legend of Zelda, PacMan, Katamari Damacy, LiLle Big Planet, Radiant Silvergun

– Music and artwork style combine to create an overall tone, or mood, for a game

– Done well, this substanOally enhances the overall gameplay experience



Finding/Making Sounds

•  Where can you find music to use in your game? –  Reminder: there is this legal framework called Copyright Law –  CreaOve Commons: use licenses that may allow free, non-‐commercial use

•  hLp://creaOvecommons.org/ •  hLp://archive.org

•  Sites with CreaOve Commons licensed music –  New Grounds, Jamendo

•  hLp://www.newgrounds.com/audio/ •  hLp://www.jamendo.com/en/creaOvecommons/ •  Look for “ALribuOon, Non-‐commercial”

–  “No DerivaOve Works” is OK, so long as you don’t modify •  If you use in your game, make sure you provide aLribuOon

–  Put name of arOst in your game (About page, splash screen, etc.) –  Is polite to send them an email telling them about the use—will make

them jazzed



Finding/Making Sounds (cont’d)

•  Find someone to create music for you – Music student at UCSC, for example –  One team has already put up announcements for a music person

•  It has never been cheaper to create high quality music –  Instruments, microphones, mixing technology are all at historically low prices

–  Has led to a proliferaOon of music –  Biggest problem: finding an audience –  Games provide a good audience –  Sales of many videogames larger than most music album sales

–  For many musicians, might have larger audience for video game soundtrack than for tradiOonal album



Finding/Making Sounds (cont’d)

•  Use your voice! –  Your voice is wonderfully adaptable and expressive

•  Consider: –  Record a raw voice clip –  Bring into an ediOng sodware suite –  Tweak/filter/alter unOl it suits your game –  Can do much worse…

•  Tools –  Audacity

•  hLp://audacity.sourceforge.net/ •  Free, open source sound recorder/editor

–  FL Studio (grown-‐up commercial version of Fruity Loops) •  hLp://flstudio.image-‐line.com/documents/what.html



Playing Sounds in XNA

•  Two ways •  Hard (but powerful) way

–  XACT audio tool •  Cross-‐plahorm audio creaOon tool

– Many neat features –  Edit volume, pitch, looping of sound clips –  Can easily group together sound clips

•  Easy (and 95% sufficient) way – Use Simplified Sound API –  Can start, stop, and pause sound playing – Much, much easier to use



Simple Sound API

•  Two ways to play music – As a song

•  Good for background music, or other long sounds

– As a sound effect •  Good for short duraOon sounds



XNA Simple Sound API

•  Supported music types: wav, wma, mp3 •  Add sound into project Contents folder

– Audio files treated like other files in content pipeline –  Copy sound file into project Contents folder –  Right-‐click on Contents folder inside Visual Studio C# Express

•  Add ExisOng Item … select audio file you just copied in – Will now be visible inside Visual Studio – Need to double-‐check the Content Processor

•  Sound Effect – XNA Framework – sound effects •  Song – XNA Framework -‐ songs



XNA Song API

•  Create a variable of type Song –  Used to load songs via the content pipeline –  Song mySong;

•  Load sound file –  mySong = Content.Load<Song>(@”{name of song file

without extension}”)

•  To play a sound, call Play() method on MediaPlayer object –  MediaPlayer.Play(mySong);

•  To pause/resume, call Pause()/Resume() on MediaPlayer object –  MediaPlayer.Pause(); // no argument –  MediaPlayer.Resume(); // no argument



XNA Sound Effect API

•  Create a variable of type SoundEffect – Used to load sounds via the content pipeline –  SoundEffect soundEffect;

•  Load sound file –  soundEffect = Content.Load<SoundEffect>(@”{name of

sound file without extension}”)

•  To play a sound, call Play() method on SoundEffect object –  Returns a soundEffectInstance object –  Can use this to stop, pause, and restart sound –  SoundEffectInstance soundEffectInstance =

soundEffect.Play();



Demo of Song and Sound Effect API

// Demo of use of Songs and Sound Effects inside XNA

•  CauOon: TreaOng a song as a sound effect can lead to very long compile Omes – SoluOon: keep sound effects short



3D in XNA



2D to 3D •  Many current games use 3D graphics

– Much more complex than 2D graphics

– This course will provide a basic overview of 3D graphics

– CMPS 160, 161, 164 provide greater depth

Ratchet and Clank



3D Camera

•  Analogy –  2D is like puvng sOckers on a page

•  Place sOcker (sprite) at x,y coordinate •  If a sOcker is placed at 50,50, you see it

–  3D is like recording a video with a camera •  What is recorded (shown on screen) is what camera sees

•  Can have objects in a scene that aren’t visible –  Can have 3D object at 50,50,50, but if camera is poinOng in the opposite direcOon, won’t see it!

•  Introduces rotaOon –  Camera can potenOally be rotated around all 3 axes

–  Objects can also be rotated around 3 axes

–  Affects what shows up on screen



3D Coordinate System •  3D graphics requires use of x,y,z coordinates •  So, which direcOon is posiOve z?

–  Is it back away from you, or towards you? –  Either choice would work, need to pick one

•  Right handed vs led handed coordinate systems –  XNA uses right handed coordinate system

•  Place hands, palms up –  Point fingers in direcOon of posiOve X –  Curl fingers in direcOon of posiOve Y –  Thumb is poinOng in direcOon of posiOve Z

Right-‐handed coordinate system



Camera

•  Camera is comprised of two Matrix objects – View matrix holds informaOon on

•  LocaOon of camera in world •  Camera direcOon •  Camera orientaOon

– Projec3on matrix holds informaOon on •  View angle •  Aspect raOo •  Near and far plane LocaOon (x,y,z)

OrientaOon

DirecOon



Matrix Structure

•  XNA provides a Matrix structure – A 4x4 matrix, in row vector layout

•  Row vector matrices view vectors as a row from led to right •  column vector matrices view vectors as a column from top to boLom

–  Built-‐in matrix operaOons •  +, -‐, *, /, ==

– Also, convenience matrices •  IdenOty, Up, Down, Led, Right

–  Large number of convenience methods •  RotaOons, views into 3D world, determinants, invert



Vector3 Structure

•  Represents either: –  An X, Y, Z coordinate, or, –  Distances along X, Y, Z coordinates

(e.g., a vector) •  Oden a unit vector

–  all values between 0 and 1 –  X, Y, Z properOes (floats)

•  Built-‐in operators –  +, -‐, *, /, ==, !=

•  Convenience vectors –  UnitX, UnitY, UnitZ, Up, Down, Led, Right

•  Many convenience methods –  InterpolaOons, rotaOons, distance, dot product, normalizaOon

(x,y,z) coordinate

x

y

z

(x,y,z) vector



CreaOng an XNA Camera – View Matrix

•  View matrix –  Use CreateLookAt method of Matrix structure –  Parameters (all Vector3)

•  cameraPosiOon – loca3on of camera (x,y,z) •  cameraTarget – coordinate of point where camera is looking •  cameraUpVector – vector indicaOng up posiOon

cameraPosiOon (x,y,z)

cameraUpVector cameraTarget (x,y,z)



CreaOng an XNA Camera – ProjecOon Matrix •  ProjecOon Matrix

–  Use CreatePerspecOveFieldOfView method –  Parameters (all floats)

•  fieldOfView – angle of camera view, in radians –  Typically 45degrees – pi/2 radians

•  aspectRaOo –  Typically width of screen divided by height of

screen •  nearPlaneDistance

–  Distance from camera to near viewing plane –  Objects between camera and near plane are

not shown! •  farPlaneDistance

–  Distance from camera to far viewing plane –  Objects beyond far plane are not shown!

cameraPosiOon (x,y,z)



Drawing Triangles

•  All complex 3D shapes seen in games are composed of a series of triangles –  A triangle has 3 points, one for each corner

•  Points are more typically known as ver3cies •  Minimum number of points to unambiguously define a plane

•  VertexPosiOonColor object –  Represents the x,y,z locaOon of a vertex –  Also has a color for the vertex

–  VertexPositionColor v = new VertexPositionColor(new Vector3(0,1,0), Color.Blue);

–  Need 3 verOcies to draw a triangle



Vertex DeclaraOon

•  XNA requires you to tell the graphics device what kind of vertex data you will be using – Unclear why XNA can’t just figure this out, or handle mulOple types seamlessly

–  Probably due to structure of DirectX API, or capabiliOes of graphics hardware

–  For now, treat as a must-‐do, black box –  Put following in your main, derived from Game class – GraphicsDevice.VertextDeclaration = new

VertexDeclaration(GrahpicsDevice, VertexPositionColor.VertexElements);



Actually drawing the triangles

•  In XNA, all 3D rendering is handled by a shader –  Shaders defined using High Level Shader Language (HLSL)

–  Permits creaOon of wide range of visual effects – More on shaders in a few classes

•  XNA provides a default shader –  Called BasicEffect – Will use this for now

•  BasicEffect is a type of effect –  Effects contain a series of EffectPass –  Each pass handles some aspect of puvng things on screen



Using Basic shader

•  Five steps •  Create Shader •  Copy over camera informaOon

•  Iterate through EffectPasses

•  Examine source code from example in Chapter 9 of XNA 3.0



Using Basic shader

Five steps: 1.  Create Shader

–  BasicEffect effect = new BasicEffect(GraphicsDevice, null); 2.  Copy over camera informaOon

–  effect.View = camera.view; –  effect.Projection = camera.projection;

3.  Set world matrix –  effect.World = … (more on this in a few slides)

4.  Enable vertex capabiliOes (varies by Vertex type) –  Effect.VertexColorEnabled = true; // for VertexPositionColor –  Effect.Texture = myTexture; // for VertexPositionTexture

Effect.TextureEnabled = true; 5.  Iterate through EffectPasses

–  Call to DrawUserPrimiOves inside EffectPass puts triangles on screen



IteraOng Through Effect Passes

•  Each Effect has calls to begin()/end() •  Effects are comprised of passes

–  Each pass requires a call to begin()/end()

effect.Begin(); foreach (EffectPass pass in effect.CurrentTechnique.Passes) { pass.Begin(); GraphicsDevice.DrawUserPrimitives<VertexPositionColor> (PrimitiveType.TriangleStrip, verts, 0, 1); pass.End(); } effect.End();

Draws verOcies

Passes

# of triangles (the “primiOve shape” in this context) to draw

Index into verts array



Triangle Drawing •  Examine this line of code

– GraphicsDevice.DrawUserPrimitives<VertexPositionColor>

(PrimitiveType.TriangleStrip, verts, 0, 1); –  What is a TriangleStrip?

•  Three ways to draw triangles –  Triangle List

•  Each set of three verOcies defines a triangle •  Memory inefficient, since triangles oden share edges in complex 3D meshes

–  Triangle Strip •  Builds triangle out of first three verOcies •  Each addiOonal vertex creates new triangle using that vertex, plus previous two verOcies

–  Triangle Fan •  Each addiOonal vertex creates new triable using that vertex, the previous vertex, plus the first vertex

hLp://escience.anu.edu.au/lecture/cg/surfaceModeling/image/ surfaceModeling015.png



World Matrix •  Each triangle has 3 verOcies, and each vertex has an x,y,z

posiOon –  This posiOon is given with respect to an origin locaOon –  That is, locaOon is with respect to a local coordinate system

•  World matrix –  Translates from the local coordinate

system to the world (i.e., visible on screen) coordinate system

Local coordinate system offset, no rota3on (Note: example uses led handed coordinate system, XNA uses right-‐handed coordinates) Source: MSDN DirectX documentaOon

Local coordinate system offset and rotated www1.adept.com/main/KE/DATA/V%20Plus/V%20Language%20User/images/World+yaw.gif

local

world



TranslaOon

•  A transla3on shids the local coordinate system relaOve to the world coordinate system

•  XNA provides a method for this – Matrix.CreateTranslaOon

•  3 parameters are x,y,z movements •  Matrix.CreateTranslation(0.01f, 0, 0); // Shift right (positive)

along x axis •  Matrix.CreateTranslation(-0.01f, 0, 0); // Shift left (negative)

along x axis •  MulOply world matrix by translaOon matrix to cause shid –  All translaOons and rotaOons in 3D graphics accomplished via matrix mulOplicaOon



RotaOon

•  A rota3on shids the local coordinate system by an angle relaOve to the world coordinate system

•  XNA helper methods – Matrix.CreateRotaOonX, Matrix.CreateRotaOonY, Matrix.CreateRotaOonZ

•  RotaOons around single axes •  Matrix.CreateRotationY(angle in radians);

– Matrix.CreateFromYawPitchRoll •  Rotate around mulOple axes •  Matrix.CreateFromYawPitchRoll(yaw rad., pitch rad.,

roll rad.) •  DemonstraBon of example triangle drawing code



Triangles are fine, but models are beLer

•  Today’s 3D games have a large number of objects –  TheoreOcally possible to create these objects by manually wriOng C# code to create each individual triangle

–  In pracOce, this is rarely done. –  Far beLer to use a 3D modeling tool

•  Maya, XSI, 3DS Max, Blender – Allows arOsts to create objects in world – Allows programmers to focus on behavior of objects – Modeling tools permit much faster creaOon and ediOng of models



3D Modeling

•  Several ways to model a 3D object –  Polygonal modeling

•  Objects are subdivided into a series of polygons (triangles) •  Can only approximate curved surfaces •  Dominant modeling form in computer games and computer graphics due to speed of rendering

– NURBS •  Surfaces are defined by spline curves •  Curves defined and controlled by control points

–  Splines and patches •  Curved lines define surfaces. Between polygons and NURBS

–  PrimiOves modeling •  Objects built up from primiOve shapes (balls, cubes, cylinders, etc.)



3D Model formats

•  There is a huge number of 3D model formats – No dominant standard

–  Interchange among models is oden lossy

•  XNA supports two 3D model formats –  .X (DirectX) –  .FBX

•  Originally for FilmBox by Kaydara, then Alias, now Autodesk



Models in XNA •  Model

–  Represents some enOty •  A person, a car, or potenOally a complex scene with many parts

–  Typically used to represent one logical object (e.g., a person) that has mulOple parts (head, arms, legs, etc.)

–  A model contains mulOple meshes and bones •  Mesh

–  Represents a single physical object –  Triangles, textures, shaders, etc. –  XNA ModelMesh class

•  Bone –  Represents placement of each mesh relaOve to

other meshes –  A transformaOon matrix

Model

ModelMesh

1

N

Car

Body +bone

Wheel +bone

Door +bone

Bone

1

N



Models in XNA (cont’d) •  ModelMesh contains:

–  List of ModelMeshPart –  List of Effects –  VerOcies for triangles that

comprise this mesh •  VertexBuffer

–  Also has a bounding sphere •  ModelMeshPart

–  Represents a set of triangles that share the same materials (e.g., shader, or Effect)

–  Has indexes into the ModelMesh –  StarOng index, number of triangles,

number of primiOves to use from parent ModelMesh’s VertexBuffer

Model

ModelMesh

1

N

Bone

1

N

ModelMeshPart Effect

1

N

1

N

VertexBuffer

1

1

VerOcies

1

N

Effect 1

1

StartIndex NumVerOcies PrimiOveCount

1

1

1

1



Working with Models in XNA

•  Bring model into XNA project inside Visual Studio

•  Load model into XNA via Content manager

•  Draw model by iteraOng through all contained meshes



ImporOng Models into Visual Studio

•  Import model into Content directory – Copy .x or .|x file, along with associated texture images, into Content directory in XNA project

–  Inside Visual C#, right-‐click on Content directory •  Add… Add ExisOng Item •  Select .x or .|x file

– Similar process to adding bitmap textures, etc.



Loading Model via Content manager

•  Models are loaded via the content manager – Model model = Content.Load<Model>(@”{name of my model without .x or .Ex extension}”)

•  XNA parses the model file, and fills in verOcies, textures, and effects in Model, and ModelMeshes –  In XNA, this is a robust operaOon, big Ome savings –  In many open source 3D game engines, model import can be a big problem

– At present, typically is not safe to assume model import will work smoothly in a given 3D game engine

– Need to test your tool chain



Drawing a Model

•  Iterate through all of the meshes –  Iterate through each effect for each mesh

•  Set lighOng, camera, and world for each effect –  Draw each mesh using Draw() method on ModelMesh class

foreach (ModelMesh mesh in model.Meshes) { foreach (BasicEffect be in mesh.Effects) { be.EnableDefaultLighting(); be.Projection = camera.projection; be.View = camera.view; be.World = world * mesh.ParentBone.Transform; } mesh.Draw(); } •  Examine example code from Chapter 10 of Learning XNA 3.0



More on Models

•  ExplanaOon of parts of XNA models – hLp://blogs.msdn.com/shawnhar/archive/2006/11/20/models-‐meshes-‐parts-‐and-‐bones.aspx

Documents

UC)SantaCruz Computer)Science)–Game)Design) …€“ Sales)of)many)videogames)larger)than)mostmusic)album) sales) – For)many)musicians,)mighthave)larger)audience)for)video) game)soundtrack)than)for)tradiOonal)album)