CS 354Introduction

Mark KilgardUniversity of TexasJanuary 17, 2012

CS 354 2

CS 354—Computer Graphics

Instructor: Mark Kilgard Principal Software Engineer, NVIDIA Previously at Silicon Graphics

Teaching Assistant: Randall Smith Ph.D. student with Dr. Fussell TA’ed the course last semester

Burdine Hall (BUR) 116 Tuesday/Thursday, 9:30-11:00 a.m.

CS 354 3

Expectations For us

Teach practical graphics expertise

Well-prepared lectures Grades fairly reflect

your participation and performance

Available to you Use my office hours Other times, use Randy If not satisfied, contact me

mjk@cs.utexas.edu

For you Attendance at every

class Expect regular quizzes at

the beginning of class

Programming assignments You need to know C/C+

+ Use office hours if you

need help No cheating

CS 354 4

Grading

Testing 60% Daily quizzes 10%

2-3 questions, easy

Homework 10% Occasional, mostly

math questions

Exams 40% Mid-term 15% Final 25%

Software projects 40% Simple Rendering Object loader Interaction Shading Performance analysis

CS 354 5

Textbook

Interactive Computer Graphics: A Top-Down Approach With Shader-Based OpenGL by Edward Angel and Dave Shreiner Addison-Wesley, 6th edition

Regular readings assigned Match up with lecture topics

CS 354 6

Helpful Resources Learning OpenGL

OpenGL-oriented books

Supplemental books

OpenGLProgramming

Guidea.k.a. “the red book”

OpenGLSuperBible

OpenGLA Primer

Eric Lengyel’sMathematics for 3D

Game Programmingand Computer Graphics

Real-Time Rendering byEric Haines, Tomas Akenine-Moller,Eric Haines, Naty Hoffman

CS 354 7

Computer Graphics Nexus of several disciplines

HumanPerception

ArtisticExpression

Physicsof Light

Geometry andMathematics of

Surfaces

Computer ScienceVLSI Hardware

Design Display & InputTechnology

Animation &Simulation

CS 354 8

Roles for Computer Graphics

[Pixar 2010]

Story telling

Product design

[CATIA]

Roles for Computer Graphics

CS 354 9

Roles for Computer Graphics

Training

[Commercial simulators]

Gaming

[Skyrim]

CS 354 10

Roles for Computer GraphicsUser interfaces

[Android 4.0]

Navigation

[Audi]

CS 354 11

Roles for Computer Graphics

Printing Digital imaging & video

[HP Deskjet]

[Canon]

CS 354 12

What does computer graphics study?

Computer-based simulation of Shape

Appearance

Motion

[Litke et.al. 2001]

[Chai & Hodgins, 2005]

[Sloan et.al. 2005]

CS 354 13

Not covered in the class

Digital content creation No Photoshop, no Maya or 3D Studio Max Computer Science class, not an art class

C/C++ programming I expect you know C or C++ under Linux

Not just the language Need to know debugging and software practices

Since course’s programming projects assume Linux software development

Use the Computer Science labs (ESN or Painter)

CS 354 14

Visual Thinking Human visual system = highest bandwidth input to your brain

Very natural to want computers “feeding” this input to your brain Because people think visually

Innate intuition for 3D imagery in particular Computer graphics

Takes an abstract representation of a “scene” within a computer’s memory and converts it to a concrete signal (an image, or animating images) representing a view of that scene

Computer graphics practitioners have a good practical and theoretical understanding of how to do this

Amazing progress in the last 40 years Your brain

Takes concrete signals (images, the visual world) and converts those signals (back!) into an abstract representation of a scene

We have only the most rudimentary notions of how this process works

Teaching a computer to do this is the field of “computer vision”

CS 354 15

Reductionist Approach Can’t simply “take a picture” like a camera

Instead images are “synthesized” from an abstract model of a scene and its view

Must build graphics out of little bits of work and data Philosophers and scientists break down phenomenon to

smallest observable units Examples

Philosophical atoms (Leucippus, ancient) Triangles (Plato, ancient) Chemical atoms (Dalton, pre-modern) Physical atoms (Bohr, modern) Sub-atomic quantum particles, waves, strings

Reductionism in graphics Pixels, vertexes, and triangles (Plato redux?) Discretization pervades, ideal for digital computers

CS 354 16

Your Modern World View:Discrete Information, Particularly Images You believe in digital information

Impossible to avoid this belief in a modern, computerized world

Text is digital, video is digital, music is digital, communication is digital, even identity is becoming digital

You believe exact copies are possible, even common Pervasive new belief that’s incognizant to world just 50

years ago Discrete images implies pixels

Descartes’s Cartesian plane computerized Image ≈ grid of discrete color So images are simply numbers

So can be processed as numbers!

CS 354 17

Shape: Objects to Triangles

Digital artists approximate solid object shape with meshes of triangles Was Plato right?

More triangles makes the mesh an increasinglyaccurate approximation of the bunny shape

CS 354 18

What will you learn

Fundamentals of computer graphics Transformations and viewing Rasterization and ray tracing Lighting and shading Graphics hardware technology Mathematics for computer graphics

Practical graphics programming OpenGL programming Shader programming Performance analysis

CS 354 19

Expect programming projects using OpenGL Explicitly not just an “OpenGL” class

But you will learn and use OpenGL When relevant Direct3D is discussed too

You’ll use GLUT so programs can work on Windows, Mac, and Linux

Plenty of resources to learn OpenGL www.opengl.org

CS 354 20

What is OpenGL? Its specification is titled “The OpenGL Graphics

System” Not just for 3D graphics; imaging too “GL” standard for “Graphics Library” “Open” means industry standard meant for broad

adoption with liberal licensing Standardized in 1992

By Silicon Graphics And others: Compaq, DEC, Intel, IBM, Microsoft Originally meant for Unix and Windows workstations

Now de facto graphics acceleration standard Now managed by the Khronos industry consortium Available everywhere, from supercomputers to cell

phones Alternative: Direct3D provides similar functionality

with a very different API for Microsoft Windows platforms

[Marathon Oil visionarium]

[ES2 tablet]

[MacBook]

CS 354 21

OpenGL as an Evolving Standard

29%

17%

15%

15%4%

2%

2%

2%2%

2%

2%

2%

1% 1%

4%

15%

Multi-vendor

Silicon Graphics

Architectural Review Board

NVIDIA

ATI

Apple

Mesa3D

Sun Microsystems

OpenGL ES

OpenML

IBM

Intense3D

Hewlett Packard

3Dfx

Other

EXT

SGISGISSGIX

ARB

NV

Others Others

44% of extensions are “core” or multi-vendor Lots of vendors have initiated extensions

Extending OpenGL is industry-wide collaboration

ATI

APPLE

MESA

Source: http://www.opengl.org/registry (Dec 2008)

CS 354 22

Forces Driving Improvements in Computer Graphics

Human desire for VisualIntuition and Entertainment

Embarrassing

Parallelism ofGraphics

Increasing

Semiconductor

Density

Particularly thehardware-amenable,

latency tolerantnature of rasterization

Particularlyinteractive video games

ComputerGraphics

Moore’sLaw

CS 354 23

Key Trend in OpenGL Evolution

Fixed-function Programmable

SimpleConfigurability

ComplexConfigurability

Shaders!

High-level languages

CS 354 24

Many Perspectives on OpenGL

Programmer’s view: Application Programming Interface (API) Accelerated access to 3D graphics hardware

Performance and functionality improves with time Graphics Architect’s view:

Detailed functional graphics pipeline Well suited to VLSI hardware design

Software or Hardware System Designer’s view: Stable standard component for building larger system

Platform neutral so fits into other systems well Examples: WebGL, Apple’s compositing desktop

Student’s view: Real-world implementation of graphics concepts

Good pedantic structure for learning graphics Practical skill you can really use

CS 354 25

Programmer’s View:OpenGL API Example

Let’s draw a triangleglShadeModel(GL_SMOOTH); // smooth color interpolationglEnable(GL_DEPTH_TEST); // enable hidden surface removal

glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT); glBegin(GL_TRIANGLES); { // every 3 vertexes makes a triangle glColor4ub(255, 0, 0, 255); // RGBA=(1,0,0,100%) glVertex3f(-0.8, 0.8, 0.3); // XYZ=(-8/10,8/10,3/10)

glColor4ub(0, 255, 0, 255); // RGBA=(0,1,0,100%) glVertex3f( 0.8, 0.8, -0.2); // XYZ=(8/10,8/10,-2/10)

glColor4ub(0, 0, 255, 255); // RGBA=(0,0,1,100%) glVertex3f( 0.0, -0.8, -0.2); // XYZ=(0,-8/10,-2/10)} glEnd();

Pro Tip: use curly braces to “bracket” nested OpenGLusage; no semantic meaning, just highlights grouping

CS 354 26

Initial Logical Coordinate System

Think of drawing into a [-1,+1]3 cube

(-0.8, 0.8) (-0.8, 0.8)

(0, -0.8)

origin at (0,0)

CS 354 27

Visualizing Normalized Device Coordinates

What does this simple triangle look like with the[-1,+1]3 cube’s coordinate system? We call this coordinate system “Normalize Device Coordinate”

or NDC spaceWire frame cube showsboundaries of NDC space

From NDC views, youcan see triangle isn’t“flat” in the Z direction

Two vertices have Z of -0.2—third has Z of 0.3

CS 354 28

Programmer’s View:GLUT API Example

Windowing code#include <GL/glut.h> // includes necessary OpenGL headers

void display() { // << insert code on prior slide here >> glutSwapBuffers();}

void main(int argc, char **argv) { // request double-buffered color window with depth buffer glutInitDisplayMode(GLUT_RGBA | GLUT_DOUBLE | GLUT_DEPTH); glutInit(&argc, argv); glutCreateWindow(“simple triangle”); glutDisplayFunc(display); // function to render window glutMainLoop();}

FYI: GLUT = OpenGL Utility Toolkit

CS 354 29

Reductionism for Rendering Objects and Scenes

Programmers want to render “objects” Say a fire truck or molecule Arranged relative to other objects (a scene) & then viewed

Graphics pipeline approach—used by OpenGL and GPUs Break objects into geometry batches

Batches may be meshes or “patches” Batches reduce to polygonal primitives

Typically triangles But also lines, points, bitmaps, or images

Geometric primitives are specified by vertices So vertices are assembled into primitives

Primitives are rasterized into fragments Fragments are shaded Raster operations take shaded fragments and

update the framebuffer

CS 354 30

Programmer’s View: AlsoProgramming Shaders inside GPU

Multiple programmable domains within the GPU Can be programmed in high-level language

OpenGL Shading Language (GLSL)

GeometryProgram

3D Applicationor Game

OpenGL API

GPUFront End

VertexAssembly

VertexShader

Clipping, Setup,and Rasterization

FragmentShader

Texture Fetch

RasterOperations

Framebuffer Access

Memory Interface

CPU – GPU Boundary

OpenGL 3.3

Attribute Fetch

PrimitiveAssembly

Parameter Buffer Readprogrammable

fixed-function

Legend

CS 354 31

Example Simple GLSL Shaders

Vertex Shader Operates on each vertex

of geometric primitives

Passes through per-vertex color

Transforms the vertex to match fixed-function processing

Fragment Shader Operates on each

fragment (think pixel)

Outputs the fragment’s interpolated color to the framebuffer

void main(void){ gl_FrontColor = gl_Color; gl_Position = ftransform();}

void main(void){ gl_FragColor = gl_Color;}

Shaders are way more interesting than these minimal examples

CS 354 32

Examples of Complex Shaders

CS 354 33

Building Up Shaders

) + () + ((( ) =) =

Diffuse GlossSpecularDecal

Result

CS 354 34

OpenGL’s Design Philosophy

High-performance Assumes hardware

acceleration

Defined by a specification Rather than a de-facto

implementation

Rendering state machine Procedural Not a window system, not

a scene graph

No initial sub-setting Extensible

Data type rich Cross-platform

Window system-independent core

X Window System, Microsoft Windows, OS/2, OS X, etc.

Multi-language bindings C, FORTRAN, etc. Not merely an API, rather

a system

Later had OpenGL ES subset for embedded devices

CS 354 35

OpenGL state machine

Complicated from inception

CS 354 36

Higher-level View of OpenGL

From OpenGL 3.0 specification, unchanged since 1.0

CS 354 37

Evolved OpenGL Data Flow

vertexshading

rasterization& fragment

shading

textureraster

operationsframebuffer

pixelunpack

pixelpack

vertexpuller

clientmemory

pixeltransfer

glReadPixels / glCopyPixels / glCopyTex{Sub}Image

glDrawPixelsglBitmapglCopyPixels

glTex{Sub}ImageglCopyTex{Sub}Image

glDrawElementsglDrawArrays

selection / feedback / transform feedback

glVertex*glColor*glTexCoord*etc.

blendingdepth testingstencil testingaccumulation

storage

operations

CS 354 38

Buffer-centric Evolution Data moves onto GPU, away from CPU

Apps on CPUs just too slow at moving data otherwise

Vertex Array Buffer Object (VaBO)

Transform Feedback Buffer (XBO)

Parameter Buffer Object

(PaBO)

Pixel Unpack Buffer (PuBO)

Pixel Pack Buffer (PpBO)

Uniform BufferObject (UBO)

Texture Buffer Object (TexBO)

Vertex Puller

Vertex Shading

Geometry Shading

FragmentShading

Texturing

Array Element Buffer Object (VeBO)

Pixel Pipeline

vertex data

texel data

pixel data

parameter data

glBegin, glDrawElements, etc.

glDrawPixels, glTexImage2D, etc.

glReadPixels,etc.

Framebuffer

CS 354 39

Extensibility

OpenGL is evolved via extensions Legalistic—expressed as “amending” OpenGL’s

detailed specification Coordinated through Khronos

But vendors don’t need any blessing or permission to make extensions

Allows priorities to be shaped quickly Example: Direct3Dism extensions have greatly

reduced functional incompatibilities keeping Direct3D apps migrate to OpenGL

All the latest GPU features available in OpenGL Example: Tessellation shaders

CS 354 40

framebuffer

updates

Software and Hardware Architectures Incorporating 3D Pipelines

X Window System GeForce 6 Architecture

X server

Graphics Kernel driver

GPU Hardware

OpenGLdriver

Application

OpenGL library

GLUT

X clientlibraries

Operating systemkernel (Linux)

vertexprocessing

rasterization

shading

CS 354 41

Hardware Platforms depending on the OpenGL Architecture

Workstation PCs

Consumer PCs

High-end Visual Computing Solution (VCS)

Embedded Applications

Handheld Devices

Game Consoles

ConventionalPC

OpenGLProducts

Unconventionalnon-PCOpenGL

platforms

CS 354 42

Student’s View of OpenGL

You can learn OpenGL gradually Lots of its can be ignored for now The “classic” API is particularly nice

“Deprecation” has ruined the pedagogical niceness of OpenGL; ignore deprecation

Plenty of documentation and sample code Makes concrete the abstract graphics

pipeline for rasterization

CS 354 43

Next Lecture

3D Viewing How are 3D scenes processed so they can be turned

into an image on the screen? Expect a short quiz on today’s lecture

Easy questions

Assignments Reading

Chapter 1, pages 1-38 Makes sure your CS Unix account is active

Homework next time will be compiling a simple OpenGL example