CISC 3620 Lecture 2 GLSL and Shaders - Michael I...

CISC 3620 Lecture 2GLSL and Shaders

Topics:

Models and ArchitecturesWebGL Overview

Shaders and GLSLColor and attributes

More GLSL

Models and Architectures

Models and Architectures: Objectives

● Learn the basic design of a graphics system● Introduce pipeline architecture● Examine software components for an

interactive graphics system

Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

Image Formation Revisited

● Can we mimic the synthetic camera model to design graphics hardware software?

● Application Programmer Interface (API)– Need only specify

● Objects● Materials● Viewer● Lights

● But how is the API implemented?

Object Specification

● Most APIs support a limited set of primitives including– Points (0D object)– Line segments (1D objects)– Polygons (2D objects)– Some curves and surfaces

● Quadrics● Parametric polynomials

● All are defined through locations in space or vertices

Camera Specification

● Six degrees of freedom for lens position– Position of center of lens– Orientation

● Lens parameters: type, focal length

● Film size: 2D● Orientation of film plane:

another 6D

Lights and Materials

● Types of lights– Point sources vs distributed sources– Spot lights– Near and far sources– Color properties

● Material properties– Absorption: color properties– Scattering

● Diffuse● Specular

https://bensimonds.com/2010/08/27/plausiblematerials/

Physical Approaches● Ray tracing: follow rays of

light from center of projection until they either are absorbed by objects or go off to infinity– Can handle global effects

● Multiple reflections● Translucent objects

– Slow– Must have all objects and lights

available for rendering anything

● Radiosity: Approximation to ray tracing for opaque but diffuse objects – still slow

https://commons.wikimedia.org/w/index.php?curid=1729662

Practical Approach

● Process objects one at a time in the order they are generated by the application– Can consider only local lighting

● Pipeline architecture

● All steps can be implemented in hardware on the graphics card

Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015 https://www.ntu.edu.sg/home/ehchua/programming/opengl/CG_BasicsTheory.html

Vertex Processing

● Much of the work in the pipeline is in converting object representations from one coordinate system to another– Object coordinates– Camera (eye) coordinates– Screen coordinates

● Every change of coordinates is equivalent to a matrix transformation

● Vertex processor also computes vertex colors

Vertext Processing: Projection

● Projection is the process that combines the 3D viewer with the 3D objects to produce the 2D image– Perspective projections: all projectors meet at the center

of projection– Parallel projection: projectors are parallel, center of

projection is replaced by a direction of projection

Vertext Processing:Primitive Assembly

Vertices must be collected into geometric objects before clipping and rasterization can take place

– Line segments– Polygons– Curves and surfaces

Vertex Processing:Clipping

Just as a real camera cannot “see” the whole world, the virtual camera can only see part of the world or object space

– Objects that are not within this volume are said to be clipped out of the scene

Rasterization

● If an object is not clipped out, the appropriate pixels in the frame buffer must be assigned colors

● Rasterizer produces a set of fragments for each object● Fragments are “potential pixels”

– Have a location in frame bufffer– Color and depth attributes

● Vertex attributes are interpolated over objects by the rasterizer

Fragment Processing

● Fragments are processed to determine the color of the corresponding pixel in the frame buffer

● Colors can be determined by texture mapping or interpolation of vertex colors

● Fragments may be blocked by other fragments closer to the camera – Hidden-surface removal

Programming with WebGL:Shader overview

All WebGL programs need two programmed shaders

● Vertex shader – Vertex processor– Processes each vertex independently– Outputs final position

● Fragment shader – Fragment processor– Processes each pixel independently (interpolated by rasterizer)– Outputs final color

https://www.ntu.edu.sg/home/ehchua/programming/opengl/CG_BasicsTheory.html

Vertex Shader Applications

● Moving vertices– Mainly coordinate transformations, projections– But also: Morphing, Wave motion, Fractals

● Lighting– More realistic models– Cartoon shaders

Fragment Shader Applications

Per fragment lighting calculations

per vertex lighting per fragment lighting

Fragment Shader Applications

Texture mapping

smooth shading environment mapping

bump mapping

Writing Shaders

● First programmable shaders were programmed in an assembly-like manner

● OpenGL extensions added functions for vertex and fragment shaders

● Cg (C for graphics) C-like language for programming shaders Works with both OpenGL and DirectX Interface to OpenGL complex

● OpenGL Shading Language (GLSL)

● OpenGL Shading Language● Part of OpenGL 2.0 and up● High level C-like language● New data types

– Matrices– Vectors– Samplers

● As of OpenGL 3.1, application must provide shaders

Simple Vertex Shader

attribute vec4 vPosition;

void main(void)

gl_Position = vPosition;

Execution Model

VertexShader

PrimitiveAssembly

ApplicationProgram

gl.drawArrays Vertex

Vertex dataShader Program

Simple Fragment Shader

precision mediump float;void main(void){ gl_FragColor = vec4(1.0, 0.0, 0.0, 1.0);}

Execution Model

FragmentShader

Application

Frame BufferRasterizerFragment Fragment

Shader Program

Revisit: JSFiddle triangle

27Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015

Programming with WebGL:Shader details

Data Types

● C types: int, float, bool● Vectors: float vec2, vec3, vec4 Also int (ivec) and boolean (bvec)

● Matrices: mat2, mat3, mat4 Stored by columns Standard referencing m[row][column]

● C++ style constructors vec3 a =vec3(1.0, 2.0, 3.0) vec2 b = vec2(a)

No Pointers

● There are no pointers in GLSL● We can use C structs which can be copied back

from functions● Because matrices and vectors are basic types

they can be passed into and output from GLSL functions, e.g.

mat3 func(mat3 a)● variables passed by copying

Qualifiers

● GLSL has many of the same qualifiers such as const as C/C++

● Need others due to the nature of the execution model● Variables can change

– Once per primitive – “uniform”

– Once per vertex – “attribute”

– Once per fragment – “varying”

– At any time in the application● Vertex attributes are interpolated by the rasterizer into

fragment attributes

“uniform” Qualifier

● Variables that are constant for an entire primitive● Can be changed in application and sent to shaders● Cannot be changed in shader● Used to pass information to shader such as the time

or a bounding box of a primitive or transformation matrices

“attribute” Qualifier

● Attribute-qualified variables can change at most once per vertex

● There are a few built in variables such as gl_Position but most have been deprecated

● User defined (in application program) attribute float temperatureattribute vec3 velocityrecent versions of GLSL use in and out qualifiers to get to

and from shaders

“varying” Qualifier

● Variables that are passed from vertex shader to fragment shader

● Automatically interpolated by the rasterizer● With WebGL, GLSL uses the varying qualifier in both shadersvarying vec4 fColor;

● More recent versions of WebGL use out in vertex shader and in in the fragment shaderout vec4 fColor; //vertex shader

in vec4 fColor; // fragment shader

Our Naming Convention

● Attributes passed to vertex shader have names beginning with v (v Position, vColor) in both the application and the shader– Note that these are different entities with the same name

● Varying variables begin with f (fColor) in both shaders– must have same name in both shaders

● Uniform variables are unadorned and can have the same name in application and shaders

Example: Vertex Shader

attribute vec4 vColor;varying vec4 fColor;void main(){ gl_Position = vPosition; fColor = vColor;}

Corresponding Fragment Shader

precision mediump float;

varying vec4 fColor;void main(){ gl_FragColor = fColor;}

Sending Colors from Application

var cBuffer = gl.createBuffer();gl.bindBuffer( gl.ARRAY_BUFFER, cBuffer );gl.bufferData( gl.ARRAY_BUFFER, flatten(colors), gl.STATIC_DRAW );

var vColor = gl.getAttribLocation( program, "vColor" );gl.vertexAttribPointer( vColor, 3, gl.FLOAT, false, 0, 0 );gl.enableVertexAttribArray( vColor );

Sending a Uniform Variable

// in applicationvar color = vec4(1.0, 0.0, 0.0, 1.0);colorLoc = gl.getUniformLocation( program, ”color" ); gl.uniform4fv( colorLoc, color);

// in fragment shader (similar in vertex shader)uniform vec4 color;void main(){ gl_FragColor = color;

In-class exercisePut color of triangle in “uniform”

Operators and Functions

● Standard C functions– Trigonometric– Arithmetic– Normalize, reflect, length

● Overloading of vector and matrix typesmat4 a;vec4 b, c, d;c = b*a; // column vector stored as 1d arrayd = a*b; // row vector stored as 1d array

Swizzling and Selection

● Can refer to array elements using [] or selection (.) operator with x, y, z, w r, g, b, a s, t, p, qa[2], a.b, a.z, a.p are the same

● Swizzling operator lets us index multiple components in ordervec4 a, b;a.yz = vec2(1.0, 2.0, 3.0, 4.0);b = a.yxzw;

Functions named by type

● All versions of OpenGL do no use function overloading

● So that there are multiple versions for a given logical function

● Example: sending values to shadersgl.uniform3f – 3D floats gl.uniform2i – 2D integersgl.uniform3dv – 3D double vectors

● Underlying storage mode is the same

WebGL function format

gl.uniform3f(x,y,z)

belongs to WebGL canvas

function name

x,y,z are variables

gl.uniform3fv(p)

p is an array

dimension

Programming with WebGL:Color and attributes

Color and attributes: Objectives

● WebGL primitives● Adding color● Vertex attributes

WebGLPrimitives

GL_TRIANGLE_STRIPGL_TRIANGLE_STRIP GL_TRIANGLE_FANGL_TRIANGLE_FAN

GL_POINTSGL_POINTS

GL_LINESGL_LINES

GL_LINE_LOOPGL_LINE_LOOP

GL_LINE_STRIPGL_LINE_STRIP

GL_TRIANGLESGL_TRIANGLES

WebGL only uses triangles

● Triangles– Simple: edges cannot cross

– Convex: All points on line segment between two points in a polygon are also in the polygon

– Flat: all vertices are in the same plane● Application program must tessellate a polygon into triangles

(triangulation)● OpenGL 4.1 contains a tessellator but not WebGL

nonsimple polygonnonconvex polygon

Polygon Testing

● Conceptually simple to test for simplicity and convexity – but time consuming

● Earlier versions assumed both – and left testing to the application

● Present version only renders triangles● Need algorithm to triangulate an arbitrary

polygon – Will discuss later in course

Attributes

● Attributes determine the appearance of objects– Color (points, lines, polygons)– Size and width (points, lines)– Stipple pattern (lines, polygons)– Polygon mode

● Display as filled: solid color or stipple pattern● Display edges● Display vertices

● Only a few (gl_PointSize) are supported by WebGL functions

RGB color

● Each color component is stored separately in the frame buffer

● Usually 8 bits per component in buffer● Color values can range from 0.0 (none) to 1.0 (all) using

floats or over the range from 0 to 255 using unsigned bytes

Indexed Color

● Colors are indices into tables of RGB values● Requires less memory indices usually 8 bits not as important now• Memory inexpensive• Need more colors for shading

Smooth Color

● Default is smooth shading– Rasterizer interpolates vertex colors across visible

polygons

● Alternative is flat shading– Color of first vertex

● determines fill color– Handle in shader

Setting Colors

● Colors are ultimately set in the fragment shader but can be determined in either shader or in the application

● Application color: pass to vertex shader as a uniform variable or as a vertex attribute

● Vertex shader color: pass to fragment shader as varying variable

● Fragment color: can alter via shader code

In-class exercisePut color of triangle in “varying”

Programming with WebGL:More GLSL

More GLSL: Objectives

● Coupling shaders to applications– Reading– Compiling– Linking

● Vertex Attributes● Setting up uniform variables● Example applications

Linking Shaders with Application

● Read shaders● Compile shaders● Create a program object● Link everything together● Link variables in application with variables in

shaders– Vertex attributes– Uniform variables

Program Object

● Container for shaders – Can contain multiple shaders– Other GLSL functions

var program = gl.createProgram();

gl.attachShader( program, vertShdr ); gl.attachShader( program, fragShdr ); gl.linkProgram( program );

Reading a Shader

● Shaders are added to the program object and compiled

● Usual method of passing a shader is as a null-terminated string using the function

gl.shaderSource( fragShdr, fragElem.text );● If shader is in HTML file, we can get it into

application by getElementById method● If the shader is in a file, we can write a reader to

convert the file to a string

Adding a Vertex Shader

var vertShdr;var vertElem = document.getElementById( vertexShaderId );

vertShdr = gl.createShader( gl.VERTEX_SHADER );

gl.shaderSource( vertShdr, vertElem.text );gl.compileShader( vertShdr );

// after program object createdgl.attachShader( program, vertShdr );

Shader Reader

● Following code may be a security issue with some browsers if you try to run it locally– Cross Origin Request

function getShader(gl, shaderName, type) { var shader = gl.createShader(type); shaderScript = loadFileAJAX(shaderName); if (!shaderScript) { alert("Could not find shader source: "+shaderName); }}

Precision Declaration

● In GLSL for WebGL we must specify desired precision in fragment shaders– artifact inherited from OpenGL ES– ES must run on very simple embedded devices that

may not support 32-bit floating point– All implementations must support mediump– No default for float in fragment shader

● Can use preprocessor directives (#ifdef) to check if highp supported and, if not, default to mediump

Pass Through Fragment Shader

#ifdef GL_FRAGMENT_SHADER_PRECISION_HIGH precision highp float;#else precision mediump float;#endif

varying vec4 fcolor;void main(void){ gl_FragColor = fcolor;}

Summary

● Models and Architectures● WebGL Overview● Shaders and GLSL● Color and attributes● More GLSL

CISC 3620 Lecture 2 GLSL and Shaders - Michael I...

Documents

CISC 3620 Lecture 12: Rendering pipeline revisitedm.mr-pc.org/t/cisc3620/2019sp/lecture12.pdf · 2019. 5. 1. · 4 Overview At end of the geometric pipeline, vertices have been assembled

lecture09-switched-Ethernet -network-layer › courses › cs4450 › 2019sp › lecture09...Spanning Tree Protocol ++ (incorporating failures) Example: Suppose link 2-4 fails •

CISC 7610 Lecture 9 Image retrieval - Michael I Mandelm.mr-pc.org/t/cisc7610/2016sp/lecture09.pdf · Color histogram example Draw a 3D color histogram for the following image Draw

CIS 381: Social & Ethical Issues of Computingdkoop/cis381-2019sp/lectures/lecture18.pdfCIS 381: Social & Ethical Issues of Computing Security Dr. David Koop D. Koop, CIS 381, Spring

CIS 381: Social & Ethical Issues of Computing › ~dkoop › cis381-2019sp › lectures › lecture... · 2019-02-15 · CIS 381: Social & Ethical Issues of Computing Intellectual

January 30, 2019 11:11 Differential Forms – 9in x 6in b3414-main …math.mit.edu/classes/18.952/2019SP/18.952_book_2019.pdf · 2019-02-08 · January 30, 2019 11:11 Differential

Lecture 20: Synchronizationebirrell/classes/cs105/2019sp/lectures/20-S… · Why Concurrent Programs? 1.06 0.54 0.28 0.29 0.3 0 0.2 0.4 0.6 0.8 1 1.2 1 2 4 8 16 Elapsed time (s) Threads

UC Berkeley’s CS10 Spring 2019 Final Exam …cs10.org/fa19/exams/final/2019Sp/answers.pdfA software engineer at YouTube who advised you not to avoid challenge. A backend engineer

CIS 381: Social & Ethical Issues of Computingdkoop/cis381-2019sp/lectures/lecture27.pdf · Alternative List of Fundamental Principles • Be impartial. • Disclose information that

:لϰلأΐ سردϢΐGoogle+SEO+2019SP+type@asset+… · :دϴدج حϡطصم ϱϡع نلآا فرّعتنس ϵϖ ثΦبϢΐ ΡئΑتن ΗΦϗص ϵϮϰ ) Search Engine Results Page

Data Visualization (DSC 530/CIS 568)dkoop/dsc530-2019sp/lectures/lecture02.pdfBack To Black Amy Winehouse 2007 Good Girl Gone Bad Rihanna 2007 Marvin Gaye '50' Marvin Gaye 2014 X Chris

Lecture 1: Introduction & DSP - m.mr-pc.orgm.mr-pc.org/t/csc83060/2019sp/lecture01.pdf · Aliasing For example, for cos(!n);!= 2ˇr ! 0 all (integer) r values appear to the same after

CIS 381: Social & Ethical Issues of Computingdkoop/cis381-2019sp/lectures/lecture02.pdf · CIS 381: Social & Ethical Issues of Computing Technologies as Catalysts for Change Dr. David

CISC 1600 Lecture 1.2 Introduction - Michael I Mandelm.mr-pc.org/t/cisc1600/2016sp/lecture_1_2.pdf · Types of Markup (HTML does all three) Presentational markup: Used by traditional

CISC 7610 Lecture 6 Midterm review - Michael I Mandelm.mr-pc.org/t/cisc7610/2018fa/midtermReview.pdf · CISC 7610 Lecture 1 Introduction to multimedia databases Main question –

UC Berkeley’s CS10 Spring 2019 Final Exam: Prof. Dan Garciacs10.org/fa19/exams/final/2019Sp/exam.pdf · SID: _____ (The Bool block on the right is used for Questions 7 & 8; 2,3

Introduction to Data Management CSE 414 · Introduction to Data Management CSE 414 Spark CSE 414 -2019sp 2. Class Overview •Unit 1: Intro ... •Spark supports interfaces in Java,

Troubleshooting Deep Neural Networksm.mr-pc.org/t/csc83060/2019sp/troubleshooting-subset.pdf · 2019-02-22 · Learning curve from the paper He, Kaiming, et al. "Deep residual learning

CISC 3620 Lecture 7 Lighting and shadingm.mr-pc.org/t/cisc3620/2019sp/lecture07.pdf · Light Sources In the Phong Model, we add the results from each light source Each light source

CISC 7610 Lecture 9 Image retrieval - Michael I Mandelm.mr-pc.org/t/cisc7610/2017fa/lecture09.pdf · CISC 7610 Lecture 9 Image retrieval Topics: How hard is computer vision? Image