Real-Time Shading Using Programmable Graphics Hardware-Shader Programming

(1)

Real-Time Shading Using Programmable Graphics Hardware

Shader Programming

Wan-Chun Ma

(2)

Today’s Schedule

 Cg Programming

 Cg Runtime Setup in OpenGL

 Multi-texturing in OpenGL

(3)

(4)

Data Types

 float = 32-bit IEEE floating point

 half = 16-bit IEEE-like floating po

int

 fixed = 12-bit fixed [-2,2) clamping (OpenGL only)

 bool = boolean

(5)

Declarations of Array /

Vector / Matrix

 Declare vectors (up to length 4) and matrice

s (up to size 4x4) using built-in data types :

 _float4_mycolor;

 _float3x3_mymatrix;

 Declare more general arrays exactly as in C:

 _float_{lightpower[4];}

 _{But, arrays are first-class types, not pointers}  _{Implementations may subset array}

(6)

Arithmetic of Vector /

Matrix

 Component-wise + - * / for vectors

 Dot product

 _dot(v1,v2); _{// returns a scalar}

 Matrix multiplications:

 _assuming_float4x4_{M and}_float4_v

 _{matrix-vector: mul(M, v);}_{// returns a vector}  _{vector-matrix: mul(v, M);}_{// returns a vector}  _{matrix-matrix: mul(M, N);}_{// returns a matrix}

(7)

New Vector Operators

 Swizzle operator extracts elements fr

om vector

_{a = b.xxyy;}

 Vector constructor builds vector

(8)

GPU Pipeline Worklow

Application Vertex Program

Fragment

Program Framebuffer Connector Connector Connector

Cg Vertex Shader

Cg Fragment Shader

(9)

What is a Connector?

 Describes shader inputs and outputs

 Defines an interface that allows mixi

ng-and-matching of programs

 It’s a struct, but with optional ext ra information

(10)

Connect Application to

Vertex

 Connectors define an interface that

allows mixing-and-matching of programs struct a2v { float4 pos; float4 color; float2 uv;

floatw1; // skinning weight #1

floatw2; // skinning weight #2

};

(11)

Connect Vertex to

Fragment

 Similar to previous a2f one

struct v2f {

float4 pos : POSITION; // must have POSITION

float4 color; float2 uv; };

(12)

Connect Fragment to

Screen

 Fragment shader only output color to

framebuffer (screen)

struct f2s {

float4 color : COLOR0; // must have POSITION

float4 color; float2 uv; };

(13)

Specify Connector

Registers

 These semantics match input/output

variable with designated registers

struct v2f {

float4 pos : POSITION; float4 color : COLOR0; float4 normal : NORMAL; float2 uv : TEXCOORD0; float4 data1 : TEXCOORD1; float4 data2 : TEXCOORD2;

(14)

Define a Vertex Program

v2f main(a2v IN, // input connector

uniform float4x4 m1,

uniform float4x4 m2) {

v2f OUT; // output connector // some computation

OUT.pos = IN.w1*(mul(m1,vin.pos)) + IN.w2*(mul(m2,vin.pos)); OUT.color = IN.color;

OUT.uv = IN.uv;

return OUT; // output to next level

}

Note that “uniform” variables don’t come in via a connector

(15)

Define a Fragment

Program

f2s main(v2f IN, // input connector

uniform sampler2D tex) {

f2s OUT; // output connector // some computation

OUT.color = IN.color*tex2D(tex, IN.uv);

return OUT; // output to next level

(16)

Cg Runtime Setup in

OpenGL

(17)

Core Cg Runtime

 Does not make any 3D API call  Allows you to:

 Create a context: _{cgCreateContext}₍₎

 Compile a program for a given profile (vp30, fp3

0): cgCreateProgram(), cgGetProgramString(), et c...

 _{Manage program parameters:}

 Iterate through the parameters: cgGetFirstParameter(), c gGetNextParameter(), etc...

 Get parameter information: type, semantic, register, ...

 Handle errors: _cgGetError_(),_{cgSetErrorCallback}₍₎

(18)

OpenGL Cg Runtime

 Allows you to:

Load a shader: _{cgGLLoadProgram}₍₎

Enable a profile: _{cgGLEnableProfile}₍₎

Tell OpenGL to render with it: _cgGLBindP

rogram()

Set parameter values:

 cgGLSetParameter{1234}{fd}{v}()  cgGLSetParameterArray{1234}{fd}()  cgGLSetTextureParameter()

(19)

Coffee Break

 Next section: Multi-texturing in Ope nG_L

(20)

Multi-texturing in

OpenGL

(21)

What is Multi-texturing

 As its name, multi-texturing means th

at you are displaying more than one t extures at the same time

 Usually a object is mapped with

several textures

Textures: diffuse, specular, glossy, the

other shading coefficient

(22)

Multi-texturing (Old)

 Since the hardware doesn’t’ support

multi-texturing, we need to use

“blending” to combine two (or more) textures

glTexCoordPointerEXT(2,GL_FLOAT,0,m, uv0);

glBindTexture(GL_TEXTURE_2D,tex0);

glDrawElements(GL_TRIANGLES,n,GL_TYPE_INDEXE_ARRAY,ind);

glEnable(GL_BLEND);

glBlendFunc(GL_SRC_ALPHA,GL_ONE_MINUS_SRC_ALPHA);

glTexCoordPointerEXT(2,GL_FLOAT,0,m, uv1);

glBindTexture(GL_TEXTURE_2D, tex1 );

glDrawElements(GL_TRIANGLES,n,GL_TYPE_INDEXE_ARRAY,ind);

(23)

Multi-texturing (

_New)

 Use OpenGL multi-texturing

extension

 No need for blending (we actually do

blending in fragment shader)if(!GLEW_ARB_multitexture) return false;

// texture 0 on arb texture unit 0

glActiveTextureARB(GL_TEXTURE0_ARB);

// rendering

glEnable(GL_TEXTURE_2D);

(24)

Multi-texturing (

_New)

 Use Cg runtime to enable

multi-texturing

if(!GLEW_ARB_multitexture) return false;

cgGLSetTextureParameter(cg_para0,tex0);

cgGLEnableTextureParameter(cg_para0);

cgGLSetTextureParameter(cg_para1,tex0); cgGLEnableTextureParameter(cg_para1); glActiveTextureARB(GL_TEXTURE1_ARB); // rendering glEnable(GL_TEXTURE_2D); draw_object();

(25)

Multi-texturing (

_New)

 Declare multiple texture coordinates

void draw_object(){ glBegin(GL_QUADS); glMultiTexCoord2fARB(GL_TEXTURE0_ARB, 0.0, 1.0); glMultiTexCoord2fARB(GL_TEXTURE1_ARB, 0.0, 1.0); glVertex3f(0.0, 0.0, 0.0); glMultiTexCoord2fARB(GL_TEXTURE0_ARB, 0.0, 0.0); glMultiTexCoord2fARB(GL_TEXTURE1_ARB, 0.0, 0.0); glVertex3f(0.0, 1.0, 0.0); glMultiTexCoord2fARB(GL_TEXTURE0_ARB, 1.0, 0.0); glMultiTexCoord2fARB(GL_TEXTURE1_ARB, 1.0, 0.0); glVertex3f(1.0, 1.0, 0.0); glMultiTexCoord2fARB(GL_TEXTURE0_ARB, 1.0, 1.0); glMultiTexCoord2fARB(GL_TEXTURE1_ARB, 1.0, 1.0); glVertex3f(1.0, 0.0, 0.0); glEnd();

(26)

(27)

What is Bump Mapping

 The idea is to perturb the surface nor

mal of a given surface_{, and use the}

normal to calculate the shading [Blinn1978]

 Bump mapping simulates the bumps or w

rinkles in a surface without the need for geometric modifications to the mo del

(28)

What is Bump Mapping

(29)

What is Bump Mapping

 Bump mapping has widely used in

games

(30)

Texture-Space

_Lighting

 One convenient space for per-pixel lighting

operations is called texture_-space

 _{It represents a}_{local coordinate system}_defined

at each vertex of a set of geometry

 _{You can think of the Z axis of texture}-space to

be roughly parallel to the vertex normal

 _{The X and Y axes are perpendicular to the Z ax}

is and can be arbitrarily oriented around the Z axis

(31)

Why Texture-Space

_?

 Texture-space gives us a way to redef ine the lighting coordinate system on a per-vertex basis

(32)

Why Texture-Space?

 If we used model-space or world-space bump

maps, we would have to regenerate the entir e bump map every time the object morphed or rotated in any way, because the bump map no rmals will no longer be pointing in the cor rect direction in model or world space

 We would have to recompute the bump maps fo

r each instance of each changing object eac h frame_{, and it seems not a efficient way}

(33)

Why Texture-Space?

 Texture-space is a surface-local basis in which we define our normal maps

 Therefore, we must rotate the

light_{ing and viewing directions into}

this space as well_{. We must move the}

light vectors into texture_{-space befor} e performing the per-pixel dot produc t

(34)

Generate Texture-Space

 Mainly there are two ways to generate

texture-space:

_{Have the artist}to draw a normal map

and a tangent map for the corresponding model

_{When loading in a model, create the}

per-vertex tangent-space

 These will store the axes of the texture-spac

e basis

 Generate the texture-space vectors from the v

ertex positions and bump map texture coordina tes

(35)

Generate Texture-Space

 For each triangle in the model:

_{Use the x,y,z position and the s,t bump}

map texture coordinates

Create plane equations of the form:

 Solve for the texture gradients dsdx,

dsdy, dsdz, etc.

Ax + Bs + Ct + D = 0 Ay + Bs + Ct + D = 0 Az + Bs + Ct + D = 0

(36)

Generate Texture-Space

 Now treat the dsdx, dsdy, and dsdz as a 3D

vector representing the S axis (dsdx, dsdy, dsdz₎

 Usually we call it tangnet G

 Use a cross product NxS to generate T

 Usually we call it binormal B

 Notice that the up axis of texture-space is clos

e to parallel with the normal N of the vertex

 Tangent and binormal is exchangeable to each

(37)

Transform into Texture-Space

 These 3 a xes together make up a 3x3 r otation matrix

 Putting an model-space XYZ vector thr

ough this 3x3 matrix produces a vecto r expressed in texture-Space

Tx Bx Nx

Ty By Ny

(38)

Per-Vertex Texture-Space

 Now we have what we need to move a lighting

and viewing directions into a local space d efined at each vertex (or fragment) via the texture-space basis matrix

 We can do this on the GPU with:

 _{Vertex shader: calculate the rotated lighting a}

nd viewing directions first, than transfer them to fragment shader (interpolated result)

 _{Fragment shader: use interpolated position for}

(39)

(40)

Vertex Shader

struct v2f {

float4 P2D : POSITION; // projected 2D position

float4 C : COLOR0; // color

float4 T : TEXCOORD0; // texture coord

float3 P3D : TEXCOORD1; // vertex 3D position

float3 N : TEXCOORD2; // normal

float3 G : TEXCOORD3; // tangent

float3 B : TEXCOORD4; // binormal

};

(41)

Vertex Shader

 Main (application-to-vertex) arguments v2f main( float4 C : COLOR, float4 P : POSITION, float4 N : NORMAL, float4 T : TEXCOORD0,

uniform float4x4 ModelViewProj,

uniform float4x4 ModelView,

(42)

Vertex Shader

{

v2f OUT;

OUT.P2D = mul(ModelViewProj, P); OUT.P3D = P.xyz;

OUT.T = T;

OUT.N= normalize(N.xyz); // normal

OUT.G= normalize(2.0*C.xyz - 1.0); // tangent

OUT.B= normalize(cross(OUT.G, OUT.N)); // binormal

return OUT; }

(43)

Fragment Shader

 Fragment-to-screen data structure

struct f2s {

float4 C : COLOR0; };

(44)

Fragment Shader

 Main (application-to-vertex) arguments

f2s main( v2f IN,

uniform sampler2D tex00, // texture 00 : diffuse texture

uniform sampler2D tex01, // texture 01 : specular texture

uniform sampler2D tex02, // texture 02 : eye texture

uniform sampler2D tex03, // texture 03 : height bump

uniform sampler2D tex04, // texture 04 : normal bump

uniform float3 L, // world-space lighting direction

(45)

Fragment Shader

 Main body

{

f2s OUT; OUT.rgb = 0;

L = normalize(float3(dot(L,IN.G), dot(L,IN.B), dot(L,IN.N))); V = normalize(float3(dot(V,IN.G), dot(V,IN.B), dot(V,IN.N)));

float3 H = normalize(L+V);

float3 N = tex2D(tex02, IN.T.xy)*2.0 – 1.0;

float diff = dot(N, L); if(diff > 0){

float spec = pow(dot(N, H), tex2D(tex01, IN.T.xy)*128)/4; OUT.C.rgb = diff*tex2D(tex00, IN.T.xy) + spec;

}

return OUT; }

(46)

(47)

Calculate Normal From

Height

 Normal = normalize(dx, dy, 1)

_{dx = Hd – Hc} _{dy = Hb – Ha}

_{Usually we’ll add a “bumpy factor”, mu}

ltiply the bumpy factor to dx and dy

a

c x d

(48)

Fragment Shader (Original)

 Main body

{

f2s OUT; OUT.rgb = 0;

L = normalize(float3(dot(L,IN.G), dot(L,IN.B), dot(L,IN.N))); V = normalize(float3(dot(V,IN.G), dot(V,IN.B), dot(V,IN.N)));

float3 H = normalize(L+V);

float3 N = tex2D(tex02, IN.T.xy)*2.0 – 1.0;

float diff = dot(N, L); if(diff > 0){

float spec = pow(dot(N, H), tex2D(tex01, IN.T.xy)*128)/4; OUT.C.rgb = diff*tex2D(tex00, IN.T.xy) + spec;

}

return OUT;

(49)

Fragment Shader (Height)

 Normal from height

{ ... skip

float tex_w = 1200; float tex_h = 600; float bump = 1.25;

float2 uv = IN.T.xy; uv.x -= 1/tex_w;

float s_left = tex2D(tex03, uv).x; uv.x += 2/tex_w;

float s_right = tex2D(tex03, uv).x; uv.x -= 1/tex_w; uv.y -= 1/tex_h;

float s_down = tex2D(tex03, uv).x; uv.y += 2/tex_h;

float s_up = tex2D(tex03, uv).x;

float3 N = normalize(float3((s_right-s_left)*bump, (s_down, s_up)*bump, 1));

(50)

Try This...

 Anisotropic reflectance model

(51)