128
1/59 Presentation Introduction Formalizing the Problem Sampling VPLs: Metropolis Instant Radiosity Accumulating VPL contributions Coherent Metropolis Light Transport Conclusion Interactive Light Transport with Virtual Point Lights Benjamin Segovia 1,2 1 ENTPE: Ecole Nationale des Travaux Publics de l’Etat 2 LIRIS: Laboratoire d’InfoRmatique en Image et Syst` emes d’information Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Interactive Light Transport with Virtual Point Lights

  • Upload
    others

  • View
    4

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Interactive Light Transport with Virtual Point Lights

1/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Interactive Light Transport with Virtual PointLights

Benjamin Segovia1,2

1ENTPE: Ecole Nationale des Travaux Publics de l’Etat2LIRIS: Laboratoire d’InfoRmatique en Image et Systemes d’information

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 2: Interactive Light Transport with Virtual Point Lights

2/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Presentation

1 Introduction

2 Formalizing the Problem

3 Sampling VPLs: Metropolis Instant Radiosity

4 Accumulating VPL contributions

5 Coherent Metropolis Light Transport

6 Conclusion

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 3: Interactive Light Transport with Virtual Point Lights

3/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Summary

1 Introduction

2 Formalizing the Problem

3 Sampling VPLs: Metropolis Instant Radiosity

4 Accumulating VPL contributions

5 Coherent Metropolis Light Transport

6 Conclusion

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 4: Interactive Light Transport with Virtual Point Lights

4/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Why a Ph.D. in computer graphics?

Movie / FX industry

Fast and robust renderingalgorithms;

Not necessary real-timebut speed is fundamental.

Figure: Poseidon, 2006, renderedwith Mental Ray

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 5: Interactive Light Transport with Virtual Point Lights

4/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Why a Ph.D. in computer graphics?

Figure: Thee Dragon Room,rendered with yaCORT

Lighting design

Physically-based renderingtools;

Not necessary real-time.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 6: Interactive Light Transport with Virtual Point Lights

4/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Why a Ph.D. in computer graphics?

Video Games

The most realisticrendering with strictconstraints;

Real time (more than 30frames per second).

Figure: A Quake 3 scene,rendered with Qrender

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 7: Interactive Light Transport with Virtual Point Lights

5/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

What Does this Ph.D. Contain?

Common approach in science

1 Identify the physical problem→ Simulating light transport;

2 Propose an appropriate mathematical formalism→ The related physical quantities and the light transportequations;

3 Design algorithms to solve these equations

→ Computer science

Numerical schemesAlgorithms, codes . . .

The contribution of this Ph.D. thesis is mostly contained by the thirdpoint

→ Numerical schemes to solve the light transport equationsBenjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 8: Interactive Light Transport with Virtual Point Lights

6/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Overview of the Presentation

First, introduction of necessary concepts

Physics: Physics of light transport → quantities andequations;

Mathematics: Roots of Monte-Carlo and introduction of theappropriate formalism;

Computer Graphics: Most common algorithms used tocompute virtual pictures.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 9: Interactive Light Transport with Virtual Point Lights

6/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Overview of the Presentation

Then, presentation of the contributions

Two classes of contributions:

Coding Techniques: Once the set of VPLs has beencomputed, how can we accumulate their contributions ?→ we present two techniques using graphics hardware;

Sampling Techniques: How can we generate efficient sets ofVPLs?

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 10: Interactive Light Transport with Virtual Point Lights

7/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Summary

1 Introduction

2 Formalizing the Problem

3 Sampling VPLs: Metropolis Instant Radiosity

4 Accumulating VPL contributions

5 Coherent Metropolis Light Transport

6 Conclusion

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 11: Interactive Light Transport with Virtual Point Lights

8/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

The Rendering Equation - Three Point Form [Vea97]

Formalizes the behavior of materials and surfaces

L(x′→x′′) = Le(x′→x′′)+

M

L(x→x′)fs(x→x′→x′′)G (x↔x′)dA(x)

where:

L is the equilibrium outgoing radiance function;

Le(x′→x′′) is the emitted radiance leaving x′ in the direction of x′′;

fs(x→x′→x′′) is the BSDF of the material;

M is the union of all the surfaces of the scene;

A is the Lebesgue (i.e. uniform) area measure on M;

G(x↔x′) is the geometric term between x and x′.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 12: Interactive Light Transport with Virtual Point Lights

9/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

The Measurement Equation

Response of a given captor / sensor

Ij =

M×M

W(j)e (x→x′)L(x→x′) G (x↔x′) dA(x) dA(x′)

where W (j)(x, ω) is the responsivity of sensor j .

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 13: Interactive Light Transport with Virtual Point Lights

10/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Issues with Light Transport

High dimensional problem: light may bounce many times . . .

High frequency problem: many discontinuities (shadows forexample).

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 14: Interactive Light Transport with Virtual Point Lights

10/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Issues with Light Transport

High dimensional problem: light may bounce many times . . .

High frequency problem: many discontinuities (shadows forexample).

Integrand has very poor properties →

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 15: Interactive Light Transport with Virtual Point Lights

10/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Issues with Light Transport

High dimensional problem: light may bounce many times . . .

High frequency problem: many discontinuities (shadows forexample).

Integrand has very poor properties →

Use Monte-Carlo integration!

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 16: Interactive Light Transport with Virtual Point Lights

11/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Monte-Carlo Integration is Basically . . .

We want to integrate I =∫

Ω f (x)dµ(x)where

Ω is a given space;

µ is an associated measure;

f is a measurable function on (Ω, µ).

With sufficient properties ...

N random variables (Xn)n∈[1...N] with density p, then:

limN→∞

IN = limN→∞

1

N

N∑

n=1

f (Xn)

p(Xn)= I almost surely

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 17: Interactive Light Transport with Virtual Point Lights

12/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

The Path Integral Formulation

Make the light transport problem an integration one

Inject the rendering eq. into the measurement eq. and expand it:

Ij =∑

k=1

Mk+1

[

Le(xk→xk−1)G (x0↔x1) W(j)e (x1→x0)

(∏k−1

i=1 fs(xi+1→xi→xi−1)G (xi↔xi+1))dA(x0) ... dA(xk)]

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 18: Interactive Light Transport with Virtual Point Lights

12/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

The Path Integral Formulation

Make the light transport problem an integration one

Inject the rendering eq. into the measurement eq. and expand it:

Ij =∑

k=1

Mk+1

[

Le(xk→xk−1)G (x0↔x1) W(j)e (x1→x0)

(∏k−1

i=1 fs(xi+1→xi→xi−1)G (xi↔xi+1))dA(x0) ... dA(xk)]

The path integral formulation

Ij =

Ωf (j)(x)dµ(x)

Ω is the set of all finite length paths, µ its natural measure andf (j) obtained with the expansion.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 19: Interactive Light Transport with Virtual Point Lights

13/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

A Short Pause Before the Remainder!

OK, a short summary!

Monte-Carlo rendering is:

Sample a path x with density p(x);

Evaluate f (j)(x)p(x) ;

Accumulate.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 20: Interactive Light Transport with Virtual Point Lights

13/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

A Short Pause Before the Remainder!

OK, a short summary!

Monte-Carlo rendering is:

Sample a path x with density p(x);

Evaluate f (j)(x)p(x) ;

Accumulate.

Most Monte-Carlo rendering methods → propose new ways togenerate paths x .

Basically, this Ph.D. presents new Monte-Carlo renderingtechniques.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 21: Interactive Light Transport with Virtual Point Lights

14/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Short Overview of Path Integration

Core algorithm: path tracing [Kaj86]

We generate a light path backward from the camera for eachcamera sensor (i.e. for each pixel)

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 22: Interactive Light Transport with Virtual Point Lights

14/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Short Overview of Path Integration

Core algorithm: path tracing [Kaj86]

We generate a light path backward from the camera for eachcamera sensor (i.e. for each pixel)

Many, many similar techniques

Bidirectional path tracing [VG94, LW93];

Light tracing [DLW93].

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 23: Interactive Light Transport with Virtual Point Lights

15/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Path Tracing

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 24: Interactive Light Transport with Virtual Point Lights

15/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Path Tracing

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 25: Interactive Light Transport with Virtual Point Lights

15/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Path Tracing

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 26: Interactive Light Transport with Virtual Point Lights

15/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Path Tracing

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 27: Interactive Light Transport with Virtual Point Lights

15/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Path Tracing

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 28: Interactive Light Transport with Virtual Point Lights

16/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Problems with these ”Pure” Path Tracing methods

No computation coherency

Per-pixel computations are independent;

No factorization.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 29: Interactive Light Transport with Virtual Point Lights

16/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Problems with these ”Pure” Path Tracing methods

No computation coherency

Per-pixel computations are independent;

No factorization.

We must design efficient techniques

Most of them propose to use biased estimators:

Photon Maps [Jen01, Jen96, Jen97];

Radiance / Irradiance Caches [WRC88, War94, K05];

And . . .

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 30: Interactive Light Transport with Virtual Point Lights

16/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Problems with these ”Pure” Path Tracing methods

No computation coherency

Per-pixel computations are independent;

No factorization.

Instant Radiosity [Kel97] → Replaces complete paths by”Virtual Point Lights”

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 31: Interactive Light Transport with Virtual Point Lights

17/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Instant Radiosity

Principles

Splits each path x = x0, x1, . . . , xn into three parts:

xc = x0, x1 is the camera sub-path;

xv is a geometric Virtual Point Light (VPL);

xs is the remainder of the path connected to a light source.

x0

xc

xv

xs

x1

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 32: Interactive Light Transport with Virtual Point Lights

17/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Instant Radiosity

Principles

For all sensors (i.e. pixels), use the same (xv , xs) light paths;

Two-pass algorithm:

Propagation of light paths from the light sources (sampling);Accumulation of VPL contributions (gathering).

Do not forget: a VPL is a light path, not only a point!

x0

xc

xv

xs

x1

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 33: Interactive Light Transport with Virtual Point Lights

18/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

The Two Steps of Instant Radiosity

Particle Propagation

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 34: Interactive Light Transport with Virtual Point Lights

18/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

The Two Steps of Instant Radiosity

Particle Propagation

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 35: Interactive Light Transport with Virtual Point Lights

18/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

The Two Steps of Instant Radiosity

Particle Propagation

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 36: Interactive Light Transport with Virtual Point Lights

18/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

The Two Steps of Instant Radiosity

Particle Propagation

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 37: Interactive Light Transport with Virtual Point Lights

18/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

The Two Steps of Instant Radiosity

Particle Propagation

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 38: Interactive Light Transport with Virtual Point Lights

18/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

The Two Steps of Instant Radiosity

Particle Propagation

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 39: Interactive Light Transport with Virtual Point Lights

18/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

The Two Steps of Instant Radiosity

Incoming Radiance Field Integration

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 40: Interactive Light Transport with Virtual Point Lights

18/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

The Two Steps of Instant Radiosity

Incoming Radiance Field Integration

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 41: Interactive Light Transport with Virtual Point Lights

18/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

The Two Steps of Instant Radiosity

Incoming Radiance Field Integration

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 42: Interactive Light Transport with Virtual Point Lights

18/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

The Two Steps of Instant Radiosity

Incoming Radiance Field Integration

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 43: Interactive Light Transport with Virtual Point Lights

18/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

The Two Steps of Instant Radiosity

Incoming Radiance Field Integration

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 44: Interactive Light Transport with Virtual Point Lights

19/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Advantages and Drawbacks of Instant Radiosity

Advantages

Simple → the incoming radiance field is replaced by a set ofpoints;

Fast → can be easily implemented with coherent ray tracingor rasterization.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 45: Interactive Light Transport with Virtual Point Lights

19/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Advantages and Drawbacks of Instant Radiosity

Advantages

Simple → the incoming radiance field is replaced by a set ofpoints;

Fast → can be easily implemented with coherent ray tracingor rasterization.

Drawbacks

Variance problems → how must the VPLs be located?

Does not handle all lighting phenomena → caustics . . .

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 46: Interactive Light Transport with Virtual Point Lights

20/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Summary

1 Introduction

2 Formalizing the Problem

3 Sampling VPLs: Metropolis Instant Radiosity

4 Accumulating VPL contributions

5 Coherent Metropolis Light Transport

6 Conclusion

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 47: Interactive Light Transport with Virtual Point Lights

21/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Goal of Metropolis Instant Radiosity (MIR)

Properties of VPLs is fundamental

We must find VPLs which illuminate parts of the scene seen by thecamera!

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 48: Interactive Light Transport with Virtual Point Lights

21/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Goal of Metropolis Instant Radiosity (MIR)

Solution: Combine the robutness of Metropolis LightTransport and the efficiency of Instant Radiosity

Principle of MIR

Use the path sequence of Metropolis Light Transport tosample VPLs (”MLT part”);

For each path, store the second point as a VPL;

Accumulate all VPL contributions (”IR part”).

With this sampler, all VPLs will bring the same amount ofpower to the camera

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 49: Interactive Light Transport with Virtual Point Lights

22/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Metropolis Light Transport [VG97]

Principle (a short version)

Consider the whole camera integrand f (c);

Sample N paths with a density proportional to f (c);

Count for each pixel j , the number Nj of paths which get intoit;

With Nj , N, and∫

Ω f (c)(x)dµ(x), compute the per-pixel

histogram of f (c);

We have the intensity of each pixel!

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 50: Interactive Light Transport with Virtual Point Lights

22/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Metropolis Light Transport [VG97]

Numerical schemes behind it

Compute∫

Ω f (c)(x)dµ(x)→ Use a standard bidirectional path tracer;

Sample N paths with a density proportional to f (c)

→ Use a Metropolis-Hastings algorithm.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 51: Interactive Light Transport with Virtual Point Lights

22/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Metropolis Light Transport [VG97]

Metropolis-Hastings

Goal: given function f , sequentially sample random variablesXi with a density proportional to f ;

Xi+1 and Xi are correlated by a mutation.

The density of Xi is not exactly f , but with good properties(”ergodicity”), we can use all Xi :as if their densities were fas if they were independent

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 52: Interactive Light Transport with Virtual Point Lights

23/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

MLT - Initial Path

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 53: Interactive Light Transport with Virtual Point Lights

24/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

MLT - Candidate

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 54: Interactive Light Transport with Virtual Point Lights

25/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

MLT - Candidate accepted → Count its contribution

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 55: Interactive Light Transport with Virtual Point Lights

26/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

MIR - Compute the power received by the camera

With a bidirectional path tracer (or any other technique)compute the power Pc =

Ω f (c)(x)dµ(x) received by the camera.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 56: Interactive Light Transport with Virtual Point Lights

27/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

MIR - Sample ”path VPLs”

The core idea of the method

MLT algorithm provides complete paths x0, x1, xv, x s;

Remove points x0 and x1 and consider (xv, x s) as a ”pathVPL”.

x0

xc

xv

xs

x1

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 57: Interactive Light Transport with Virtual Point Lights

27/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

MIR - Sample ”path VPLs”

The core idea of the method

MLT algorithm provides complete paths x0, x1, xv, x s;

Remove points x0 and x1 and consider (xv, x s) as a ”pathVPL”.

x0

xc

xv

xs

x1

Path VPL

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 58: Interactive Light Transport with Virtual Point Lights

27/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

MIR - Sample ”path VPLs”

The core idea of the method

MLT algorithm provides complete paths x0, x1, xv, x s;

Remove points x0 and x1 and consider (xv, x s) as a ”pathVPL”.

We do not know the outgoing radiance functions of the VPLs;

But, we can prove that these VPLs bring the sameamount of power to the camera

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 59: Interactive Light Transport with Virtual Point Lights

28/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

MIR - Cluster ”path VPLs” into ”geometric VPLs”

Cluster path VPLs with the same VPL location into one geometric VPL

When applying mutations, VPL locations may remain unchanged:

The candidate is rejected and the path is duplicated;

Only the sub-path xc = x0, x1 is mutated;

Only the sub-path x s is mutated.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 60: Interactive Light Transport with Virtual Point Lights

28/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

MIR - Cluster ”path VPLs” into ”geometric VPLs”

Cluster path VPLs with the same VPL location into one geometric VPL

When applying mutations, VPL locations may remain unchanged:

The candidate is rejected and the path is duplicated;

Only the sub-path xc = x0, x1 is mutated;

Only the sub-path x s is mutated.

x0

xc

xv

xs

x1

Before

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 61: Interactive Light Transport with Virtual Point Lights

28/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

MIR - Cluster ”path VPLs” into ”geometric VPLs”

Cluster path VPLs with the same VPL location into one geometric VPL

When applying mutations, VPL locations may remain unchanged:

The candidate is rejected and the path is duplicated;

Only the sub-path xc = x0, x1 is mutated;

Only the sub-path x s is mutated.

x0

xc

xv

xs

x1

Mutation

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 62: Interactive Light Transport with Virtual Point Lights

28/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

MIR - Cluster ”path VPLs” into ”geometric VPLs”

Cluster path VPLs with the same VPL location into one geometric VPL

When applying mutations, VPL locations may remain unchanged:

The candidate is rejected and the path is duplicated;

Only the sub-path xc = x0, x1 is mutated;

Only the sub-path x s is mutated.

x0

xc

xv

xs

x1

After

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 63: Interactive Light Transport with Virtual Point Lights

28/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

MIR - Cluster ”path VPLs” into ”geometric VPLs”

Cluster path VPLs with the same VPL location into one geometric VPL

When applying mutations, VPL locations may remain unchanged:

The candidate is rejected and the path is duplicated;

Only the sub-path xc = x0, x1 is mutated;

Only the sub-path x s is mutated.

x0

xc

xv

x1

2 path VPLs into 1 geometric VPL

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 64: Interactive Light Transport with Virtual Point Lights

29/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

MIR - Accumulate the VPL contributions

Set of m geometric VPLs xvi

They bring a fixed amount of power to the camera equal to

Pi = ki · Pc/n;

n is the total number of path VPLs;

ki is the number of path VPLs connected to xvi.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 65: Interactive Light Transport with Virtual Point Lights

29/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

MIR - Accumulate the VPL contributions

Set of m geometric VPLs xvi

They bring a fixed amount of power to the camera equal to

Pi = ki · Pc/n;

n is the total number of path VPLs;

ki is the number of path VPLs connected to xvi.

We do not know the ”VPL power”

Suppose that the power of the VPL is equal to 1;

Compute the intensity of every pixel;

Evaluate the total power P ′i ;

Scale all pixel intensities by Pi/P′i .

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 66: Interactive Light Transport with Virtual Point Lights

30/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

MIR - Decrease the VPL correlation

Classical Issue with Metropolis-Hastings

Example: If a VPL is on a wall, there is a high probability that thenext one will be on the wall too.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 67: Interactive Light Transport with Virtual Point Lights

30/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

MIR - Decrease the VPL correlation

Classical Issue with Metropolis-Hastings

Example: If a VPL is on a wall, there is a high probability that thenext one will be on the wall too.

Increase Variance!

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 68: Interactive Light Transport with Virtual Point Lights

30/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

MIR - Decrease the VPL correlation

Classical Issue with Metropolis-Hastings

Example: If a VPL is on a wall, there is a high probability that thenext one will be on the wall too.

Increase Variance!

Replace Metropolis-Hastings by Multiple-Try Metropolis-Hastings

Idea (simplified explanation): generate many candidates atonce and try to keep the best one;

Does not really change the conception of the algorithm;

Details in the Ph.D. thesis.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 69: Interactive Light Transport with Virtual Point Lights

31/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Results - MH vs MTMH - Same Computation Times

MH MTMH

Figure: Exploration of left/right contributions (256 VPLs)

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 70: Interactive Light Transport with Virtual Point Lights

32/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Results and Comparisons

Different tests were made

Test scenes

With directly-lit scenes;

With many light sources;

With difficult visibility layouts.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 71: Interactive Light Transport with Virtual Point Lights

32/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Results and Comparisons

Different tests were made

Other algorithms

Standard Instant Radiosity [Kel97];

Power Sampling Technique [WBS03];

Bidirectional Instant Radiosity.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 72: Interactive Light Transport with Virtual Point Lights

33/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Results - Simple Scenes - 256 VPLs - Same ComputationTimes

Reference (standard)

STD - 0.02% BIR - 0.007%

Power Sampling - 0.008% MIR - 0.009%

Figure: Tests with Shirley’s Scene 10.Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 73: Interactive Light Transport with Virtual Point Lights

34/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Results - Difficult Visibility - 1024 VPLs - SameComputation Times

Standard / Power Sampling Bidirectional MIR

Figure: Indirect illumination stress tests.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 74: Interactive Light Transport with Virtual Point Lights

35/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Results - Difficult Visibility - 1024 VPLs - SameComputation Times

Standard / Power Sampling Bidirectional MIR

Figure: Indirect illumination stress tests.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 75: Interactive Light Transport with Virtual Point Lights

36/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Advantages

Thanks to MLT → Robust and fast sampling strategies;

Thanks to Instant Radiosity → Fast and efficient gatheringtechniques:→ We can use IGI;→ We can use rasterization techniques . . .

Non-intrusive algorithm → Can be used in any pre-existingrenderer already using VPLs and Path Tracing.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 76: Interactive Light Transport with Virtual Point Lights

37/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Drawbacks and Future Work

Does not handle flickering problems during animations

Nothing is made to ensure temporal coherency→ if one sample changes, the whole sequence is modified;

Solution: Reuse the previous samples with a sequentialsampler (see [GDH06]).

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 77: Interactive Light Transport with Virtual Point Lights

37/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Drawbacks and Future Work

And glossy and specular reflections ?!

What happens if a part of the scene is seen through a highlyglossy or a specular reflection ?

Solution - Already implemented in yaCORT:→ Instead, find the second diffuse surface to deposit the VPLwith probability P ;→ While gathering, compute a camera sub-path which has alength with the same probability.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 78: Interactive Light Transport with Virtual Point Lights

37/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Drawbacks and Future Work

And Caustics ?!

Does not easily handle caustics.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 79: Interactive Light Transport with Virtual Point Lights

38/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Summary

1 Introduction

2 Formalizing the Problem

3 Sampling VPLs: Metropolis Instant Radiosity

4 Accumulating VPL contributions

5 Coherent Metropolis Light Transport

6 Conclusion

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 80: Interactive Light Transport with Virtual Point Lights

39/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Non-interleaved Deferred Shading of Interleaved SamplePatterns

Goal

We want to accumulate the contributions of a VPL set.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 81: Interactive Light Transport with Virtual Point Lights

39/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Non-interleaved Deferred Shading of Interleaved SamplePatterns

Goal

We want to accumulate the contributions of a VPL set.

Issues

Many light sources == Large fillrate;

Many light sources == Many rasterization steps.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 82: Interactive Light Transport with Virtual Point Lights

39/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Non-interleaved Deferred Shading of Interleaved SamplePatterns

Goal

We want to accumulate the contributions of a VPL set.

Issues

Many light sources == Large fillrate;

Many light sources == Many rasterization steps.

Strategy: combine two techniques

Deferred Shading → geometry rasterized once;

Interleaved Sampling → decreases fill rate and maintains goodimage quality.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 83: Interactive Light Transport with Virtual Point Lights

40/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Deferred Shading [DWS+88, ST90]

Principles

The per-pixel geometric information is stored in a GeometricBuffer (G-buffer) (Normals, positions and colors)

The G-buffer is then read to perform any lightingcomputation.

It greatly simplifies the rendering pipeline and it alsoprevents the geometry from being reprojected each time a

shading pass is performed.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 84: Interactive Light Transport with Virtual Point Lights

41/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Interleaved Sampling [KH01]

Instead of evaluating all VPL contributions for all pixels, weuse separate subsets of VPLs for every neighbor pixel.

(a) Standard Sampling (b) Interleaved Sampling

Already used in ray tracing → ”Instant Global Illumination”Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 85: Interactive Light Transport with Virtual Point Lights

41/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Interleaved Sampling [KH01]

Instead of evaluating all VPL contributions for all pixels, weuse separate subsets of VPLs for every neighbor pixel.

Already used in ray tracing → ”Instant Global Illumination”Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 86: Interactive Light Transport with Virtual Point Lights

42/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Overview of the Algorithm

Creation Splitting Shading Gathering

Discontinuity Filtering Blending

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 87: Interactive Light Transport with Virtual Point Lights

42/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Overview of the Algorithm

Creation Splitting Shading Gathering

Discontinuity Filtering Blending

Conservative extension of deferred shading: all algorithms usingdeferred shading may also be used with our system.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 88: Interactive Light Transport with Virtual Point Lights

43/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Core of the Algorithm: G-buffer Splitting

Principle

G-buffer G split into n × m smaller tiled sub-buffers Gi ,j

Texel (x , y) from G goes to texel (x/n, y/m) of sub-bufferGi , j with i = x mod n and j = y mod m.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 89: Interactive Light Transport with Virtual Point Lights

43/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Core of the Algorithm: G-buffer Splitting

Principle

G-buffer G split into n × m smaller tiled sub-buffers Gi ,j

Texel (x , y) from G goes to texel (x/n, y/m) of sub-bufferGi , j with i = x mod n and j = y mod m.

Fast Solution - Two-pass splitting

Small blocks are split;

Split blocks are translated.

Results: fast

A 1024 × 1024 192 bit G-buffer is split in 7 ms on a 6800GT;

20 ms with a one-pass splitting.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 90: Interactive Light Transport with Virtual Point Lights

44/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Core of the Algorithm: Filtering

Coherency of the pixels

Discontinuity buffer;

Box blur on continuous zones of the screen.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 91: Interactive Light Transport with Virtual Point Lights

44/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Core of the Algorithm: Filtering

Coherency of the pixels

Discontinuity buffer;

Box blur on continuous zones of the screen.

+X

P

+X

PFigure: Box Blur

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 92: Interactive Light Transport with Virtual Point Lights

44/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Core of the Algorithm: Filtering

Coherency of the pixels

Discontinuity buffer;

Box blur on continuous zones of the screen.

Interleaved Sub-sampling

Figure: Quality

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 93: Interactive Light Transport with Virtual Point Lights

45/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Results - 500 sources - 1024× 768 - IS 8 × 6

Fully interactive applications

No visibility for secondary light sources;

Fast!

69 f/s 36 f/s (×31) 64 f/s 29 f/s (×25)

58 f/s 29 f/s (×26) 57 f/s 29 f/s (×30)Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 94: Interactive Light Transport with Virtual Point Lights

46/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Results - Physically Based Rendering - 1280 × 1024 - IS2 × 2

Physically based rendering

Visibility for secondary light sources

0.7 s - 14 f/s (×3.4) 4.0 s - 2.5 f/s (×1.5)

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 95: Interactive Light Transport with Virtual Point Lights

47/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Conclusion

To sum up . . .

Generic extension of deferred shading;

Trade-off between quality and speed.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 96: Interactive Light Transport with Virtual Point Lights

48/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Summary

1 Introduction

2 Formalizing the Problem

3 Sampling VPLs: Metropolis Instant Radiosity

4 Accumulating VPL contributions

5 Coherent Metropolis Light Transport

6 Conclusion

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 97: Interactive Light Transport with Virtual Point Lights

49/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Issues with Virtual Point Lights

VPL based techniques

Fast;

Simple;

Elegant.

But:

Do not handle all lighting phenomena;

Use the same VPL family for all pixels.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 98: Interactive Light Transport with Virtual Point Lights

49/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Issues with Virtual Point Lights

VPL based techniques

Fast;

Simple;

Elegant.

But:

Do not handle all lighting phenomena;

Use the same VPL family for all pixels.

Alternative approach

Instead of making Instant Radiosity more robust, make MetropolisLight Transport more coherent and faster.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 99: Interactive Light Transport with Virtual Point Lights

50/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Advantages and Drawbacks of Metropolis Light Transport

Advantages

Conceptually super simple;

Very robust → it samples the density we want;

Handles all kinds of lighting phenomena.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 100: Interactive Light Transport with Virtual Point Lights

50/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Advantages and Drawbacks of Metropolis Light Transport

Advantages

Conceptually super simple;

Very robust → it samples the density we want;

Handles all kinds of lighting phenomena.

Drawbacks

Pretty difficult to implement;

Slow! → does not use efficient techniques like rasterization orcoherent ray tracing.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 101: Interactive Light Transport with Virtual Point Lights

51/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Coherent Metropolis Light Transport

Core idea

Replace standard MCMC mutations by Multiple-try ones.Goal: Generating many paths at the same time.

Uses SIMD computations;

Factorizes cache accesses!Fundamental for any commercial renderer

x

D

D

E

ED diffuse

eye

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 102: Interactive Light Transport with Virtual Point Lights

52/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Split MLT in Three Steps

Step 1: Exploration of the sample space with MCMC mutations

Standard Metropolis Light Transport

timeBenjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 103: Interactive Light Transport with Virtual Point Lights

52/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Split MLT in Three Steps

Step 1: Exploration of the sample space with MCMC mutations

Provides a set of n path samples (x i )i∈[1...n] with density

f (c)/||f (c)|| (Squares)

timeBenjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 104: Interactive Light Transport with Virtual Point Lights

52/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Split MLT in Three Steps

Step 2: Fast exploration of the lens sub-space

Lens sub-space: ES∗DS∗(L|D)Sub-paths from the camera which intersect two diffuse surfaces

timeBenjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 105: Interactive Light Transport with Virtual Point Lights

52/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Split MLT in Three Steps

Step 2: Fast exploration of the lens sub-space

Use multiple-try mutations. At each step, two families:

”Candidates” x∗1 . . . x∗

p (Disks);

”Competitors” x∗∗1 . . . x∗∗

p (Triangles).

timeBenjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 106: Interactive Light Transport with Virtual Point Lights

52/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Split MLT in Three Steps

Step 3: Accumulate all sample contributions

Each family: use of the ”expected value”: accumulate x∗ accordingto Rg and x∗∗ according to 1 − Rg ;

Each element: accumulate each element x proportionally to f (c)(x)

time

Camera

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 107: Interactive Light Transport with Virtual Point Lights

53/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Implementation - Mutation strategies

Exploration of the entire space: standard MLT with bidirectionalmutations only;

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 108: Interactive Light Transport with Virtual Point Lights

53/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Implementation - Mutation strategies

Exploration of the lens sub-space: use of lens and caustics pertuba-tions.

Jittering;

Gaussiandistributions aroundthe initial samples;

”Packetize” therays → use SIMDinstructions.

S

D

D

E L

a) b)

c)Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 109: Interactive Light Transport with Virtual Point Lights

53/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Implementation - Mutation strategies

Exploration of the lens sub-space: use of lens and caustics pertuba-tions.

With pictures . . .

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 110: Interactive Light Transport with Virtual Point Lights

54/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Results

Quality equivalent to the quality obtained with MLT but . . .

As MLT, some parameters have to be carefully tuned:

Lengths of MTMH sub-sequences;

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 111: Interactive Light Transport with Virtual Point Lights

54/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Results

Quality equivalent to the quality obtained with MLT but . . .

As MLT, some parameters have to be carefully tuned:

Lengths of MTMH sub-sequences;

σ and the number of MTMH candidates.

(a) σ = 16 pixels (b) σ = 32 pixels (c) σ = 64 pixels

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 112: Interactive Light Transport with Virtual Point Lights

54/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Results

Quality equivalent to the quality obtained with MLT but . . .

As MLT, some parameters have to be carefully tuned:

Lengths of MTMH sub-sequences;

σ and the number of MTMH candidates.

Overall performance with SIMD

Speed-up from 1.5 to 2.3.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 113: Interactive Light Transport with Virtual Point Lights

55/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Results - Cache Simulation

Affinity with caches is fundamental

Distribution ray tracing in complex scenes [CLF+03, Chr06];

Multi-resolution geometry caching;

On-the-fly tesselation;

Memory systems with difficult layouts:Cell processors (PS3, Blade Center)Xenon (XBOX 360)LarabeePC cluster . . .

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 114: Interactive Light Transport with Virtual Point Lights

55/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Results - Cache Simulation

Theater Three Dragons4 Tri 16 Tri 128 Tri 4 Tri 16 Tri 128 Tri

MLT 53% 61% 60% 54% 63% 70%CMLT 86% 92% 98% 81% 95% 99%

IR 87% 93% 99% 82% 95% 99%

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 115: Interactive Light Transport with Virtual Point Lights

56/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

CMLT - Conclusion and Remarks

Faster than MLT

Simple extension of MLT → reorganization of thecomputations;

Not real time (and not even interactive) but may be a goodalternative.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 116: Interactive Light Transport with Virtual Point Lights

56/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

CMLT - Conclusion and Remarks

Faster than MLT

Simple extension of MLT → reorganization of thecomputations;

Not real time (and not even interactive) but may be a goodalternative.

But . . .

As MIR, does not handle flickering problems during animation.It is a major problem with ”importance driven” methods.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 117: Interactive Light Transport with Virtual Point Lights

57/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Summary

1 Introduction

2 Formalizing the Problem

3 Sampling VPLs: Metropolis Instant Radiosity

4 Accumulating VPL contributions

5 Coherent Metropolis Light Transport

6 Conclusion

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 118: Interactive Light Transport with Virtual Point Lights

58/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Conclusion

During three years . . .

Many implementations: GPU, coherent ray tracing;

Many numerical schemes.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 119: Interactive Light Transport with Virtual Point Lights

58/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Conclusion

But . . . no ”ultimate” renderer was found!

However . . . some personal points of view

For absolute realism and large interactivity:

Monte-Carlo + Brute Force + Carefully DesignedImplementation is the only solution

No compression, no PRT, no expensive pre-computation;

Rasterization vs ray tracing → Geometric efficiency vs lightingsimulation efficiency?

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 120: Interactive Light Transport with Virtual Point Lights

59/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Merci!

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 121: Interactive Light Transport with Virtual Point Lights

59/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Per Christensen.Ray Tracing for the Movie ”Cars”.In IEEE Symposium on Interactive Ray Tracing, pages 1–6,2006.

Per Christensen, David M. Laur, Julian Fong, Wayne L.Wooten, , and Dana Batali.Ray Differentials and Multiresolution Geometry Caching forDistribution Ray Tracing in Complex Scenes.Proceedings of Eurographics 2003, pages 543–552, 2003.

Philip Dutre, Eric Lafortune, and Yves Willems.Monte Carlo Light Tracing with Direct Computation of PixelIntensities.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 122: Interactive Light Transport with Virtual Point Lights

59/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

In Compugraphics ’93 (Proceedings of Third InternationalConference on Computational Graphics and VisualizationTechniques), 1993.

Michael Deering, Stephanie Winner, Bic Schediwy, ChrisDuffy, and Neil Hunt.The Triangle Processor and Normal Vector Shader: a VLSISystem for High Performance Graphics.In SIGGRAPH ’88 (Proceedings of the 15th annual conferenceon Computer graphics and interactive techniques), pages21–30, 1988.

Abhijeet Ghosh, Arnaud Doucet, and Wolfgang Heidrich.Sequential Sampling for Dynamic Environment MapIllumination.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 123: Interactive Light Transport with Virtual Point Lights

59/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

In Proceedings of the 17th Eurographics Symposium onRendering, pages 116–126, 2006.

Henrik Wann Jensen.Global Illumination Using Photon Maps.In Proceedings of the 7th Eurographics Symposium onRendering, pages 21–30, 1996.

Henrik Wann Jensen.Rendering Caustics on Non-Lambertian Surfaces.In Computer Graphics Forum (Proceedings of Eurographics1997), pages 57–64, 1997.

Henrik Wann Jensen.Realistic image synthesis using photon mapping.A. K. Peters, Ltd., 2001.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 124: Interactive Light Transport with Virtual Point Lights

59/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

James Kajiya.The Rendering Equation.In SIGGRAPH ’86 (Proceedings of the 13th annual conferenceon Computer graphics and interactive techniques), pages143–150, 1986.

Alexander Keller.Instant Radiosity.In SIGGRAPH ’97 (Proceedings of the 24th annual conferenceon Computer graphics and interactive techniques), pages49–56, 1997.

Alexander Keller and Wolfgang Heidrich.Interleaved Sampling.In Proceedings of the 12th Eurographics Symposium onRendering, pages 269–276, 2001.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 125: Interactive Light Transport with Virtual Point Lights

59/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

Jaroslav Krivanek.Radiance Caching for Global Illumination Computation onGlossy Surfaces.PhD thesis, Universite de Rennes 1 and Czech TechnicalUniversity in Prague, 2005.

Eric Lafortune and Yves Willems.Bi-directional Path Tracing.In Compugraphics ’93 (Proceedings of Third InternationalConference on Computational Graphics and VisualizationTechniques), pages 145–153, 1993.

Takafumi Saito and Tokiichiro Takahashi.Comprehensible Rendering of 3-D Shapes.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 126: Interactive Light Transport with Virtual Point Lights

59/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

In SIGGRAPH ’90 (Proceedings of the 17th annual conferenceon Computer graphics and interactive techniques), pages197–206, 1990.

Eric Veach.Robust Monte-Carlo Methods for Light Transport Simulation.PhD thesis, Standford University, 1997.

Eric Veach and Leonidas Guibas.Bidirectional Estimators for Light Transport.In Proceedings of the 5th Eurographics Symposium onRendering, pages 147–162, 1994.

Eric Veach and Leonidas Guibas.Metropolis Light Transport.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 127: Interactive Light Transport with Virtual Point Lights

59/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

In SIGGRAPH ’97 (Proceedings of the 24th annual conferenceon Computer graphics and interactive techniques), pages65–76, 1997.

Gregory Ward.The RADIANCE Lighting Simulation and Rendering System.In SIGGRAPH ’94 (Proceedings of the 21st annual conferenceon Computer graphics and interactive techniques), pages459–472, 1994.

Ingo Wald, Carsten Benthin, and Philipp Slusallek.Interactive Global Illumination in Complex and HighlyOccluded Environments.In Proceedings of the 14th Eurographics Symposium onRendering, pages 74–81, 2003.

Gregory Ward, Francis Rubinstein, and Robert Clear.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights

Page 128: Interactive Light Transport with Virtual Point Lights

59/59

PresentationIntroduction

Formalizing the ProblemSampling VPLs: Metropolis Instant Radiosity

Accumulating VPL contributionsCoherent Metropolis Light Transport

Conclusion

A Ray Tracing Solution for Diffuse Interreflection.In SIGGRAPH ’88 (Proceedings of the 15th annual conferenceon Computer graphics and interactive techniques), pages85–92, 1988.

Benjamin Segovia Interactive Light Transport with Virtual Point Lights