The Natural-Constraint Representation of the Path Space for Efficient Light Transport Simulation

Anton S. Kaplanyan1,2 and Johannes Hanika1 and Carsten Dachsbacher1

1Karlsruhe Institute of Technology, 2Lightrig

Im going to present our paper on a formulation of light transport based on natural constraints, which is very robust and particularly well-suited for glossy transport.1Motivation2

[Wikipedia Commons]

[Wikipedia Commons]

Rendered with manifold exploration [JakobMarschner12]

Rendered with out method

JewelryScenes with complex glossy transportMany materials are glossyPlastic, metal, coating, etc.Hard to render glossy highlights

WantRobust glossy rendering

2

Intuition: Constraints in a Lens SystemTwo endpoints with specular constraints3

Our initial observation stems from lens design and the recent manifold exploration method, where a path with specular interactions can be seen as a constraint satisfaction problem between two end points, where the constraints are shown on the left in their respective tangent frames.3

Idea: Deviate from Specular PathRough scattering: use soft constraints4

So, what if we make a lens rough? It turned out that the specular constraints turn into soft constraints, which can be deviated from their central position. We propose to represent a light path in this new domain of end points and soft constraints in between.4OutlineIntroductionPrior workNew generalized coordinatesProperties of the domainOverview: Metropolis light transportNew half-vector mutationResultsConclusion

5TheoryPracticeMy talk consists of two parts: first I will talk about the theoretical properties of the new domain. And in the second part I will discuss a practical method for rendering images in this domain.5Rendering with Light TransportGenerate image by computingFlux incident on the sensor

Sample all possible pathsStochastic integration6

So, we are interested in rendering an image. That involves computing a flux incident on the sensor carried by light paths. In order to compute this quantity, we need to integrate over all possible paths from the light source to the sensor through the virtual scene. Such space of all paths is called a path space.We usually employ stochastic integration, like MC or MCMC, due to the high dimensionality of the problem.6EmissionBSDFsAbsorptionGeometric termsPath Integral Framework7Lets formalize the problem. For each pixel, we need to compute an incident flux quantity I_j by taking a path integral over all possible paths in area measure. The path x^bar here is a set of vertices. And the integrand f() is a measurement contribution function that defines the amount of energy carried by a path. -> It consists of the descriptions of the light source and the sensor, as well as the BSDFs (..), which describe the materials.Most of these distributions are defined in the solid angle domain. To be able to integrate in area measure, we need to convert them using the geometric terms. Note that a BSDF becomes singular for pure specular interactions, which poses some problems for the integration.7Prior Work: Specular PathsSpecular paths are hard constraintsObey Fermat principle [Alhazen1021]

Ray transfer matrices used in optics [Gauss1840]Pencil tracing in graphics [Shinya87]8

Start pointEndpointOptical system

In fact, that means that the specular paths define hard constraints in the path space that define the optimal Fermat trajectory. To formalize the specular constraint, we first define a halfway vector, which is a vector that bisects incident and outgoing direction for reflection. Then perfect specular reflection constraint states that this halfway vector should coincide with surface normal. For perfect refraction the half vector can be easily generalized.This problem of specular paths has been known from the lens design. Gauss first proposed to use the first order approximation of the lens system response to the incident ray.This matrix contains geometric properties of the optical system and can be also used to compute the energy density change along the path through the optical system. In fact, it tracks the deformation of a unit tangent frame along the path. This is a convenient way to compute the energy throughput along the path. This has been applied in graphics for pencil tracing by Shinya et al.8Prior Work: Specular PathsRendering specular paths with geometric knowledgeSolving for constraints [MitchellHanrahan92]First and second order analysis [ChenArvo00]Predictor-corrector perturbations [JakobMarschner12]

Our work is inspired by manifold exploration9

[Mitchell and Hanrahan 1992]

[Chen and Arvo 2000]

[Jakob and Marschner 2012]Rendering specular paths with geometric knowledge has been studied in graphics since then.E.g., Mitchell and Hanrahan employed the knowledge about implicit geometry to find specular caustics.Chen and Arvo used first and second order approximation of geometry around the path to interpolate between sparsely sampled specular paths.The predecessor of our work, manifold exploration by Wenzel Jakob and Steven Marschner, exploits the knowledge of differential geometry around the path to solve for specular constraints along the path in Metropolis light transport framework. We base our domain on the results of the manifold exploration work.9TheoryThe Domain of Halfway Vectors for Light PathsHere, I will start with the theoretical properties of the new domain.10Generalized Coordinates11

First, let us define the new domain: each path in it is represented by two end points and a series of halfway vectors in between. This is so called generalized coordinates, coordinates known from physics that reparameterize the system in a more convenient way.Note, that this parameterization is unique only on a small island of smooth geometry.

In case of scattering, the half vectors deviate around the surface normal, whereas for pure specular interactions, they perfectly match the surface normal.11

Deviating the Half VectorsA path in generalized coordinates12Here is the example of the generalized coordinates domain in action. We can move the end points. Or we can fix the end points and change the half vectors (on the left) in their respective tangent frames. Each change leads to a new light path.12Path Contribution with New Formulation13EmissionScattering distributionsCamera responsivityTransfer matrix + Geometric term= Generalized G

Area measureHalf vector domainNow, let us write the energy carried by a path. It consist of the description of light source emission and camera responsivity. In addition, there are scattering distributions that are defined in half vector domain. Note that such scattering distributions can be obtained from BSDFs by a change of variables.Then, the transfer matrix provides a change of energy density throughout the path and the last geometric term is needed to convert the solid angle quantity to the area measure. Note that these two terms form a so called generalized geometric term throughout the whole path.This new measurement contribution function contains only one geometric term, thus avoiding most of the singularities that arise in geometric terms due to one over distance squared term in them (here is a simple example on the right).This can be seen as the generalization of ray transfer matrix method to arbitrary scattering along the path.13Simplified Measurement14Lets see how we can convert between different domains.1.The conversion from rendering equation to path integral requires a change of measure from solid angle to area measure. This is done using the geometric term.2. On the other hand, we can compute scattering in half vector domain by applying the Jacobian change of variables known from microfacet theory.3. Finally, the change of variables from surface area measure to half vector domain was actually introduced in manifold exploration work and is represented as a block-tridiagonal matrix.This way we can convert our quantities from one domain to another and show the equivalence of the new measurement contribution function to the old one. Please refer to the paper for more details.14Decomposition of Path IntegralDecorrelated islandsSet of 2D integralsEasy-to-predict spectrum

Mostly changes local BSDFWell-studied sampling15

It turned out that there are interesting properties of the path integral in this new domain. There is a simple experiment on the right. We change only one half vector at x_2, while keeping all others fixed. There is a change of measurement contribution of the whole path shown in false colors on the bottom left in the unit circle domain of the half vector h_2. The change of the local BSDF at x_2 is shown next to it. You can see that they closely match, which means that the major change to the path energy comes just from the local BSDF.This shows that the path integral can be decomposed into a series of almost independent 2D integrals at each half vector. This also makes it easy to estimate various properties, e.g., spectral bandwidth of the path integral in the new domain.15Practical RenderingMutation Strategy for Metropolis Light TransportHere I will show how we can practically integrate and render images in this new domain.16Metropolis Light TransportTake a path and perturb it [VeachGuibas97]Specialized mutation strategiesManifold exploration (ME) [JakobMarschner12]17

Our method works as a perturbation in the Metropolis light transport framework. MLT has been introduced to graphics by Eric Veach with the set of specialized mutation strategies for perturbing light paths. This set has been recently extended by a new manifold exploration mutation, which can connect through multiple vertices while preserving specular constraints.17Metropolis Light Transport18Here is a short outline of Metropolis light transport.First we initialize Markov chain with a light path that comes from some alternative method like BDPT.Then we run the random walk in path space by mutating the current pathAt every step a mutation routine proposes a new path.This path is either accepted or rejected with some probabilityThen the resulting path is accumulated on the image

We provide a mutation strategy that can mutate in the new domain18Half-vector Space MutationMutation1. Perturb half-vectors2. Find a new pathSimilar machinery to ME (see ME paper)Specular chains: fall back to ME

Jump over geometric partsTake prediction as a proposal19

Our mutation strategy on a high level first represents the path in the new domain just by computing the half vectors.Then we perturb all half vectors along the path at once (Ill talk about it shortly).Then we find a trajectory that corresponds to the new half vectors by applying a predictor-corrector algorithm that is similar to the one used in manifold exploration. In fact, it falls back to manifold exploration in case of specular interactions, where the half vectors are fixed.

As I mentioned before, the path can be uniquely defined in this domain only in some local geometric island, where the geometry is smooth enough. However, it is often the case that the proposal runs out of the valid geometric configuration. In this case we try to jump from one such island to another by taking this prediction and just tracing the new path. Please refer to the paper for more details.19Importance-Sample All BSDFsQuery avg. BSDF roughnessApproximate with Beckmann lobeSample as ~2D Gaussian

Known optimal step sizes from MCMC20

When perturbing half vectors, we can take into account the local BSDFs.Due to the decorrelation of the path integral, we can account only for a local BSDF at every half vector.Luckily, half vector domain is also a suitable domain for BSDFs. We take a Beckmann equivalent of the underlying BSDF, which is close to a 2D Gaussian in our domain. The optimal step size for a 2D Gaussian is well studied in MCMC theory, so we apply it. On the bottom you can see the results of sampling with different fixed step sizes and the optimal one.20Results: Kitchen21

HSLTPSSMLTVMLTMEMLTHSLTHere are the results produced by our method. The kitchen scene has a lot of glossy materials.PSSMLT is quite noisy. VMLT is smoother, but produces noisy splotches on the corners. MEMLT generates an overall good image, yet leading to scratches at long glossy chains. Our half vector space light transport (HSLT) tries to sample the glossy highlights more uniformly.21

Results: Necklace22HSLTVMLTMEPTMEMLTHSLTThis is even more noticeable with glossy hightlights that experience many interactions. Here manifold exploration constantly jumps off the glossy highlight, as it breaks the constraint at one of the vertices. The proposed method can efficiently explore such long chains of glossy interactions.22ConclusionConvenient domain for paths on surfacesGeneralized coordinatesBeneficial properties of path integral

Sampling in generalized coordinates Robust light transport (especially glossy and specular)Importance sampling of all BSDFs along a pathPractical stratification for MLT23To conclude, we proposed to compute the light transport in the new generalized coordinates domain, where the path integral has good properties.We also propose a practical sampling method in this domain, that takes into account all BSDFs along the path. We also propose a new stratification in this domain. Please refer to the paper.23Limitations and Future WorkGeometric smoothnessLevel of detail for displaced geometry

Rare events: needle in a haystackProbability-1 search

Participating mediaMore dimensions, new soft constraints24On the other hand, we rely on the valid geometry along the path even more than manifold exploration.Also, finding small glossy highlights can be difficult. And lastly, the generalized coordinates are defined only for surfaces, making their generalization to the volumes a future work.24Thank YouQuestions?Backup: Stratification for MCMCExpected change on the image from changes in half vectors26

Without stratificationWith stratificationBackup: Spectral SamplingHow to distribute step sizes among half vectors?Spectral sampling for MC [SubrKautz2013]Convex combination based on bandwidth (see paper)27

Without redistributionWith redistribution