Verifying the “Consistency” of Shading Patterns and 3-D

Structures

Pawan Sinha & Edward Adelson

What is the 3D Structure of Each Image?

Infinite Number of Possible Interpretations

Goal 1: 3D Shape Recovery

Goal 2: Check Shading Consistency

Input Patterns

Goal 1: Propose 3D Structure

● Questions:– What distinguishes correct structure?– How to search for it algorithmically?

What Distinguishes Correct Structure?

Wrong Right

What Distinguishes Correct Structure?

● Low “Complexity”– Low Angle Variance– Planarity of Faces– Overall Compactness

How to Search?

● Minimize Cost Function– How to weight constraints?– Cumbersome

● Incremental Solution– Start with 2-D Line Drawing– “Pull” vertices until regularity is

maximized● i.e. Gradient descent in regularity space

Incremental Solution

Goal 2: Check Shading Consistency

● Given– 3D structure– 2D gray-level image

● Assume– Structure is uniformly colored

● Find– Single light source to account for shading

Quantitative Approach

● Given:– Lambertian Reflectance Model– Surface Normal– Surface Brightness

● Defines:– A cone of valid light directions for each

surface

Cone of valid light direction

N=surface normalE=brightnessarccos E

N

Quantitative Approach

● Consider cones for all surfaces● Intersection is direction of illumination

Problems

● Small changes in grey lead to no solution

Surface l3 changes brightness

Gradient SpaceInput Patterns

BeforeAfter

Intersection

Qualitative Approach

● Observation– Human vision

● Good at judging relation between brightness● Bad at judging absolute brightness

● So...– Use binary relations to:

● find light source● Not commit to particular reflectance function

Qualitative Approach

● Each surface now defines a hemisphere of possible light directions– Overall consistency implies finding a non-

null intersection of hemispheres

Qualitative Approach

● Hemisphere ~ set of vectors t s.t

Angle between s and t is less than 90sij t 0where sij satisfies

sij ni n j 2 0sij ni n j 0sij ni 0 sij n j 0 if surfacei brighter than surface j

sij ni 0 sij n j 0 if surface j brighter than surfacei

Qualitative Approach

● Hemisphere ~ set of vectors t s.t

sij is defined between a surface i and surface j, with normals ni and nj

sij t 0where sij satisfies

sij ni n j 2 0sij ni n j 0sij ni 0 sij n j 0 if surfacei brighter than surface j

sij ni 0 sij n j 0 if surface j brighter than surfacei

Qualitative Approach

● Hemisphere ~ set of vectors t s.t

s is perpindicular to the average of the normals

sij t 0where sij satisfies

sij ni n j 2 0sij ni n j 0sij ni 0 sij n j 0 if surfacei brighter than surface j

sij ni 0 sij n j 0 if surface j brighter than surfacei

Qualitative Approach

● Hemisphere ~ set of vectors t s.t

s is perpindicular to plane defined by the normals

sij t 0where sij satisfies

sij ni n j 2 0sij ni n j 0sij ni 0 sij n j 0 if surfacei brighter than surface j

sij ni 0 sij n j 0 if surface j brighter than surfacei

Qualitative Approach

● Hemisphere ~ set of vectors t s.t

The angle between s and ni is less than 90 & the angle between s and nj is greater

sij t 0where sij satisfies

sij ni n j 2 0sij ni n j 0sij ni 0 sij n j 0 if surfacei brighter than surface j

sij ni 0 sij n j 0 if surface j brighter than surfacei

Solution

● Solution to constraints lie on a convex polygon on the unit sphere.

Direction of agreement

Is it consistent?

● If no polygons found that satisfy all constraints, then– Shading is not consistent

No maximum

Largely Solveable

● For each polygon, count # of constraints matched– A maximum indicates most likely lighting

direction

Compound Edges

● What do unsatisfied constraints represent?– Compound edge – where surface changes

color, not just shading

Compound edges

Justification

● Why do shape derivation with line drawings and not brightness?– Humans can use edges

● Can humans use grey level?– It seems like it– An experiment

An Experiment

● Random Height Tesselation– Can human determine 3d structure?

3D Representation

Overhead View

Limitations

● Polyhedral objects only● Later research addresses contoured

objects w/ smoothly changing brightness.

Conclusions

● Humans mostly use edges to determine 3D structure

● Use shading to verify this determination

● Algorithm effective for polyhedra