The Plenoptic Function Lázaro Hermoso Beltrán. 2 Previous Concepts “The body of the air is full...

The Plenoptic Function

Lázaro Hermoso Beltrán

2

Previous Concepts“The body of the air is full of an infinite number of radiant pyramids caused by the objects located in it”

Leonardo Da Vinci

• Pencil of rays: The set of light rays passing through any point in space

• Plenoptic function • Plenus : complete or full• Optic

3

Definition (I)“Radiance received along any direction V arriving at any point

E in space, at any time ‘t’ and over any range of wavelength”

• 7-Dimensional function

),,,,,,( tEzEyExvvPI

4

Geometric Components of the pencil of ray lights

ANGULAR COORDINATES

– E = (Ex, Ey, Ez) • Viewpoint

– . • Direction of the ray light passin trough the

Viewpoint

CARTESIAN COORDINATES

– Comonly used in machine vision

)( , vvV

),,,,,,( tEzEyExvvPI

),,,,,,( tEzEyExyxPI

5

How would obtain plenoptic function

1. Placing an imaginary eye at every position2. Record the intensity of light at every angle3. For every wavelength4. At every time t

It does not need specify the direction of gaze!!

)( , vv

6

The Holographic Movie Example

• Black and white photo

• Color photograph

• Color movie

• Color holographic movie

)( P

),( P

),,,,,( tEEEP zyxvv

),( tP vv

7

Plenoptic Measurements in Human Vision

• PF potential iformation available

• Observer takes samples from PF

• Not all the information is usefull. Enviroment has determined what is usefull (movements predators). Enviroment constraints

8

P.M. in Human Vision

•Measurements has limitedresolution and number ofSamples

•Space and time the more importants

9

P.M. in Human Vision

• An example of what we can not measure

http://web.mit.edu/persci/demos/Lightness/gaz-teaching/flash/koffka-movie.swf

10

P.M. in Human Vision

11

Plenoptic Measurements: Red Bar Example

(Vx, Vy, Vz)

12

P.M. in Machine Vision

•Develop a taxonomy of derivative types

•“Periodic Table”of visual elements

•Constraints Available information (PF)

13

Aplications: Image Based Rendering

• IBR : Sampling and Rendering• Problem, PF is 7D function.We need reduce it!

• RGB representation

• Plenoptic modelling

• Light Field/Lumigraph P(s, t, u, v)

• 2D-Image Mosaicing

),,,,( EzEyExvvP

14

RGB – Representation

• Three 5-D functions

),,,,( EzEyExvvPG

),,,,( EzEyExvvPR ),,,,( EzEyExvvPB

15

Plenoptic Modelling

• Without time and wavelength

• Taking samples with cameras in enough close positions

• We can reconstruct PF interpolating the samples

),,,,( EzEyExvvP

16

Light Field

• Assumption that that rays does not change intensity along direction

• Parametrization rays with 2 parallel planes

17

2D-Image Mosacing•To paste serial 2D images to obtain a wider image. The images share the same proyection point

•Panoramic mosaic or Panorama

•We reconstruct a 2D Plenoptic function

18

Interpolate Views

19

References

• E.H. Adelson and J. Bergen, “The plenoptic function andthe elements of early vision” In Computational Models ofVisual Processing, pages 3-20. MIT Press, 1991

• The Plenoptic Illumination Function Tien-TsinWong1 Chi-Wing Fu2 Pheng-Ann Heng1 Chi-Sing Leung3

• McMillan, Bishop. “Plenotic Modeling: An Image-Based Rendering System”

• Shum, Kang. “A Review of Image-based Rendering Techniques”