Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
School of Electrical Engineering and Computer Science
Kyungpook National Univ.
An Analysis of Visual Adaptation and
Contrast Perception for Tone MappingIEEE Transactions on Pattern Analysis and Machine Intelligence
Vol. pp, No. 99, 2011
Sira Ferradans, Marcelo Bertalmio, Edoardo Provenzi and Vicent Caselles
Presented by Il-Su Park
Abstract
Tone mapping
– Problem of compressing the range of a High Dynamic
Range image
Proposed method
– Tone mapping operator
• Visual adaptation
– Based on experiments on human perception
» Point out the importance of cone saturation
• Local contrast enhancement
– Based on a variational model inspired by color vision
phenomenology
2 / 29
Introduction
Tone mapping operators
– Perceptually-based TMOs
• Global
– Very fast
– Not introduce halos or artifacts
– Poor contrast rendition
– Stevens‟ law
– Naka-Rushton formula
– Weber-Fechner‟s law
• Local
– Improving details
– Create halos and artifacts next to edges
3 / 29
– Gradient-based TMOs
• Shrinking large intensity gradients
• Preserving small fluctuations corresponding to fine details
– Proposed method
• Main purpose of TM
– Emulating as much as possible the perception of contrast and
color produced by the real-world scene
• Following the TM philosophy proposed by Ward et al.
• Replicating basic properties of the HVS
– Taking into account localized cone saturation
– Computing the semisaturation constant in the Naka-Rushton
equation
4 / 29
Visual adaptation
Human visual systems
– Operating over such a huge range
• From to
– Cannot operate over entire range simultaneously
– Adapting locally to the average of nearby retinal
illumination values
– Handling a smaller magnitude interval
6 210 cd/m 6 210 cd/m
5 / 29
– Neuroscience experiments
• Using very simple, non-natural images
– Superimposing brief pulses of light with intensity I on a uniform
background
• Measuring the electric response inside the retina
– Changing accordingly to the empirical law
» Michaelis-Menten equation or Naka-Rushton equation
max
n
n n
s
Vr
V
(1)
where is the light level at which the photoreceptor response is half maximal,
the change of electric potential is the photoreceptor‟s
physiological response to , and
is the highest difference of potential that can be generated, and
is a constant.
s
V
maxVn
6 / 29
– Foundational experiments of psychophysics
• Developed by the German physician E.H. Weber
• Measuring the noticeable difference between the background
light and the superimposed light
• Weber‟s law
– Constant ratio between the Just Noticeable Difference(JND)
and background intensity I
– Linear relationship between and in log-log units
– Not hold for low intensity values
k (2)
where is a perceptual constant called Weber fraction.0k
log log log k
7 / 29
• Weber-Fechner‟s law
– Introducing the concept of „dark light‟
– Matching the experimental data also at low intensity values
– Plotting JND vs. I data in log-log scale
» Known as threshold versus radiance(t.v.r.) or threshold
versus intensity(t.v.i.)
km
(3)
where is a quantity often interpreted as internal noise in the visual
mechanism.
0m
8 / 29
• Cone saturation
– Increasing very steeply after a certain point
Fig. 1. Cones saturate when the stimuli field is not steady
but pulsed: perceptual t.v.r. curves of cones for pulsed
and steady stimuli, figure adapted from Shevell [34].
9 / 29
• Rewriting Weber-Fechner‟s law
– Using JND as a sensation magnitude
– Integrating eq. (4)
k sm
(4)
where is the increment in the sensation-magnitude function ,
also called “perceived brightness”.
s s
0logs k m s (5)
where .1
kk
10 / 29
– Perceived brightness
• Weber-Fechner equations (WF)
– Describing detectable-difference threshold for small, steady-
state stimuli
• Naka-Rushton equations (NR)
– Modeling electrical responses to flashed stimuli
Fig. 2. Tone mapping curves for the logarithmic mapping (Weber-Fechner law),
the Naka-Rushton equation and the curve for the first stage of proposed method. 11 / 29
– First stage of proposed method
• For radiances below
– Weber-Fechnner‟s law
• For radiances above
– Naka-Rushton equation
0log ,
,
M
n
Mn n
s
k m s
c
(6)
where is chosen so as to ensure the continuity of at .0s cM
12 / 29
• TM results obtained using eqs. (1), (5) and (6)
– Left-hand image
» Overexposing the brightest regions
– Middle image
» Overall poor contrast
– Right-hand image
» Combining the best characteristics of the other two images
Fig. 3. Left: TM output using the Naka-Rushton equation. Middle: TM output using a logarithmic
mapping (Weber-Fechner‟s law.) Right: TM output of the first stage of our method, eq. (6). 13 / 29
Local contrast enhancement
Reproduction of the color and contrast perception
– Bertalmio et al.
• Proposing a perceptually-inspired color enhancement method
– Performing local contrast enhancement
– Approximating color constancy
• Defining an energy functional with three terms
– Local contrast, dispersion and attachment
0, , , ,wE I C I D I A I I (7)
E I
14 / 29
– Contrast term
» Measuring a weighted average contrast over the image
– Dispersion term
» Measuring the average departure of the values of the
pixels in from a given middle value
– Attachment to the original
» Measuring the average departure of the values of the
pixels in I from corresponding values in
I
0I
15 / 29
– Second step of proposed method
• Establish a close relationship between the functional in eq.
(7) and Retinex theory
– Final output of proposed method
• Image minimizing eq. (8)
2
2 2
0
1,
4E I w x y I x I y dxdy
I x dx I x I x dx
(8)
where we have chosen specific values for the constants
the Gaussian kernel gives locality to the contrast measure, and
is the average value of the original image .
, and w
0I
I
16 / 29
– First example
• Simple image with gray bands
• Preserving Mach band phenomena
Fig. 4. Top row: (a) original image, (b) after first stage of our TMO, (c) final output of
our TMO. Bottom row: corresponding profiles of scan lines.
17 / 29
– Second example
• Not changing the overall colors and contrast
• More detail on the stone arch and columns
Fig. 5. (a) Output of first stage (b) Output of second stage (c) Visualization of error of first stage (d)
Visualization of error of final output. Error computed with the metric of Aydin et al. [42].
(a) (b) (d)(c)
18 / 29
Implementation
Implementation details of proposed method
– Deal with each of the three color channel separately
– Background intensity
• Geometric average between the arithmetic mean and the
median of the radiance values
• Specifying the overall brightness of the final output
0.5 0.5median meanb
10b
19 / 29
– Semisaturation constant
• Center value in the 4-order operative range of the cones
– Inflexion point
• Switch from Weber-Fechner to Naka-Rushton
– Weber-fractions
• Taken from experimental data
• for red and green cones
• for blue cones
– Another parameter
10 10 10log log 0.37 4 log 1.9s b b
210M s
1001.85
k
1008.7
k
1.20.74, 10sn m
20 / 29
– Applying eq. (6) to the input HDR image
– Normalizing the result
– Looking for the image minimizing eq. (8)I
0 0
1
1
2
1 2
n
n
In
I x t I x R x
I xt
(9)
where ,
is the mean of and
with being a normalized 2D Gaussian kernel of effective radius .
0.2t
0 0I
, signn
n n
IR x w x y I x I y dy
(10)
w
21 / 29
– Effect of varying ρ and σ
Fig. 6. On the first row, effect of varying ρ. From left to right: ρ = −1, 0, 1, 2. Second row, effect
of varying σ. From left to right: σ = 50%, 20%, 4%, 2% of the number of rows or columns or
the image, whichever is greater.
22 / 29
Results and comparisons
First stage of proposed method with three gobal
TMOs
– Less contrast distortion
Fig. 7. Comparison with global TMOs. TM results (top row) and distortion maps (bottom row) for: (a)
Drago et al. [46] (b) Pattanaik et al. [8] (c) Reinhard and Devlin [9] (d) first stage of our TMO.
(a) (b) (d)(c)
23 / 29
Importance of having an objective quality metric
– Producing lots of amplification errors
– Ameliorating error
• Selecting an even higher value of σ
Fig. 8. (a) Output of the first stage. (b) Output of the second stage. (c) Visualization of the first
stage error. (d) Visualization of the final output error. Error computed with the metric of Aydin et
al. [42].
(a) (b) (d)(c)
24 / 29
Results for proposed method and for several
state of the art methods
Fig. 11. Results of our method for several images obtained from the MPI database, see
http://www.mpi-inf.mpg.de/resources/hdr/gallery.html for further details. 25 / 29
Fig. 12. TM results (rows 1 and 3) and distortion maps (rows 2 and 4) for these TMOs: (a) Drago et
al. [46] (b) Mantiuk et al. [21] (c) Pattanaik et al. [8] (d) Reinhard and Devlin [9] (e) the proposed
method. Original image from row 1 courtesy of ILM. Original image from row 3 courtesy of R.
Mantiuk.
(a) (b) (d)(c) (e)
26 / 29
Numerical results
Fig. 10. Error percentages computed with Aydin et al. [42], averaged over our image data set,
for several TMOs (see text.)
27 / 29
Computing statistics for the number of pixels
with error
– Slightly poorer performance with respect to the loss of
contrast
Fig. 9. Percentage of pixels with errors, for our image set and for the methods considered.
Horizontal axis: images in Figs. 11 and 12.
28 / 29
Conclusions and perspectives
Proposed method
– Tone mapping operator
• Inspired by basic perceptual properties of human vision
– Visual adaptation
– Local contrast enhancement
– Two parameters
• Control the overall brightness and the contrast of the final
image
29 / 29