Upload
annabella-strickland
View
219
Download
2
Tags:
Embed Size (px)
Citation preview
Ph.D. Graduates: Niranjan Damera-Venkata Niranjan Damera-Venkata (HP Labs)(HP Labs) Thomas D. Kite Thomas D. Kite (Audio Precision)(Audio Precision)
Graduate Student: Vishal MongaVishal Monga
Support: HP LabsHP Labs National Science Foundation National Science Foundation
Embedded Signal Processing Laboratory
The University of Texas at Austin
Austin TX 78712-1084
http:://www.ece.utexas.edu/~bevans
Brian L. EvansBrian L. Evans
How to Make Printed and Displayed Images How to Make Printed and Displayed Images Have High Visual QualityHave High Visual Quality
UT Center for Perceptual Systems
2
OutlineOutline
• Introduction
• Grayscale halftoning for printing– Screening– Error diffusion– Direct binary search– Linear human visual system model
• Color halftoning for display– Optimal design– Linear human visual system model
• Conclusion
UT Center for Perceptual Systems
3
Need for Digital Image HalftoningNeed for Digital Image Halftoning
• Many devices incapable of reproducing grayscale (e.g. eight bits/pixel or gray levels from 0 to 255)– Laser and inkjet printers
– Facsimile machines
– Low-cost liquid crystal displays
• Grayscale imagery binarized for these devices
• Halftoning tries to reproduce full range of gray while preserving quality and spatial resolution– Screening methods are fast and simple
– Error diffusion gives better results on some media
UT Center for Perceptual Systems
4
Digital Halftoning MethodsDigital Halftoning Methods
Clutered Dot ScreeningAM Halftoning
Blue-noise MaskFM Halftoning 1993
Dispersed Dot ScreeningFM Halftoning
Green-noise HalftoningAM-FM Halftoning 1992
Error DiffusionFM Halftoning 1975
Direct Binary SearchFM Halftoning 1992
UT Center for Perceptual Systems
5
Screening (Masking) MethodsScreening (Masking) Methods
• Periodic array of thresholds smaller than image– Spatial resampling leads to aliasing (gridding effect)
– Clustered dot screening produces a coarse image that is more resistant to printer defects such as ink spread
– Dispersed dot screening has higher spatial resolution
– Blue noise masking uses large array of thresholds
UT Center for Perceptual Systems
6
current pixel
weights
3/16
7/16
5/16 1/16
+ _
_+
e(m)
b(m)x(m)
difference threshold
compute error
shape error
u(m)
)(mh
Error Diffusion
Spectrum
Grayscale Error DiffusionGrayscale Error Diffusion
• Shape quantization noise into high frequencies
• Design of error filter key to quality
• Not a screening technique
2-D sigma-delta modulation
UT Center for Perceptual Systems
7
1 sampledelay
Inputwords
To output device
4 2
2 2
Assume input = 1001 constant
Time Input Feedback Sum Output 1 1001 00 1001 10 2 1001 01 1010 10 3 1001 10 1011 10 4 1001 11 1100 10
Periodic
Average output = 1/4(10+10+10+11)=10014-bit resolution at DC!
Added noise
f
12 dB(2 bits)
If signal is in this band, you are better off
Simple Noise Shaping ExampleSimple Noise Shaping Example
• Two-bit output device and four-bit input words– Going from 4 bits down to 2 increases noise by ~ 12 dB
– Shaping eliminates noise at DC at expense of increased noise at high frequency.
UT Center for Perceptual Systems
8
Modeling Grayscale Error DiffusionModeling Grayscale Error Diffusion
• Sharpening caused by a correlated error image [Knox, 1992]
Floyd-Steinberg
Jarvis
Error images Halftones
UT Center for Perceptual Systems
9
Modeling Grayscale Error DiffusionModeling Grayscale Error Diffusion
• Apply sigma-delta modulation analysis to 2-D– Linear gain model for quantizer in 1-D [Ardalan and Paulos, 1988]
– Linear gain model for grayscale image [Kite, Evans, Bovik, 2000]
– Signal transfer function (STF), noise transfer function (NTF)
Ks
us(m)
Signal Path
Ks us(m)
un(m)
+
n(m)
Kn un(m) + n(m)
Noise Path
)(1)(
zz
zH
N
BNTF n
zz
z
HK
K
X
BSTF
s
ss
11)(
Q(.)u(m) b(m) {
1 – H(z) is highpass so H(z) is lowpass
UT Center for Perceptual Systems
10
Problems with Error DiffusionProblems with Error Diffusion
• Objectionable artifacts– Scan order affects results– “Worminess” visible in constant graylevel areas
• Image sharpening– Error filters due to [Jarvis, Judice & Ninke, 1976] and [Stucki, 1980]
reduce worminess and sharpen edges– Sharpening not always desirable: may be adjustable by
prefiltering based on linear gain model [Kite, Evans, Bovik, 2000]
• Computational complexity– Larger error filters require more operations per pixel– Push towards simple schemes for fast printing
UT Center for Perceptual Systems
11
Direct Binary SearchDirect Binary Search
• Minimize mean-squared error between lowpass filtered versions of grayscale and halftone images– Lowpass filter is based a linear model of the human visual
system (contrast sensitivity function)
– Iterative method that gives a practical upper bound on the achievable quality for a halftone of an original image
• Each iteration visits every pixel– At each pixel, consider changing state of the pixel (toggle)
or swapping it with each of its 8 nearest neighbors that differ in state from it
– Terminate when if no pixels are changed in an iteration
UT Center for Perceptual Systems
12
Direct Binary SearchDirect Binary Search
• Advantages– Significantly improved
image quality over screening and error diffusion methods
– Quality of final solution is relatively insensitive to the choice of starting point (initial halftone)
– Has application in off-line design of threshold arrays for screening methods
• Disadvantages– Computational cost and
memory usage is very high in comparison to error diffusion and screening methods
– Increase complexity makes it unsuitable for real-time applications such as printing
UT Center for Perceptual Systems
13
Contrast Sensitivity FunctionContrast Sensitivity Function
• Contrast needed at a particular spatial frequency for visibility– Angular dependence modeled
with a cosine function
– Modify it at low frequency to be lowpass, which gives better correlation with psychovisual results
– Useful in image quality metrics for optimization and performance evaluation
UT Center for Perceptual Systems
14
Color Halftoning by Error Diffusion for DisplayColor Halftoning by Error Diffusion for Display
• Input image has a vector of values (e.g. Red-Green-Blue) at each pixel– Error filter has matrix-valued coefficients
– Algorithm for adapting matrix coefficients[Akarun, Yardimci, Cetin 1997]
vectormatrix
kmekhmtk
+ _
_+
e(m)
b(m)x(m)
difference threshold
compute error
shape error
u(m)
)(mh
t(m)
UT Center for Perceptual Systems
15
Matrix Gain Model for the QuantizerMatrix Gain Model for the Quantizer
• Replace scalar gain with a matrix
– Noise uncorrelated with signal component of quantizer input
– Convolution becomes matrix–vector multiplication in frequency domain
12minarg
uubu
A
CCmuAmbK
Es
IK
n
zNzHIzB
n
zXIKzHIKzB1
s
Noise component of output
Signal component of output
u(m) quantizer inputb(m) quantizer output
UT Center for Perceptual Systems
16
Optimum Color Noise ShapingOptimum Color Noise Shaping
• Vector color error diffusion halftone model– We use the matrix gain model [Damera-Venkata and Evans, 2001]– Predicts signal frequency distortion– Predicts shaped color halftone noise
• Visibility of halftone noise depends on– Model predicting noise shaping– Human visual system model (assume linear shift-invariant)
• Formulation of design problem– Given HVS model and matrix gain model find the color
error filter that minimizes average visible noise power subject to certain diffusion constraints
UT Center for Perceptual Systems
17
Linear Color Vision ModelLinear Color Vision Model
• Pattern-Color separable model [Poirson and Wandell, 1993] – Forms the basis for S-CIELab [Zhang and Wandell, 1996]
– Pixel-based color transformation
B-W
R-G
B-Y
Opponentrepresentation
Spatialfiltering
E
UT Center for Perceptual Systems
18
Linear Color Vision ModelLinear Color Vision Model
• Undo gamma correction on RGB image
• Color separation– Measure power spectral distribution of RGB phosphor excitations
– Measure absorption rates of long, medium, short (LMS) cones
– Device dependent transformation C from RGB to LMS space
– Transform LMS to opponent representation using O
– Color separation may be expressed as T = OC
• Spatial filtering is incorporated using matrix filter
• Linear color vision model
Tmdmv
)( where )(md
is a diagonal matrix
)(md
UT Center for Perceptual Systems
19
Original Image
Sample Images and optimumcoefficients for sRGB monitor
available at:http://signal.ece.utexas.edu/~damera/col-vec.html
UT Center for Perceptual Systems
21
ConclusionsConclusions
• Design of “optimal” color noise shaping filters– We use the matrix gain model [Damera-Venkata and Evans, 2001]
• Predicts shaped color halftone noise
– HVS could be modeled as a general LSI system
– Solve for best error filter that minimizes visually weighted average color halftone noise energy
• Future work– Above optimal solution does not guarantee “optimal” dot
distributions• Tone dependent error filters for optimal dot distributions
– Improve numerical stability of descent procedure
UT Center for Perceptual Systems
22
Designing the Error FilterDesigning the Error Filter
• Eliminate linear distortion filtering before error diffusion
• Optimize error filter h(m) for noise shaping
Subject to diffusion constraints
where
22min mnmhImvmb
EE n
11mhm
mv
linear model of human visual system
* matrix-valued convolution
Backup Slides
UT Center for Perceptual Systems
23
Generalized Optimum SolutionGeneralized Optimum Solution
• Differentiate scalar objective function for visual noise shaping w/r to matrix-valued coefficients
• Write norm as trace and differentiate trace usingidentities from linear algebra
i0ih
mb
d
Ed n
2
xxx Tr
A
X
XA
d
Trd
BA
X
BXA
d
Trd
BXABXA
X
BXAX
d
Trd
ABBA
TrTr
Backup Slides
UT Center for Perceptual Systems
24
Generalized Optimum Solution (cont.)Generalized Optimum Solution (cont.)
• Differentiating and using linearity of expectation operator give a generalization of the Yule-Walker equations
where
• Assuming white noise injection
)()()()()()( qpsirphqvsvkirkv nnp q s
ank
)()()( mnmvma
kkmnmnkrnn )( )()( E
kvkmnmakran )( )()( E
• Solve using gradient descent with projection onto constraint set
Backup Slides