OBJECTIVE

• Develop a formal mathematical framework for analysis and design of error diffusion algorithms for digital image halftoning– Model halftoning as two-dimensional

delta-sigma modulation– Derive objective measures for subjective

quality of edge sharpening and noise in halftoned images

• Applications:– Design of optimal error diffusion filters

with respect to subjective quality– Optimize quality of halftoned

oversampled images

Work supported by a grant from Hewlett-Packard Laboratories

LINEAR ANALYSIS

• Assume quantizer adds white noise uncorrelated with input

• Output given by

• Signal transfer function (STF) is flat, noise transfer function (NTF) is high-pass

• Circuit is equivalent in form to a noise-shaping feedback coder

quantizer

error �lter

H(z)

�

+x(n) y(n)

x0(n)

e(n)�

+

Equivalent circuit of error diffusion

Y (z) = X(z) +N(z)(1�H(z))

ERROR IMAGE

• Error image is highly correlated with input (Knox, 1992)

• Correlation is higher for larger error filters• Degree of image sharpening increases with

correlation• Suggests linear gain model for quantizer• Signal and noise paths modeled separately

(Ardalan and Paulos, 1987)

Original image Error image

LINEAR GAIN MODEL

+

error �lter

K

H(z)

�

+x(n) y(n)

q(n)

gain block

x0(n)

e(n)�

• Output given by

• K is measured empirically; varies with image and error filter

• Accounts for image sharpening• Noise treated separately (K = 1)

Equivalent circuit with linearized quantizer

Y (z) =K

1 + (K � 1)H(z)| {z }

STF

X(z) +1�H(z)

1 + (K � 1)H(z)| {z }

NTF

N(z)

NOISE TRANSFER FUNCTION

• Similar results for 1-D delta-sigma modulators

Predicted Measured

Predicted Measured

Top: Floyd-Steinberg. Bottom: Jarvis et al.

0

0.5

1

0

0.5

10

0.5

1

1.5

z1z2 0

0.5

1

0

0.5

10

0.5

1

1.5

z1z2

0

0.5

1

0

0.5

10

0.5

1

1.5

z1z20

0.5

1

0

0.5

10

0.5

1

1.5

z1z2

SIGNAL TRANSFER FUNCTION

• Linear gain model accounts for sharpening seen with large error filters

Floyd-Steinberg STF, K = 2.0

Jarvis et al. STF, K = 4.5

0

0.5

1

0

0.5

11

2

3

4

z1z2

0

0.5

1

0

0.5

1

2

4

6

8

10

z1z2

RESULTS OF LINEAR MODEL

• Sharpening is decoupled from noise• Effect of noise shaping can be quantified

Original image Jarvis halftoned

Gain model output Difference

HISTOGRAMS

• Narrow histogram at quantizer input leads to higher effective quantizer gain, K

• Quantizer error bounded by ± 0.5

Original image Quantizer input

Quantizer error Filter output

0 0.2 0.4 0.6 0.8 10

0.005

0.01

0.015

0.02

0.025

0.03

Graylevel

Pro

babi

lity

Floyd−SteinbergJarvis et al.

−0.2 0 0.2 0.4 0.6 0.8 1 1.20

0.01

0.02

0.03

0.04

0.05

0.06

0.07

GraylevelP

roba

bilit

y

Floyd−SteinbergJarvis et al.

−0.5 0 0.50

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

0.045

Graylevel

Pro

babi

lity

Floyd−SteinbergJarvis et al.

−0.5 0 0.50

0.005

0.01

0.015

0.02

0.025

Graylevel

Pro

babi

lity

SMALL ERROR FILTERS I

• Can small error filters be designed to sharpen as much as large filters?– Design large sharpening filter– Construct smaller filter whose frequency

response is closest to the large filter in a mean square sense (Wong, 1996):

where:

• Result: sharpening ability falls off linearly as number of filter taps decreases

• Degree of sharpening related to bandwidth of error filter

are the coefficients of the large and small error filters, respectivelyis a constant chosen to satisfy the gain constraint at DC

gn = hn + �

hn; gn

�

SMALL ERROR FILTERS II

• Sharpening correlated with bandwidth

Variation of quantizer gain with filter size

Variation of quantizer gain with filter bandwidth

MeasuredBest fit

2 4 6 8 10 122

2.2

2.4

2.6

2.8

3

3.2

3.4

3.6

3.8

Number of taps

Effe

ctiv

e qu

antiz

er g

ain

MeasuredBest fit

0.1 0.2 0.3 0.41.5

2

2.5

3

3.5

4

Equivalent bandwidth of error filter

Effe

ctiv

e qu

antiz

er g

ain

VISUAL SYSTEM MODEL

• Noise isolated by subtracting sharpened, noiseless image from halftoned image

• Weighted noise figure computed using visual system model

0 10 20 30 40 50 600

0.2

0.4

0.6

0.8

1

Frequency (cycles / degree)

Con

tras

t sen

sitiv

ity

0.2

0.4

0.6

0.8

30

210

60

240

90

270

120

300

150

330

180 0

Angular

Sensitivity of human visual system

Radial

CONCLUSION

• Summary– Error diffusion can be modeled as a

noise-shaping feedback coder, a form of two-dimensional delta-sigma modulation

– Quantizer can be modeled by a gain block plus additive noise

– Objective measures of subjective quality by decoupling edge sharpening and noise effects:

• Edge sharpening proportional to gain• Weight noise by perceptual SNR measure

• Future work– Design of optimal error diffusion filters

with respect to subjective quality using constrained nonlinear optimization

– Optimize algorithm complexity and subjective quality for halftoned oversampled images

• Combine error diffusion (sharpening) with interpolation (smoothing)