INTERPOLATED HALFTONING,REHALFTONING, AND

HALFTONE COMPRESSION

Prof. Brian L. Evansbevans@ece.utexas.edu

http://www.ece.utexas.edu/~bevans

Collaboration with Dr. Thomas D. Kite andMr. Niranjan Damera-Venkata

Laboratory for Image and Video EngineeringThe University of Texas at Austin

http://anchovy.ece.utexas.edu/

2

OUTLINE

n Introduction to halftoning

n Halftoning by error diffusion4Linear gain model4Modified error diffusion

n Interpolated halftoning

n Rehalftoning

n JBIG2 halftone compression

n Conclusions

3

INTRODUCTION: HALFTONING

n Was analog, now digital processing

n Wordlength reduction for images48-bit to 1-bit for grayscale424-bit RGB to 8-bit for color displays424-bit RGB to CMYK for color printers

n Applications4Printers4Digital copiers4Liquid crystal displays4Video cards

n Halftoning methods4Screening4Error diffusion4Direct binary search4Hybrid schemes

4

EXAMPLE HALFTONES

Original image

Clustered dot screen

Dispersed dot screen

Direct binary search

Floyd Steinberg

Modified Diffusion

5

FOURIER TRANSFORMS

Original image

Clustered dot screen

Dispersed dot screen

Direct binary search

Floyd Steinberg

Modified Diffusion

6

ERROR DIFFUSION

n 2-D delta-sigma modulatorn Noise shaping feedback coder

x0(i; j)

e(i; j)H(z)

�

x(i; j) y(i; j)+

�

+

7

16

5

16

3

16

1

16

P P P P P P P

P P P P P P P

P P P F

F F F F F F F

F F F F F F F

F F

n Error filter

n Raster scan order

P = PastF = Future

n Serpentine scan also used

7

ERROR DIFFUSION (cont.)

n Quantizer

y(i; j) =

�0; x0(i; j) < 0:51; x0(i; j) � 0:5

e(i; j) = y(i; j)� x0(i; j)

x0(i; j) = x(i; j)� h(i; j) � e(i; j)

Kn Knn0(i; j) + n(i; j)

n(i; j)

n0(i; j)

Ksx0(i; j) Ksx0(i; j)

n Governing equations

n Non-linearity difficult to analyzen Linearize quantizer [Kite, Evans, Bovik & Sculley 1997]

n Separate signal and noise paths[Ardalan & Paulos 1987]

Noise Path

Signal Path

8

LINEAR GAIN MODEL

n Quantization error correlated withinput [Knox 1992]

Floyd-Steinberg Jarvis, Judice & Ninke

Ks =E[jx0(i; j)j]

2E[x0(i; j)2]

n Least squares fit of quantizer inputto output defines signal gain

n Signal gain:n Noise gain:

Ks � constant

Kn = 1

9

GAIN MODEL PREDICTIONS

n Noise transfer function (NTF)

0

0.5

1

0

0.5

10

0.5

1

1.51.7

Frequency fx / f

NFrequency f

y / f

N

Mag

nitu

de

0

0.5

1

0

0.5

10

0.5

1

1.51.7

Frequency fx / f

NFrequency f

y / f

N

Mag

nitu

de

Predicted Measured

NTF = 1�H(z)

STF =Ks

1 + (Ks � 1)H(z)

0

0.5

1

0

0.5

11

2

3

4

Frequency fx / f

NFrequency f

y / f

N

Mag

nitu

de

0

0.5

1

0

0.5

1

2

4

6

8

Frequency fx / f

NFrequency f

y / f

N

Mag

nitu

de

Floyd-Steinberg Jarvis et al.

n Signal transfer function (STF)

10

Predicting Signal Gain Ks

n Predict Ks from error filter as:

2.017.1 −= RKs

∫

∫

−

−= π

π

π

π

ωωωωωω

ωωωω

21

2

21

2

21

21

2

21

),(),(

),(

ddHX

ddX

R

X(ω1,ω2) = Fourier Transform of input to error filter

H(ω1,ω2) = Fourier Transform of error filter

),(1),( 2121 ωωωω HX −=

But X(ω1,ω2) is the error spectrum, so

11

MODIFIED ERROR DIFFUSION

n Efficient method of adjustingsharpness [Eschbach & Knox 1991]

L

+

�

x(i; j)

H(z)e(i; j)

+

�

x00(i; j)x0(i; j)y(i; j)

x(i; j) G(z)+

�

H(z)e(i; j)

�

+

y(i; j)x00(i; j)

G(z) = 1 + L (1�H(z))

n Equivalent circuit: pre-filter

n L can be chosen to compensate forfrequency distortion

12

UNSHARPENED HALFTONES

n If then (flat)

n Accounts for frequency distortion

L =1�Ks

Ks

Original image Jarvis halftone

Unsharpened halftone Residual

STF = 1

13

INTERPOLATION

n Image resizing

n Different methods (increasing cost)4Nearest neighbor (NN)4Bilinear (BL)

n Nearest neighbor, bilinear methods4Low computational cost4Artifacts masked by quantization noise in

halftone4Correct blurring by using modified error

diffusion

Halftoning

Halftone

Interpolation

Original

F(z)I(z)

14

INTERPOLATION

n Design L for flat transfer functionusing linear gain model (L is constantfor a given interpolator)

n Compute transfer function ofinterpolation by M

n Compute signal transfer function

n Compute L to flatten the end-to-endtransfer function of the system

)()1(1

)))(1(1()(

zz

zHK

HLKF

s

s

−+−+=

−

−

−

−=−−

y

Mx

x

Mx

NN z

z

z

zI

1

1

1

1)(z

22

1

1

1

1)(

−

−

−

−=−−

y

Mx

x

Mx

BL z

z

z

zI z

15

INTERPOLATION RESULTS

00.2

0.40.6

0.81 0

0.20.4

0.60.8

1

0

0.2

0.4

0.6

0.8

1

1.2

Frequency fy / f

NFrequency fx / f

N

Mag

nitu

de

00.2

0.40.6

0.81 0

0.20.4

0.60.8

1

0

0.2

0.4

0.6

0.8

1

1.2

Frequency fy / f

NFrequency fx / f

N

Mag

nitu

de

Nearest neighbor ×2 Bilinear ×2

Transfer functionL = –0.0105

Transfer functionL = 0.340

16

REHALFTONING

n Halftone conversion, manipulationn Assume input and output are error

diffused halftones4Blurring corrected by using modified

error diffusion4Noise leakage masked by halftoning464 operations per pixel

n For a 512 x 512 image416 RISC MIPS40.4 s on a 167 MHz Ultra-2 workstation

Original

Halftoning Inversehalftoning Re-halftoning

Halftone

17

REHALFTONING (cont.)

n Halftone conversion, manipulation

n Error diffused halftones

n Fixed lowpass inverse halftoningfilter, compromise cut-off frequency4Noise leakage masked by halftoning4Correct blur by modified error diffusion4Computationally efficient

Halftoneimage

Original Inverse

halftone

G(z)

halftone

Ks(1 + L (1�H(z)))

1 + (Ks � 1)H(z)

Ks

1 + (Ks � 1)H(z)

Modi�ed

18

REHALFTONING (cont.)

n Use linear gain model to design L forflat response

n Use approximation for digitalfrequency:

n Inverse halftoning filter is a simpleseparable FIR filter

n L is computed to flatten the end toend transfer function of the system

n We need to know halftoning filtercoefficients for this scheme

n Improve halftoning results usingknowledge of type of halftone beingrehalftoned

ej!� 1 + j! � !

2=2

19

REHALFTONING RESULTS

00.2

0.40.6

0.81 0

0.20.4

0.60.8

1

0

0.2

0.4

0.6

0.8

1

1.2

Frequency fy / f

NFrequency fx / f

N

Mag

nitu

de

Original image Rehalftone

Signal transfer function

20

THE JBIG2 STANDARD

n Lossy/lossless coding of bi-level textand halftone data

Document

Symbolregion

decoder

Halftone region

decoder

Genericrefinement

Symbol dictionary

decoding

Halftonedictionarydecoding

Memory

Memory

n Scan vs. random mode

21

THE JBIG2 STANDARD (cont.)

n Bi-level text coding4Hard pattern matching (lossy)4Soft pattern matching (lossless or near

lossless) may be context based

n Halftone coding4Direct halftone compression4Context based halftone coding4 Inverse halftoning and compression of

grayscale image

n Implications4Printers, fax machines and scanners,

will need to decode JBIG2 bitstreams4Fast decoding may require dedicated

hardware and embedded software4Need for low complexity, low memory

solutions

22

PROBLEMS TO BE SOLVED

n JBIG2 compression of halftones4Compress halftone directly, using a

dictionary of patterns, or4Convert halftone to grayscale (inverse

halftoning) and compress grayscaleimage

n Efficient coding of halftone data4Fax machines4Digital archiving, scanning, and

copying

n Fast algorithms for JBIG2 codec

4 Interpolated halftoning in decoder

4Rehalftoning in codec

23

PROBLEMS TO BE SOLVED

n JBIG2 embedded decoders4Low memory requirements4Low computational complexity4High parallelism

n Inverse halftoning: a robust solutionfor lossy coding of halftones4Rendering device can use a different

halftoning scheme than encoder4Multiresolution halftone rendering

(archive browsing)4High halftone compression ratios (6:1)4Quality enhancement if the encoder

halftoning method is transmitted

n Low-cost embedded implementations

24

CONCLUSIONS

n Linear gain model of error diffusion4Validate accuracy of quantizer model4Link between filter gain and signal gain

n Rehalftoning and interpolation4Efficient algorithms4 Impact on emerging JBIG2 standard

n Web site for software and papers4 http://www.ece.utexas.edu/~bevans/

projects/inverseHalftoning.html