1

Motivation Problem: Amateur photographers

take unappealing pictures (e.g. personal and business use)

Help users take better pictures with digital cameras

Solution: Improve composition during image acquisition Detect main subject in the picture Detect distracting background object

(avoidance of mergers)

Avo

id M

erg

er

Amateur

Professional

2

Mitigation of Mergers: Overview Goal: Identify background objects

merging with main subject In-focus background object Connected to main subject mask Large area relative to image size

Merger detection Color segmentation based on hue Identify distracting background object

based on distance to main subject and frequency content

Blur merging background objects to induce a sense of distance

Merging background objects: trees and bush

over right shoulder

3

Segmentation of Background Objects Hues above histogram average are dominant hues Background is a mixture of dominant hues Thresholds: average of two consecutive dominant hues

Background hues

Histogram of background hues and identified objects

Thresholds = {87, 151}

4

Merger Object Detection Define Frequency Inverse Distance Measure for

each disjoint background object Oi Decreases with distance (di) from main subject mask

Increases with high spatial frequency coefficients (wiH)

Merged object: Object with highest measure

form lExponentia ),(

formDivision ),(

),(

formLinear y)(x,)),(1(

i

i

i

Oy)(x,

),(

Oy)(x,

Oy)(x,

yxdHii

i

Hi

i

Hiii

ieyx

yxd

yx

yxd

5

Measure Selection Linear, division, and exponential forms to combine

High frequencies from residual in Gaussian pyramid Euclidean distance measured from main subject mask

Attribute Linear Divisional Exponential

Computational complexity Low High High

Merged object’s size Large Small Small

Divisional

Exponential

Exponential

Linear

6

Merger Mitigation ResultsBackground tree and bush merging with main subject

High frequency and inv. distance values for

background

Blurred tree and bush appear to be farther away

7

Per-pixel Implementation ComplexityProcess /Operation Multiply-

accumulatesCompares Memory

accesses

RGB to hue 3 6 4

Histogram and thresholding 1 2

RGB to intensity 2

Gaussian pyramid 9 4

Approx. inv. distance measure 2 1 2

Detect merged object 1 1

Gaussian pyramid reconstruction

9 1 5

TOTAL 27 11 15For comparison, JPEG compression takes ~60 operations/pixel

8

Merger Mitigation System Prototype

Merger mitigated picture

Binary main subject mask

Intensity Gaussian pyramid

Background segmentation

Inverse distance transform

Grayscale image

X

Detect merging object

Grayscale image

Reconstruct color pyramid

Color Gaussian pyramid

Transform coefficients

Original color image

9

Conclusion Contributions

Combine optical and digital image processing for improved image acquisition

Provide online feedback to amateur photographers Mitigation of mergers with background objects

Amenable to fixed-point implementation in digital still cameras Independent of scene setting or content

Deliverables Prototype development for digital still image acquisition Copies of MATLAB code, slides, and papers, available at

http://www.ece.utexas.edu/~bevans/projects/dsc/index.html

10

ION FOR IMPROVED ACQUISTIONSerene Banerjee and Brian L. Evans

Embedded Signal Processing LaboratoryWireless Networking and Communications Group

IN-CAMERA MERGER MITIGAT

11

IN DIGITAL STILL CAMERAS