Communication , Control, and Signal Processing edited by Ε . Ankan Elsevier Science Publishers Β . \' . , 1990

ADVANCEDCCDIMAGESENSORPREPROCESSOR

M.I.Loupis 1, K.Z.Dimopoulos2, B.Dimitriadis 1, J.Avaritsiotis2

1 HITEC S.A. , R + D Dept. , 18 Posidonos Ave. , GR-116 35 Athens, Greece

1213

2 National Technical University of Athens, School of Electrical Engineering, Computer Science Dept. , Microelectronics Group, GR-157 73 Zografou, Greece

ABSTRACT An advanced CCD sensor camera is presented, consisting of three hybrid in­tegrated circuits. The hybrids implement respectively an A/ D and signal condition­ing module , a memory buffer module and a preprocessing module. The preprocessing faci lities currently supported are, an AOI processor, a histogram ex­tractor and equalizer, a threshold processor and a 3χ3 correlation processor.

INTRODUCτiON

Hardware implementations for real-time image processing lack advanced

imag~ capturing and processing capabilities and especially the means of interfacing

these two. Existing CCD cameras produce analogue video outputs which need sig­

nal conditioning and digitization before effective interfacing with the digital image

processing system. Thus the well known analogue world noise interference and sig­

nal distortions are eminent and they can seriously corrupt the image, well before the

digitization phase.

Consequently a fast conversion of the analogue image near the information

source is highly desirable. Moreover, because of the computational limitations a

compact image processing system might impose, the incorporation of image

preprocessing elements at this very early stage of image capturing is considered to

be necessary. The situation is especially true in industrial quality control and inspec­

tion systems, as well as in machine vision systems used in automated manufactur­

ing environments.

ln our design, the block diagram of which is shown in Fig.1, the .CCD image

sensor, the A/ D converter, some preprocessing elements and the memory buffer

have been integrated in three hybrid microcircuits. ln this way a powertul three chip

image sensor module can be realized, with all the advantageous properties inherent

to ICs, namely speed, reliability , noise immunity, robustness, design flexibility and

1214

space as well as power economy.

ΤΟ DISPLA Y

...•..•.••... ι:t.Y.Iit.~.:ιe> .. . ~. 1

Fig.1 Advanced CCD image sensor preprocessor

CCD SENSOR HYBRID

ln more detail the CCD sensor employed as the kernel of our design is the

Philips ΝΧΑ 1111, a very flexible and advanced IC compatible to the CCIR 625 line

τν standard. ΝΧΑ 1111 produces two 288-line interlaced fields with 790 active pixels

per line. The image area is 6.4x4.8mm and. the effective number of pixels is

780χ576. ΝΧΑ 111 outputs three analogue data streams, each of which is clocked

by a 5 MHz clock. The combined analogue output has therefore a pixel peak rate of

15 MHz, although the frequency content is lower. The S/ N ratio at standard il­

lumination is 50dB. Additional ICs are requ ired to drive the sensor, namely the

Universal Sync Generator SAA 1101 (η which provides all standard pulses requ ired

for the camera operation including the 15 MHz clock, the Pulse-Pattern Generator

PCF3025 which generates all the pulses to operate the sensor, including charge

transport in the image and storage regions and information read-out. Finally the

necessary circuitry includes standard driver ICs to amplify the PCF3025 signals to 8

Volts level, which is acceptable by the sensor. The ΝΧΑ 1111 is a new version of

1215

ΝΧΑ 1011 which has been used fo r our first prototype. Further detaiis concerning

the sensor and drivers specifications are not described here for the sake of brevity.

The kernel hybrid also contains the A/ D converter. One of the most exciting

options available with ΝΧΑ 1111 was to use three rather than one A/D converters,

that is one for each output channel. The advantages are that the analogue com­

biner circuitry is no longer necessary and that the signal conditioning circuitry be­

comes much simpler. Α major drawback of this approach is that the resulting quan­

tization missalignment introduces vertical stripes along each frame, which need to

be removed at a post-processing stage [1]. ln the design presented here the alter­

native single flash A/ D approach has been adopted.

This approach, however, requires additional analogue circuitry in order to

combine the three data streams into one prior to digitization and produces triple

pixel frequency (15 MHz), but it avoids the complexity of two more A/Ds with their

re lated support circuitry , quantization distortion overheads and the digital combin­

ing circuitry. The A/ D used in our design is the Analog Devices AD9012, which is a

very fast - 100 MSPS- 8-bit A/ D with πι compatible output latches. Linearity is 0.5

LSB and Signal-to-Noise Ratio is typically 47 dB. comparable with the maximum

CCD chip specification. The conversion rate of 100 MHz, together with the 15ps jit­

ter eliminate the need for an external track-and-hold and ensure the most accurate

digitization of the analogue CCD output signal. The 50db S/N ratio of the sensor

corresponds to approximately 300: 1 or a little over 8 bits. Nevertheless signal noise

and incompatibility to the succeeding processing stages would render a 9-bit A/ D

more of a problem than of a solution.

PREPROCESSING HYBRID

The preprocessing hybrid IC contains an Area-Of-lnterest processor, a his­

togram extractor and equalizer, a threshold processor, a 3χ3 correlation processor

and it can further be extended to incorporate additional preprocessing modules,

with in the physical space limitations. The ΑΟΙ processor, when enabled, allows for

only the interesting parts of the image to be processed, as defined by the critical

time limitations of the particular application (Fig.2). The serial and highly regular na­

ture of the output data from the previous stage, offers a straightforward implemen­

tation using essentially two binary down-counters for each coordinate. Α closed­

loop control circuit between the ΑΟ Ι and the main image processing stage permits

reprogramming of the ΑΟΙ, via the control bus.

1216

NEW F" RA I"'E OUT

VERT . RETR ACE OUT

HORZ . RETR ACE OUT

DATA VALIO STRO BE OUT

Ι MRGER SYSTEM

ΟΑΤΑ BU S OUT

AODRESS BUS OU T

AOI BUS

NF

νR

HR

•sτ

DYD-DY7

OYD-OT <7

ο

' ' s ε τ

5 J z ε

X _OF F SET j X....S IZE

ORIG I NRL IMRGE

Fig.2 ΑΟΙ principle of operation

The real-time histogram extractor is implemented using a very fast memory buffer and a 24-bit digital counter (Fig.3) . Each time a new pixel value arrives, within the AOI defined previously, is interpreted by the memory buffer control logic as an address rather than data. The data corresponding to the specific memory location are clocked into the counter and incremented by one. At the end of a frame the his­togram buffer contains the gray level frequencies, if all memory locations are ini­tially reset to naught. This arrangement offers the additional advantage of averaging the histogram results over a series of 64 consecutive frames.

The histogram equalizer is linked to the threshold processor which is a set of 16 look-up tables. By exploiting the dead time between two consecutive frames, a programmable logic uses the histogram extractor results, to rearrange the con­figuration of the look-up tables' configuration and to alter the image thresholds (upper and lower) . Programming is again effectuated via the control and timing bus.

Α final correlator module is connected to the output of the histogram proces­sor. The correlator produces a 2-dimensional convolution of the image using a 3χ3 square template. The implementation of the convolution process involves loading the elements of the convolution kernel into the coefficient registers and passing the entire image through the processor. The output image has a delay of two lines com­pared the input image. lt should be noted that the kernel must be loaded through the control bus before enabling the convolution process. lf the correlator is disabled then the entire line of the frame passes through without any processing and hence without any delay as well .

1217

NE W FRAME OUT NF NF

VERT. RETRACE OU T -"" VR HORZ . R E TRACE OUT HR HR

ΟΑΗ~ VAL I D ST ROB E OUT • ST • ST

Ι ΑΙ Ο-ΑΙ 17 MRGER ΟΙΟ-017

SYSTEM ~ ΑΗ0-ΑΗ7

ι Β 256 • 2 4-ΒΙ Τ H1AGE PRO CESSING

u Η Ι S T OGRA/'1 RA M S Y ST EM D FH A BUS OU T -=α-ΟΙ? F AOOR

AODR E SS BUS OU T rα- 07 F Ε • '"" '---- ΟΑΤΑ

HD0- HD23 1 Η00-Η023

~===t: LD "" ' '° CLRn CONT RO L L OC I C CK 24-ΒΙτ TRI STιH E UNtτ RE CIST ER /COUNT ER

ΟΕ

HR / •W

Fig .3 Α real -time histogram processor in block schematics

MEMORY BUFFER HYBRID

The video memory buffer hybrid IC is effectively a dual-ported video memory

buffer. The utilization of the highly regular, serial data output stream properties al·

lows the implementation of such a buffer with simple SRAM memory chips. Data

are divided into an odd and an even data stream, destined to an odd and an even

data bank respectively . The buffer has a two non-interlaced frame storing capacity ,

that is 1 ΜΒ . The 18-bit addresses, required for accessing the 256 ΚΒ video

memory banks, are generated by an 18-bit fast binary counter which is clocked by

the pixel frequency. Therefore the memories maximum access time should not

exceed 67 ns. ln fact the CMOS SRAMS used in our design had an access time of

45ns. Other signals requi red are the vertical and horizontal retrace, which reset and

inhibit the counter respectively , during inactive periods [2].

The three hybrid Ι Cs are communicating with two synchronized buses, as

shown in fig .1, i.e. the video bus and a timing and control bus. The latter bus carries

the timing signals that synchronize the modules and the data and address lines

from the host processor for programming purposes.

IMPLEMENTATION ISSUES

The hybrids were fabricated using a two-layer palladium-silver metallization

1218

scheme. Resistors were screen-printed and all the other add-on components were

reflow-soldered SMDs. Resistors were trimmed by abrasion after they were over­

glared for stabil ity.

There was a question of whether to use a one- or two-dimensional sensor.

The one-dimensional sensor requires a high precision micromechanical positioning

mechanism, which will effectively replicate the orthogonal view window. Although

such a task has been accomplished by using stepper motors on PCB [3] , it would

have been quite a tedious effort to mount micromechanic devices on a thick film

hybrid. On the other hand, the two-dimensional sensor has inherent drawbacks

despite its simplicity. The qualitative exploitation of an area sensor is circa 5 times

smaller than that of a linear sensor. This together with the advent of advanced linear

sensors and micromechanical positioning mechanisms provide the initiative for a fu­

ture project.

Areas of application for this design, if it was to be commercial ly produced,

include consumer electronics, medical image processing, office technology, optical

character recogn ition instruments, industrial qualitative control using scanners and

robotic vision. Furthermore tradit ional frame-grabbing devices will be much

simplified since a great percentage of their functions will be implemented by the

CCD image preprocessing module.

CONCLUSION

An advanced CCD camera design was presented incorporaίing a multiple

preprocessing stage, which make it suitable for time demanding machine vision

systems in noisy manufacturing environments. The camera consists of three hybrid

modules, implementing an A/ D, a memory buffer and a processing module. Next

steps scheduled are the use of a linear sensor with a micromechanic positioning

mechanism and the migration to the colour domain.

REFERENCES

[1] A.Hadjiaslanis, B.Dimitriadis, "lnterfacing of the PHILIPS CCD imaging sen­sor to a high performance analog to digital conversion system for the pur­poses of digital image processing", HITEC lnternal Report, December 1987

[2] A.Hadjiaslanis, B.Dimitriadis, "Practical design of interface functionalities", ΗΠΕC lnternal Report, December 1988

[3] T.G.H.-R.Leemann, ''Hochauflosende digitalkamera fϋr die erfassung von quasistatischer bildinformation", PhD Thesis, ΕΤΗ ZUrich , 1986

Communication, Control, and Signal Processing

Proceedings of the 1990 Bilkent International Conference on Ν ew Trends in Communication, Control, and Signal Processing held at Bilkent University, Ankara, Turkey, 2-5 J uly 1990

Volume ΙΙ

Edited by

Erdal Ankan

Electrical and Electronics Engineering Department Bilkent University · Ankara, Thrkey

Elsevier

Amsterdam-Oxford-New York-Tokyo 1990

ELSEVIER SCIENCE PUBLISHERS B.V. Sara Burgerhartstraat 25 Ρ.Ο . ΒΟΧ 211, 1000 ΑΕ Amsterdam, The Netherlands

Distributors for the United States and Canada:

ELSEVIER SCIENCE PUBLISHING COMPANY, INC. 655 Α venue of the Americas New York , Ν.Υ. 10010 U.S.A.

ISBN: 0-444-88762-8

@Elsevier Science Publishers B.V., 1990

All rights reserved. Ν ο part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior written permission of the Publisher, Elsevier Science Publishers B.V., Ρ.Ο. ΒΟΧ 211, 1000 ΑΕ Amsterdam, The Netherlands.

Special regulations for readers in the U .S.A. - This publication has been registered with the copyright Clearance Center Inc. (CCC) , Salem, Massachusetts. Information can be obtained from the CCC about the conditions under which photocopies of parts of th the copyright owner, Elsevier Science Publishers B.V. , unless otherwise specified .

Νο responsibility is assumed by the Publisher for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use of operation of any methods, products, instructions or ideas contained in the material herein .

Printed by METEKSAN - Ankara, Turkey