Abstract

In this paper, a new technique is presented to enhance

the blurred images obtained from Fluorescein Angiography (FA) of the retina. One of the main steps in inspecting the eye (especially the deeper image of retina) is to look into the eye using a slit-lamp apparatus that shines a monochromatic light on to the retinal surface and captures the reflection in the camera as the retinal image. When further probing is required, such imaging is preceded by injecting a specialized dye in the eye blood vessels. This dye shines out more prominently in the imaging system and reveals the temporal as well as special behavior of the blood vessels, which, in turn, is useful in the diagnosis process. While most of the cases, the image produced is quite clean and easily used by the ophthalmologists, there are still many cases in which these images come out to be very blurred due to the disease in the eye such as cataract etc… in such cases, having an enhanced image can enable the doctors to start the appropriate treatment for the underlying disease. The proposed technique utilizes the Blind Deconvolution approach using Maximum Likelihood Estimation approach. Further post-processing steps have been proposed as well to locate the Macula in the image which is the zero-center of the image formed on the retina. The post-processing steps include thresholding, Region Growing, and morphological operations.

Introduction Fluorescein angiography is an extremely valuable

clinical test that provides information about the circulatory system of the ocular fundus (the back of the eye) not attainable with a routine examination. Fluorescein is an orange water-soluble dye that when injected intravenously in to the blood stream, it largely remains intravascular and circulates in the blood. Fluorescein angiography involves photographic surveillance of the passage of Fluorescein through the retinal and choroidal (middle layer of the eye) circulation following intravenous injection. The different, appearance of Fluorescein, both temporally as well as spatially, and the classification of the fundus diseases

render angiography a dynamic, cinematographic and deductive diagnostic method. Therefore, the knowledge for interpreting fundus Fluorescein angiograms allows an ophthalmologist specialized in ocular fundus diseases to follow a systematic, orderly and logical line of reasoning that leads to a proper diagnosis.

Fluorescein is the property of certain molecules to emit

light of a longer wavelength when stimulated by light of a shorter wavelength. The excitation peak for Fluorescein is about 490 nm (blue part of the spectrum) and represents the maximum absorption of light energy by Fluorescein. Molecules stimulated by this wavelength will be excited to a higher energy level and will emit light of a longer wavelength at about 530 nm (green part of the spectrum). As such, first a blue light is shun over the eye and enters the eye and excites the Fluorescein molecules in the retinal and choroidal circulation, and then by using a yellow-green filter lens, the image of the illuminated part is obtained. Figure 1 shows a typical experimental arrangement as a ray diagram and typically obtained images. There has been an overwhelmingly extensive application of various techniques from the Image Processing domain to the retinal image enhancement and classification. Researchers have applied almost every known technique during the past five years to improve and automate the ophthalmologic inspection procedures. However, the same is in its early stages in case of FA images.

Figure 1. Experimental setup for the FA imaging system.

Blind Restoration of Fluorescein Angiography Images

Uvais Qidwai

Computer Science & Engineering Department Qatar University

P.O. Box 2713, Doha, Qatar. uqidwai@qu.edu.qa

Umair Qidwai Department of Ophthalmology

Isra Postgraduate institute of ophthalmology Karachi, Pakistan.

2010 Digital Image Computing: Techniques and Applications

978-0-7695-4271-3/10 $26.00 © 2010 IEEE

DOI 10.1109/DICTA.2010.35

146

2010 Digital Image Computing: Techniques and Applications

978-0-7695-4271-3/10 $26.00 © 2010 IEEE

DOI 10.1109/DICTA.2010.35

146

While most of the FA images are quite clear for the ophthalmologists, there are still many cases where the result is occluded due to the eye-diseases such as cataract. As such, if FA is recommended to a patient, it comes out to be highly inconclusive leading the doctor to recommend another test (implying another dose of the dye) or would start the treatment blindly based on his/her experience. Perhaps, one of the latest works on this subject is related to the application of Fuzzy C-mean clustering algorithm to isolate the vessels from the background images as well as the disease-related spots (exudates) [1]. The results are quite good in terms of automatic classification but works well with already very clean images. Adaptive Equalization of the gray-scale has been used extensively as well for improving the image of vessels and optical disc [2, 3]. In addition to these methods, all possible morphological operations from the Image processing domain have been used for post-processing the images. These include Erosion, Dilation, opening, and closing operators [3]. A very comprehensive survey of the techniques used in enhancing the retinal images has been given in [5]. A more classical approach is also found using Bayesian classification principles in FA images [6, 7]. However, one marked commonality is the use of very clean images as the starting point. Most of the reported techniques are focused on automatic detection of the various areas of interest in the FA images and hence they assume that the image is quite clean and nothing is occluded. The proposed solution attempts to utilize the basic examination data from the ophthalmologists’ standard clinical procedures and improves the quality of the same to an extent that the doctors can make a better informed decision without having to resort to more expensive and invasive techniques.

The Deconvolution Model The retinal images are stored as matrices in (m, n) rectangular coordinates and are usually colored in the RGB color space hence producing an N�M�3 image. However, the situation is easier in case of FFA images which are essentially gray scale images. Hence the image size is restricted to the 2D only. Thus, the retinal image of interest f (m, n) is a matrix of M�N order. The general Retinal imaging system in Figure 1 can be represented more formally as a standard block diagram representation as shown in Figure 2. Hence, the observed image g (m, n) is given by:

���� �� � ��� �� ���� �� � ��� �� (1)

Where h��� �� represents the degradation model and � represents the 2-D convolution operation. The additive

Figure 2. Block diagram for the image deconvolution model.

term v��� �� represents the noise added in the degraded image, further adding to the distortion of image. The underlying blind deconvolution problem can thus be formulated as follows: Given a degraded image g(m, n) (Figure 6), with the characteristics of both h(m, n) and v(m, n) being unknown, the problem is to recover f(m, n) by deconvolving the degraded image g(m, n).

The Maximum Likelihood Deconvolution Maximum likelihood deconvolution is an improved subset of Iterative Constrained optimization algorithms [8]. The iteration is designed based upon a probability model. The mathematics of this algorithm is based upon the behavior of quantum photon emissions and diffraction. Among all known approaches, the Maximum Likelihood approach has proved to provide the best quality images. Usually, in MLE, it is known that the function H belongs to a certain family of distributions{�h(·|�), � � ��}, called the parametric model, so that H = h(m, n|�0). The value �0 is unknown and is referred to as the “true value” of the parameter. It is desirable to find some �� (the estimator) which would be as close to the true value �0 as possible. To use the method of maximum likelihood, one first specifies the joint density function for all observations. For iid sample this joint density function will be

���� ��� � � ����� � ������ � ������ � ������ (2)

Where x1 to xk represent the k-sized window of data extracted and worked upon by the iterative algorithm. Then, the extended density can be considered as a function of the parameter �. This extended density is the likelihood function of the parameter [9]:

������� ��� � � ��� � ���� ��� � � ����� � � ���������� (3)

In practice it is often more convenient to work with the logarithm of the likelihood function, ln �, called the log-likelihood, or its scaled version, called the average log-likelihood:

!"������� ��� � � ��� � # !"������� �$ ��

� !"%�

��� (4)

Where �$ is the expected log-likelihood of a single observation in the model. The method of maximum

147147

likelihood estimates � by finding a value of �i that maximizes �$�����:

��&'( �)*�"�)�

� � +"�$������ ��� � � ��� (5)

Blind Deconvolution Blind deconvolution is a subset of Iterative Constrained algorithms which produces an estimate of h(m, n) concurrently with f(m, n). It does not need the PSF h(m, n) to be measured. Probably one of the first accounts was given in [10]. In the context of Image deconvolution, it is considered as a blind deconvolution technique only if the algorithm is producing the PSF from information within the data set g(m, n). This is done by first assuming a hi(m, n), then estimating which f(m, n) could have caused g(m, n). This calculation is followed by estimating which h(m, n) could have caused g(m, n) from the estimated f(m, n), and then these steps are repeated again and again.

Proposed Algorithm For a black-box problem, as the one in hand, the inverse approach to getting the actual image from its blurred measurement is called blind deconvolution. The idea is to initialize an iterative algorithm with a pre-selected structure (state-space or transfer function) with random or flat initialization. Using this assumed structure, a new image estimate is then obtained. A tuning parameter is then adjusted based on the error between this estimate and the actual image. Since nothing is known for the true image as well, some statistical assumptions are needed to decide on the correction. Usually, some form of covariance matrix is used to adjust the underlying model and then the iterative procedure is continued until a suitable convergence is reached [8]. The general algorithm proposed in this work is shown in Figure 3.

The algorithm is initialized by assigning the actual blurred image as first image estimate, fi, and the first initial guess of the blurring function as h which is assumed to be a function of a defining set of parameters, �, and is defined within the same sample space (m, n) as that of the image under study. As shown in Figure 3, there are two iteration processes. The outer loop, Loop t, which runs up to the circular block B, is the repetition loop that repeats the same deconvolution process for each new estimated image as well as the blurring function. Once the inner deconvolution loop (Loop m, n) finishes, the resulting

image f̂ is quite improved in terms of its visual appearance. For manual inspection purposes this image is sufficiently improved for most of the ophthalmologic parameters.

Figure 3. The presented blind deconvolution algorithm.

At this point, an error parameter �i is calculated which also serves as the loop terminal marker against a preset threshold, �. Inside this outer loop, and inner loop (Loop (m, n)) executes to utilize the initial guesses and estimate new image using the deconvolution algorithm listed in Equations (1) through (5). At the end of this loop, i.e. the End of loop indicator text at Loop(m, n) terminal block A, a new estimate of the deconvolved image as well as the blurring function is available which are replaced for the previous initial guesses and are ready for deconvolution algorithm’s application again. On the other hand, if the error marker has reached the preset threshold �, the outer loop also terminates at the End of Loop marker text next to block B. At this point, a post processing step is used in order to identify/classify other most important component in the FA image, i.e., the central point of retina, called Macula. The post-processing steps are related to the Region of Interest (ROI) processing and are outlined as follows: Macula detection: 1. Locate the center of the deconvolved image at (m0,

n0). 2. Start expanding outwards by selecting m0±� rows and

n0±��columns. � can be selected in an ad hoc manner to cover enough areas where making a gray vs. light-

148148

gray dselectecolum

3. This sthreshsuitablbetwebackgr

4. FurtheMorphsucces

5. The Renhanccorres

Since the mathematithe algoritcan be deentropy odefined as given by

Where pi rbins from histogram by Equaticomplete additional not very rethe initial decreasingthe maculcorrespondthe regionentropy stain image rother artifapoint that iterations. Figure 4 represents two knee scales imagarea from timages bel30% threshseen, initiathat growsenhancememorphologinto a conn

distinction caned to be app

mn lengths averaselected area

holding at 30%le for low cen the actual round. er removal hological opessively [11]

ROI (macula) iscement is doneponding to the

region growical measure isthm at the appetected. The mf the binary the degree of u

�

K

iE

represents the the histogram

is first calculaion (6). The macula is in artifacts are in

eliable since inapproximatio

g in entropy aftla in the regds to the reductn of view. Hoarts to increasereconstruction acts as the reghas been usedFigure 4 showrepresents thethe entropy vapoints are shoges in this figuthe deconvolvelow these grayholded binary ally there isn’ts further with ent of the bigical operatorsnected closed

n be done. In troximately 5%aged. is then conve

% grayscale. contrast areas,

Macula in t

of artifacts erators “Shrin

s then marked e to enhance th

e isolated binar

ing is an iters developed topropriate pointmeasure was

thresholded uncertainty in

�

K

ii pp1

log

probability of m mapping ofated which give

entropy first sight and th

n sight. The finitial increase ons only, whfter that resultsgion under ction of uncertawever, after a

e again due to tas well as du

ion grows. Thd as a terminaws one such tre iteration nualue. Images cown in Figureure represent thed image unde

y-scale images versions respe

t much informthe iterative p

inary region s that convert tarea which is

this work, this% of the row

erted to binarThe 30% lev

such as thethe image and

is donenk” and “C

and further conhe RGB imagery mask.

rative proceduo automaticallyt where the Mselected to bimage. Entropthe image [11]

the ith bin outf the image. Aes K bins to be

increases unhen decreasesfirst turning pocould be a res

hile the contin in a better vieconsideration.ainties of artifaa certain stagethe boundary eue to appearanhis gives a 2nd al point to storend. The x-ax

umber while ycorresponding te 5. The tophe actual gray-er study. The b

correspond toectively. As ca

mation availablprocedures. Fuis done usingthe binary pathen superimp

s was w and

ry by vel is e one d the

using Clean”

ntrast e area

ure, a y stop

Macula e the py is ]. It is

(6)

t of K A 2D e used ntil a

until oint is ult of nuous ew of

This acts in e, the ffects

nce of knee-

op the xis in y-axis to the gray--scale binary o their an be le but urther g the atches posed

onpro

ReTheImafousucgravarremsesis esti

Fi

Fi

the deconvolominently. Rela

esults and De algorithm isage Processing

und to be quitech result is preay-scale formatrious time inmains the samesion for the sacarried out inimate.

(a)igure 5. Images

point 1, an

igure 4. The ent

lved image toated results are

Discussion s implementedg toolbox [11consistent with

esented here. Tt and the dye istances. Sincee as the imageame patient. Hndependently w

s correspondingnd (b) iteration

tropy trend.

o locate the Me shown in the

d in MATLA]. While theh various patieThe obtained iminjection effecte the degradaes are obtainedHowever, the d

with the sam

(b)g to (a) iteration12 and knee po

Macula more next section.

AB using the results were

ents, only one mages are in t is shown at ation source

d in the same deconvolution e flat initial

) n 4 and knee oint 2.

149149

Retinal Images For the presented work, the images were recorded for the two eyes for almost 9 minutes after injecting the Fluorescein. Initially more frequent images were obtained and then the frequency was reduced. In a normal eye, the images correspond to several phases of the dye actions as outlined below: “Choroidal phase” is within 8-12 seconds after the dye has been injected. It shows the choroid due to leakage of dye through the leaky choroidal vessels. “Arterial phase” shows arterial filling in retina. “Arteriovenous phase” shows complete filling of the arteries and capillaries with early lamellar flow in the veins. “Venous Phase” is further divided into early , mid and late venous phases. Early venous phase shows marked lamellar flow in veins while mid venous phase displays complete filling of retinal veins. Late venous phase shows the reduction of dye in arteries. “Elimination phase” shoes effects of elimination of dye. Hence different points are located at different stages in the whole FA procedure and each phase has information related to certain parts of the eye that ophthalmologists are looking for. However, when the eye is diseased with some ocular layer such as cataract, the image would like the one shown in Figure 6.

Algorithmic Implementation While applying the algorithm to this image, the first step was to resize the image to 25% in order to speed up the processing. Figure 7 shows the resulting image after the application of the blind deconvolution. The process of deconvolution was initiated with a general purpose Gaussian kernel as the estimated blurring function. Figure 8 shows the initial and the final blurring kernel resulted as a result of the deconvolution process.

Figure 6. Occluded and blurred image from Fluorescein

Angiography of the diseased eye.

Figure 7. The deconvolved image.

(a)

(b)

Figure 8. Blurring functions, (a) initial estimate of a standard Gaussian function, and (b) final estimate at which the

deconvolution iterations were stopped. Both x and y axes (in the floor plane) represent the size of the kernel while the z-

axis represents the kernel values.

While the deconvolved image is already a lot enhanced in terms of visual quality, further clarity is achieved through post-processing steps. By locating the possible location of Macula, the image becomes more informative for the ophthalmologists. Figures 9 through 11 show the results of the application of the algorithm to the images corresponding to the images taken at 15th, 28th, and 40th second. Each result is comprised of three images, the original blurred image, the deconvolved image, and the marked image for the macula. As the time a progress, the effect of the injected dye becomes prominent first and then slowly starts to diminish.

Conclusion In this paper, a new technique has been presented that can be used to restore the distorted Fluorescein Angiogram of diabetic retinopathy patient occluded by a specific blur similar to the out-of-focus blur. The proposed deconvolution approach has shown promising results and will be further explored for an ultimate clinical tool development. Such a tool will be very useful for the ophthalmological experts in diagnosing diabetic retinopathy and getting it treated as soon as possible without undergoing cataract surgery which can not only delay the diagnosis but can also lead to further impairment of vision. The post processing steps enable the usage of

150150

(a) (b)

(c)

Figure 9. Results of image enhancement for FA image at 15th second (a) original blurred image, (b) deconvolved image, and (c) Post

processed image to locate Macula.

(a) (b)

(c)

Figure 10. Results of image enhancement for FA image at 28th second (a) original blurred image, (b) deconvolved image, and (c) Post

processed image to locate Macula.

the system on another level where specific areas of the eye can by automatically identified and further enhanced. The main importance of the work is the extraction of these important areas from a very bad quality image. Getting such an image is usually not very common, but when it is obtained, it leaves the doctors guessing what to do next. Usually they will start some type of treatment based on their understanding of the disease which may or may not be correct. With our proposed technique, they can get a more inside knowledge and that will help them in making a better and more informed decision regarding the illness. Further work is underway to make the proposed algorithm more reliable and more robust to other naturally occurring factors.

(a) (b)

(c)

Figure 11. Results of image enhancement for FA image at 40th second (a) original blurred image, (b) deconvolved image, and (c) Post

processed image to locate Macula.

References [1]. Kande, G., Savithri, T., and Subbaiah, P., “Segmentation of

Vessels in Fundus Images using Spatially Weighted Fuzzy c-Means Clustering Algorithm”, International Journal of Computer Science and Network Security, VOL.7 No.12, December 2007, p. 102.

[2]. Xu, Z., Guo, X., Hu, X., Cheng, X., and Wang, Z., “The blood vessel recognition of ocular fundus”, 7th Argentinean Symposium on Artificial Intelligence, 2005, pp. 183-190.

[3]. Sagar, A. V., Balasubramanian, S., and Chandrasekaran, V., “Automatic Detection of anatomical structures in digital fundus retinal images”, IAPR Conference on Machine Vision Applications, 2007, pp. 483-486.

[4]. Iqbal, M., Aibinu, A., Gubbal, N., and Khan, A., “Automatic diagnosis of diabetic retinopathy using fundus images”, Master thesis, Belkinge Institute of Technology, Sweden, 2006.

[5]. Dimitrakos, S., "Thirty years of fundus Fluorescein angiography", Ophthalmology, Vol. 4, pp.86-98, 1992.

[6]. Gutierrez, J., Epifanio, I., De Ves, E., and Ferri, F., "An Active Contour Model for the Automatic Detection of the Fovea in Fluorescein Angiographies", Proc. Int. Conf. on Pattern Recognition, Barcelona, Spain, IEEE Computer Society Press, 2000.

[7]. Ibanez, M., and Simo, A., "Bayesian detection of the fovea in eye fundus angiographies", Pattern Recognition Letters, Vol. 20, pp.229–240, 1999.

[8]. Holmes, T. J., “Background of Deconvolution”, Media Cybernetics Application Note, August 2006, http://www.mediacy.com/pdfs/Applications/BackgroundofDeconvolution.pdf

[9]. Le Cam, L., and Yang, L., Asymptotics in statistics: some basic concepts (Second ed.). Springer. ISBN 0-387-95036-2.

[10]. Ayers, G.R., Dainty, J.C., Iterative Blind Deconvolution Method and Its Applications,

[11]. Qidwai, U., and Chen, C.H., Digital Image Processing: An Algorithmic approach with MATLAB, CRC Publishing Company, November 2009.

151151