Oriented Markov Random Field Based Dendritic SpineSegmentation for Fluorescence Microscopy Images

Jie Cheng & Xiaobo Zhou & Eric L. Miller &

Veronica A. Alvarez & Bernardo L. Sabatini &Stephen T. C. Wong

Published online: 29 June 2010# Springer Science+Business Media, LLC 2010

Abstract Dendritic spines have been shown to be closelyrelated to various functional properties of the neuron.Usually dendritic spines are manually labeled to analyzetheir morphological changes, which is very time-consumingand susceptible to operator bias, even with the assistance ofcomputers. To deal with these issues, several methods havebeen recently proposed to automatically detect and measurethe dendritic spines with little human interaction. However,problems such as degraded detection performance forimages with larger pixel size (e.g. 0.125μm/pixel insteadof 0.08μm/pixel) still exist in these methods. Moreover, theshapes of detected spines are also distorted. For example,the “necks” of some spines are missed. Here we present anoriented Markov random field (OMRF) based algorithmwhich improves spine detection as well as their geometriccharacterization. We begin with the identification of aregion of interest (ROI) containing all the dendrites andspines to be analyzed. For this purpose, we introduce an

adaptive procedure for identifying the image background.Next, the OMRF model is discussed within a statisticalframework and the segmentation is solved as a maximum aposteriori estimation (MAP) problem, whose optimalsolution is found by a knowledge-guided iterative condi-tional mode (KICM) algorithm. Compared with the existingalgorithms, the proposed algorithm not only provides amore accurate representation of the spine shape, but alsoimproves the detection performance by more than 50% withregard to reducing both the misses and false detection.

Keywords Dendritic spine . Segmentation . Automaticdetection .Microscopy image

Introduction

The dendrites are cellular extensions of a neuron with manybranches, where more than 90% of input to the neuronoccurs. The dendritic spine is a small (sub-micrometer)membranous extrusion of the dendrites that contains themolecules and organelles involved in the postsynapticprocessing of the synaptic information. The remarkableability of dendritic spines to change shape rapidly, viz. thespine plasticity, is implicated in motivation, learning, andmemory (Moser, 1994; Yuste and Bonhoeffer 2001). Theabnormalities in dendritic spine morphologies are believedto be associated with a variety of brain disorders. Inparticular, neuron morphology is illustrative of neuronalfunction and can be instructive in the dysfunction seen inneurodegenerative conditions such as Alzheimer’s diseaseand Parkinson’s disease (Fiala et al. 2002; Knobloch andMansuy 2008). Cognitive disorders such as autism, mentalretardation and fragile X Syndrome (Durand 2007;Bilousova et al. 2009), may also be resultant fromabnormalities in dendritic spines, such as the number ofspines and their maturity status.

J. Cheng :X. Zhou : S. T. C. Wong (*)The Center for Bioengineering and Informatics, The MethodistHospital Research Institute and Department of Radiology,The Methodist Hospital, Weill Cornell Medical College,Houston, TX 77030, USAe-mail: stwong@tmhs.org

V. A. AlvarezSection on Neuronal Structure (SNS),Laboratory for Integrative Neuroscience (LIN) NIAAA / NIH,5625 Fishers Lane, Room TS-24, MSC9411,Bethesda, MD 20892-9411, USA

B. L. SabatiniDepartment of Neurobiology, Harvard Medical School,220 Longwood Ave,Boston, MA 02115, USA

E. L. MillerDepartment of Electrical and Computer Engineering,Tufts University,Medford, MA 02155, USA

Neuroinform (2010) 8:157–170DOI 10.1007/s12021-010-9073-y

In order to analyze their morphological changes, usuallythe dendritic spines in the acquired microscopy images aremanually labeled. This is extremely time-consuming evenwith the computer assistance, which makes it infeasible tomanually analyze a great amount of data. The manual resultsobtained by a trained expert nevertheless are more accuratecompared with the automatic results and usually considered asthe gold standard, even with the possible bias problem.Recently, some automatic dendritic spines analysis algorithmshave been provided for both in vitro (Koh et al. 2002; Weaveret al. 2004; Bai et al. 2007; Cheng et al. 2007; Xu et al.2006; Zhang et al. 2007) and in vivo (Fan et al. 2009) neuralimages analysis. Most of these methods are based on aglobal threshold with various approaches applied, such asgeometric analysis (Koh et al. 2002), two grassfire pro-pagations (Xu et al. 2006), unsharp mask filtering (Bai et al.2007), curvilinear structure detector and linear discriminateanalysis (LDA) (Zhang et al. 2007). Cheng et al. propose anautomatic spine detection pipeline based on the adaptivethresholding and local dendrite morphology analysis (Chenget al. 2007). Although these algorithms can greatly reducethe human labor, problems still exist. For instance, theefficiency of detection will degrade for images withlarger pixel sizes, in which the spines occupy relativelyfew pixels. Also, the above mentioned algorithms areprone to break the spine necks: pixels in spine neckregions usually have low intensity values and are likelysegmented as background, which cause the spines to bebroken into several detached components thereby furtherdegrading spine detection performance. To deal withthese issues, here we propose a novel maximum aposteriori—oriented Markov random field (MAP-OMRF)framework for dendritic spine segmentation.

Markov Random Field (MRF) models were first pro-posed for image processing by Geman and Besag (Gemanand Geman 1984; Besag 1986 and 1989). Since then, theyhave been intensively studied in this field for variousapplications, largely because of their ability to encodeexpected spatial correlations through contextual constraintsof neighboring pixels. Usually, the correlation structure issuch that pixels are expected to have the same or similarintensities, i.e. the image is piecewise constant or piecewisecontinuous. In the literature, the MRF based segmentationmethods have been widely applied to medical imagesobtained from many modalities operating across a widerange of application areas. These include, but not limited to,cardiac imaging (Xiao et al. 2002; Juslin and Tohka 2006),brain imaging (Descombes et al. 1998; Calhoun et al. 2001;Salli et al., 2001; Awate and Whitaker 2007; Rajapakse etal. 2008), digital mammography (Li et al. 1995; D’Elia etal. 2004) acquired by magnetic resonance imaging (MRI),ultrasound, positron emission tomography (PET), andelectron microscopy.

Despite the successful applications in medical imagesegmentation, MRF methods have seldom been used indendritic spine detection, partially because of the need tospecialize these models to address the specific problemsdiscussed previously concerning spine segmentation. Todeal with these issues, the OMRF model is proposed in thispaper. By encoding the location orientation of the spinesrelative to the dendrite, this model allows us to overcomemany of the segmentation challenges mentioned above. Toreduce the computation complexity and make the algorithmmore robust to the background noise, a region of interest(ROI) estimation algorithm, which is based on an iterativespline background correction, is also proposed. Finally, wepresent a knowledge-based iterated conditional modes(KICM) algorithm in which biological constraints areincorporated in the transition matrix. Compared with thenormal ICM algorithm, the segmentation performance ofKICM is much improved because of its ability to accuratelylocate the spine base.

The paper is divided into the following parts: how toobtain the images are first discussed in Section “ImageAcquisition”. Then, the algorithms are presented in Section“Algorithms”, where the OMRF model is proposed. Wealso describe the computation of the orientation mapneeded by the OMRF, as well as the ROI estimationalgorithm. The KICM optimization scheme is discussednext. In Section “Results and Validation”, the segmentationand detection results are compared with other existingmethods and validated manually. Finally the conclusion isprovided and the future works are discussed in Section“Discussion and Conclusion”.

Image Acquisition

Brain slices from the hippocampus were prepared from ratpups (P7) and cultured as described in Alvarez et al(Alvarez et al. 2006). Slices were transfected with a greenfluorescent protein (GFP) expressing vector and pyramidalneurons were identified based on their characteristicmorphology at 7–20 days post-transfection (DPT). Neuro-nal morphology was studied using 2-photon laser scanningmicroscopy (2PLSM) and a custom-built microscope with awater immersion objective (Olympus LUMPlanFI/RI 60×,NA=0.9). The excitation wavelength was 910 nm providedby a Verdi 10-V.-Mirra laser (Coherent). Measurementsperformed on 100 nm diameter yellow-green fluorescentmicrospheres (FluorSpheres, Molecular Probes) indicatedthat the point-spread function of the microscope placed alower limit on measurable widths of 550 nm.

Images (512×512 pixels) of apical and basal dendrites ofhippocampal pyramidal neurons were acquired at zoom 3 and5 (image field, 64×64 µm and 42×42 µm, respectively). The

158 Neuroinform (2010) 8:157–170

3D image stacks were 16-bit grey-scale with 1 µm opticalsection spacing with 20–36 slices in each stack. The analyzeddataset includes a variety of genotypes to ascertain how wellour algorithm detects spines with a wide distribution ofmorphologies. The manual analysis of spine density, length,and width was performed by observers who were blind to thegenotype by using in-house developed software (Tavazoie etal. 2005; Alvarez et al. 2006). Spine lengths were measuredfrom the junction with the dendritic shaft to the tip. Todetermine head width and primary dendrite thickness, thefluorescence was measured in a line across the structure. Thewidth of the distribution where fluorescent intensity fell to30% of maximum was calculated as the results.

Algorithms

Although 3D image stacks were acquired, our proposedalgorithms are based on the 2D maximal intensity projec-tion (MIP) images. This choice was made because: theresolution in the z- direction is much lower compared withthat in x- and y- directions. For spines big enough, the 3Dinformation in neighboring slices is helpful for detection.However, there are many small spines appearing in onlyone slice, which limits the utility of a full 3D approach. Thesame is true for the spine necks which are much thinnercompared with the head components. Since the purpose ofour algorithm is to solve the detection problem of smallspines and the spine necks, we only propose 2D algorithms.Actually, because most of the morphological information ofthe dendritic spine is included in the MIP image, 2D basedautomatic algorithms can effectively detect and measure thedendritic spines. This has been proved by several recentlypublished papers (Bai et al. 2007; Cheng et al. 2007; Zhanget al. 2007).

The overall structure of our proposed algorithms isoutlined in Fig. 1: in order to reduce the computationalcomplexity and improve the detection performance, seg-mentation is performed only over a region of interest (ROI)covering the area around the dendrite. The algorithm usedto determine this ROI is provided in Section “Region ofInterest”. The calculation of the orientation map used byOMRF model is then described in Section “Oriented

Markov Random Field”. Based on the ROI and orientationmap, as well as the original image and the initialsegmentation results, the OMRF model is proposed inSection “Oriented Markov Random Field” for the purposeof dendritic spine segmentation. Finally, the KICM algo-rithm which can efficiently find the final segmentation ispresented in Section “Optimization Algorithm”.

Region of Interest

The main purpose of our ROI algorithm is to reduce thecomputation cost as well as improve detection perfor-mance by limiting the spine segmentation processing to alocal region much smaller than the full image. In existingspine detection algorithms, simple physical constraintsare applied to improve the detection results. Forexample, only those detected blobs within a certaindistance to the dendrite are considered to be spines (Kohet al. 2002; Cheng et al. 2007). However, manybackground pixels are still contained in the local regions.These noisy background pixels not only increase theprocessing time, but also potentially increase the risk ofmisdetection. To deal with this issue, we propose a localROI estimation method derived from an iterative splinebased background correction algorithm (ISBC) (Wahlby etal. 2002). The ISBC algorithm is initially proposed forsolving the problem of large scale intensity variations andshading effects in the image caused by uneven illumina-tion. However, it can also be applied to acquire the ROIwith minimal modification.

The background is estimated and modeled by a cubic B-spline surface patch S. The value of a surface point of thepatch S can be written as (Wahlby et al. 2002):

S u; vð Þ ¼X

klBkðuÞBlðvÞckl ð1Þ

where (u, v) is position of the pixel; ckl is a control point ofthe surface (an evenly spaced 5×5 grid of control points areused in this paper); Bk and Bl are the B-spline blendingpolynomials and the Cartesian product of the two functionsis the weight given to the control point. Eq. 1 indicates thatthe estimated value of each pixel in the patch S is theweighted summation of all control points. The control

existing algorithms

proposed algorithm KICM

proposed algorithm

Original image

Initial segmentation results image

ROI

Oriented map

OMRFFinal

results

Fig. 1 structure of the proposedalgorithms

Neuroinform (2010) 8:157–170 159

points are initially estimated by minimizing the Euclideandistance between the spline surface and the original imageusing least squares regressions. For this process, neither thebackground nor the foreground has been determined yet,thus all of the pixels in the image are used in the fittingprocedure. Since the much brighter pixels in dendrites andspines (i.e., the foreground) are included in the backgroundestimation stage, the initially estimated background must beadjusted. This is done by masking out the foreground pixelsand applying the spline surface to fit the refined potentialbackground regions. The foreground pixels are determinedthrough a simple thresholding procedure described below.The process of masking the foreground and recomputingthe background spline fit is iterated until the averagechange of pixel values between two consecutively estimat-ed backgrounds is small enough, e.g. less than the originalquantization step of the image, which is set as 1 for ourtesting. The ROI can then be obtained by thresholding thebackground compensated image. The “background com-pensated image” is calculated by subtracting the iterativelyestimated background from the original image. Since thebackground is initially set as zero, the backgroundcompensated image is initialized as the original imageitself.

At all stages during this process, the masking procedureis accomplished using a simple threshold. Suppose thestandard deviation of the pixel values in the backgroundcompensated image is σc. With mean normalized to 0, theforeground R is defined by:

R ¼ all pixel ijIcðiÞ > k � scf g ð2Þ

where Ic (i) is the pixel value at i in the backgroundcompensated image; k is a factor which is set as 10 duringour validation.

Figure 2 provides an example of this process. For thepurpose of better visualization only, Fig. 2b is the scaledfinal background compensation image in which the pixelvalues are in the range of 0 to 255. Figure 2c shows theROI obtained after thresholding, with small objectsremoved by applying opening operation (erosion followedby dilation). As we can observe from Fig. 2d, allprotrusions along the dendrite boundary, which arepotential spines, are contained in the obtained ROI. TheROI is a much reduced region compared to the originalimage which can be obtained in seconds: for the 512×512image shown in Fig. 2 and codes implemented in Matlabwithout optimization, the ROI can be obtained in less than3 s by a PC with a 2.5 GHz CPU; the number of pixels inthe ROI is 49679, which is about 81% reduction of thetotal pixels in the original image. To further test theeffectiveness of the ROI, the processing time with andwithout using ROI is compared: for ten images being

tested, the average processing time for the proposedsegmentation algorithm with using ROI is about 2’4.8”,which is about 46% reduction compared with the average3’52.2” processing time without using ROI.

Oriented Markov Random Field

General Model

MRF models are widely applied in various fields of imageprocessing due to their ability to mathematically capture theintrinsic spatial structure of images. Usually for MRF-basedmethods, segmentation is achieved through an optimizationprocess. The optimization function is derived from proba-bilistic models of the data and the prior informationconcerning the spatial structure of the segments is encodedin the MRF. Indeed, one of the most appealing aspects ofthe MRF formulation is its flexibility in representingstructural prior information across a wide variety ofapplications. For the problem of interest here, the primarysource of prior information concerning spines is related tothe structure of the dendrite. Indeed knowing the locationand orientation of the dendrite places important geometricand morphological constraints on the associated spines. Inthis paper we introduce a new MRF model which considersthe inherent orientation information in the neuron cells, viz.OMRF model.

We model the observed grey scale image as a randomfield y, with the intensity of a pixel at location i denoted bya random variable yi. The distribution of different regions isdenoted by a random field x, where the variable xi∈{1,2,...,K} means that the pixel i belongs to one of the M regiontypes. For the neuron images, K=3, representing threeclasses of interest in this application: the dendrite, spine andbackground.

Given the observed data y, the problem of segmentationamounts to the determination of x, that is, the labelsassociated with each pixel. We pose this problem in astatistical framework as a maximum a posteriori estimationproblem where x is chosen to maximize the posteriordistribution of the label based on y (Li 2001). Formally, byusing Bayes theorem this distribution is described as:

p xjyð Þ / p yjxð Þp xð Þ ð3Þ

where p(x) is the a priori probability density of theimage; p(y | x) is the conditional probability density of theobserved image given the distribution of regions. Afterremoval of the shot noise by median filter, the likelihoodfor each pixel p(y | x) can be described by independentGaussian (Hebert 1997). The assumption that the intensityof each pixel is statistically independent can greatlysimplify the processing. Under the independent and

160 Neuroinform (2010) 8:157–170

Gaussian assumption, the possibly observed image with xknown can be represented as:

p yjxð Þ / expXi2S

� 1

2s2i;xi

yi � mi;xi

� �2( )ð4Þ

Here S is the collection of all pixels; s i;xi and mi;xiare thevariance and mean value of the intensity of pixel i, which isrepresented by a random variable yi. The variance and meanvalue are class depended, i.e. they are functions of xi, whichmeans that the intensity distribution for pixels of differenttypes (e.g. spine, dendrite and background) are different. Inaddition, from a practical perspective the mean values andvariances also vary over the image: their values are alsorelated to the pixel location i. This is based on theobservation that even in the same image the intensitiesvary largely for spines or dendrite pieces at differentlocations. Thus, the mean and variance should be estimatedadaptively. Calculation of s i;xi and mi;xi will be discussed indetail in Section “Parameter Estimation”

The a priori density p(x) is described by the OMRFmodel. According to the Hammersley-Clifford theorem,any Markov random field can be described by a

probability distribution of the Gibbs form (Geman andGeman 1984):

P xð Þ ¼ Z�1e�U xð Þ ð5Þwhere P(x)denotes the probability of configurations of therandom field x and U(x) is the energy function and Z aconstant needed to ensure that P integrates to one. Wenotice that the neuron dendrites are oriented structures: thegrowing direction of each spine is normal to the orientationof local dendrite pieces. This natural orientation informa-tion can be combined into the energy function:

U xð Þ ¼ b �Xi2S

Xj2Ni

D xi; xj� �

1þ a � 1� ju*ijT v

*

ij� �h i

ð6Þ

where ~viis the orientation of the dendrite in the vicinity ofpixel i and will be described in detail in the followingsection; ~uijis the direction of the line connecting twoneighboring pixels of i and j; α is the weight factor of theorientation information; β is a parameter which controlshow strong the spatial regularization is;S is the collection ofall pixels; Ni is the 8-neighbor of pixel i;xi is the segmentassociated with pixel i; xj is the segment of neighboring

(a) (b)

(c) (d)

Fig. 2 a original image; bbackground compensated imagescaled to have pixel valuesbetween 0 and 255 c ROIobtained after segmenting thebackground compensated imageand denoising d ROI (regionsinside the high-lighted curves)in the original image

Neuroinform (2010) 8:157–170 161

pixels of i; and D(xi,xj) is the similarity function. If xi=xj, i.e. pixel i and j are of the same type, the value of D(xi,xj) isset as 0, otherwise the value is set as 1. Thus, pixel i isintended to be in the same region as most of its neighboringpixels, which will reduce the energy U(x). From Eqs. 3–6,we have:

P xjyð Þ / P yjxð Þp xð Þ/ exp �P

i2S1

2s2i;xi

yi � mi;xi

� �2 � U xð Þ� �

/ � Pi2S

12s2

i;xi

yi � mi;xi

� �2 þ U xð Þ� �

¼ Pi2S

� 12s i

2 yi�mið Þ2þbPj2Ni

D xi; xj� �

1þ a � 1� ju*ijT v

*

ij� �h i( )

ð7Þ

Orientation Map

The use of orientation is motivated by the intuition that theprobability of being segmented as a spine pixel should berelated to the directions of both ~vi and ~uij. Suppose thebackbone of the dendrite has already been obtained bythinning and trimming algorithms (Cheng et al. 2007).Assume ck is a backbone pixel, we denote by Rck ¼fck�2; ck�1; ck ; ckþ1; ckþ2g the set of local backbone pixelsfrom which we shall determine the orientation at ck. Ingreater detail, linear least squares is used to fit a linethrough these five pixels. Letting gckbe the slope of thatline,~viis defined as:

~vi¼funit vector with slope gck jckis the nearest backbone pixel to ig

ð8Þ

Figure 3 demonstrates how the orientation map isobtained. The orientation of a pixel is defined as that ofthe neighboring backbone pixel with the least Euclidiandistance, which is estimated in a local window as shown in

Fig. 3b. Figure 3c shows the sampled orientation map. Theorientation of pixels outside the ROI is set to zero, becausewe limit our attention to the ROI surrounding the dendritesfor segmentation. In Fig. 3c we notice that along the samedendritic trunk, the vectors may abruptly change direction.This is because there are possibly two angles which candescribe the orientation of the backbone, α and α+π. Sinceonly the absolute values of the scalar product between thevectors are needed in Eq. 7, this ambiguity is irrelevant tothe results. In this paper, we assume that the angles arebetween 0 and π.

Optimization Algorithm

In neuron images to be processed, there are three differentregion types, i.e. spine, dendrite and background. For ourapproach, the segmentation of neuron images is designed asa maximum a posteriori estimation (MAP) problem.Suppose y is the observed data, the optimal segmentationresults x̂ is found by solving the following MAP estimationproblem:

bx ¼ argmaxx

P xjyð Þ ð9Þ

To acquire the optimal solution, an extended iteratedconditional modes (ICM) algorithm, viz. KICM algorithmis proposed in this paper. The interleaved ICM and localparameters updating scheme is applied (Pappas 1992), i.e.,estimation of the optimal segmentation x̂ and parameterupdating are alternatively performed.

Parameter Estimation

Here we denote by q̂1and q̂2 the optimal parameterestimates for the prior model and the likelihood modelrespectively. In this paper, θ1=(α, β), where α and β are

(a) (b) (c)

Fig. 3 a original image; b initial segmentation result of the neuron without spine detection; the white curves are the extracted backbone pieces; v*

andg*

are the estimated local dendrite orientation and the growing direction of the spine respectively; c sampled orientation map

162 Neuroinform (2010) 8:157–170

defined in Eq. 6; θ2=(φ1φ2...φN), with Nbeing thenumber of total pixels in the image and fi ¼ mi;xi ; s

2i;xi

� �,

i ∈[1,2,...,N]. As mentioned in Eq. 4, mi;xi and s i;xistand forthe mean value and variance of the intensity distribution ofpixel i respectively, whose values also depend on thesegmentation result xi. For the purpose of simplifying thecomputation complexity, here θ1 is set by the user prior toexecution of the algorithm. For all images being tested, wehave α=0.5 and β=0.3. Since the two parameters are notindependent to each other, it is hard to manually find theoptimal values. Generally, the detection results are good ifα [0.3,0.7] and β [0.2,0.4]. As for θ2, ideally it should beestimated within a MAP-type framework, i.e. we maxi-mize p q2jyi; xið Þwhich is calculated based on the distribu-tion of the prior probability function p(θ2) and theconditional probability function p yi; xijq2ð Þ according tothe Bayes theorem. However, such a process would bequite computationally intensive. Thus, in this paper wepursue an ad-hoc approach that appears to work ratherwell in terms of the ultimate objective of improving ourability to characterize spine number and shape. Specifi-cally, θ2 is estimated on the assumption that the intensitiesof the same type pixels in a local window are independentand identically-distributed (i.i.d.) Gaussians (Hebert1997). Since the estimation of segmentation results x isupdated iteratively, bq2 is also iteratively estimated. Theparameter of pixel i can be iteratively estimated by thefollowing equation:

bfðkÞi ¼argmaxfðkÞi

p fðkÞi jx k�1ð Þ; y� �

¼ argmaxfðkÞi

1ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi2ps2ðkÞ

i;xk�1ð Þi

q� �M exp � Pj2S x k�1ð Þ

ið Þyj�mðkÞ

i;x k�1ð Þi

� �2

=2s2ðkÞi;x k�1ð Þ

i

0@ 1Að10Þ

where bfðkÞi is the estimated parameter of pixel i at step k;x(k−1) is the segmentation results at step k−1, with x(0)

being the initial segmentation result obtained byexisting algorithms (Koh et al. 2002; Weaver et al.2004; Bai et al. 2007; Cheng et al. 2007; Xu et al. 2006;Zhang et al. 2007); y is the observed data; S xðk�1Þ

i

� �is the

set of pixels in a local window around pixel i, with thesame class as pixel i, i.e. all pixels belong to class xi k�1ð Þ.The size of the local window is image pixel sizedependent and based on the normal size of thedendritic spines. For example, the 15×15 window ischosen for the images being tested with pixel size of0.125 μm/pixel, in which the spine areas are generallybelow 100 pixels. Suppose M is the total number of

pixels in the set Sðxðk�1Þi Þ, a necessary condition for

maximizing p fðkÞi jxðk�1Þ; y� �

, or equivalently maximizing

ln p fðkÞi jx k�1ð Þ; y� �

, is @ ln p

@mðkÞi;xðk�1Þi

¼ 0 and @ ln p

@sðkÞi;xðk�1Þi

¼ 0. Solving

this, we can find the adaptive ML estimates:

bmðkÞi;xðk�1Þ

i

¼ 1

M

XMj¼1

yj ð11Þ

bs2ðkÞi;x k�1ð Þ

i

¼ 1

M

XMj¼1

yj � bmðkÞi;x k�1ð Þ

i

� �2

ð12Þ

where j [1,M] are all the pixels in a 15×15 window aroundpixel i with the same class, i.e. all pixels belong to classxi k�1ð Þ.

Estimation with KICM

The optimal segmentation result bx is further estimated afterthe parameters bq1and bq2are obtained. Here an extendediterated conditional modes (ICM) algorithm, viz. KICMalgorithm, is applied to acquire the optimal solution. ICM(Besag 1986) is based on two assumptions: 1) Theobservation components y1,...ym are conditionally indepen-dent given x, and each yi has the same known conditionaldensity function p yijxið Þwhich depends only on its label xi,i.e., p yjxð Þ ¼ Q

i2Sp yijxið Þ; 2) The labeling results satisfy the

Markovianity: x depends on the labels in the localneighborhood.

The algorithm sequentially updates the segmentationresult of every point in the image xðkÞi into x kþ1ð Þ

i bymaximizing the conditional probabilityP x kþ1ð Þ

i jy; xðkÞS�fig� �

,which is performed in a raster scan. The iteration stopswhen the number of pixels that change during a cycle isless than a threshold.

xðkþ1Þi ¼ argmax

xi2LPðxijy; xðkÞS�figÞ ð13Þ

Here L={1,2,3} represents the three different regiontypes in neuron images. The initial estimate x(0) can beobtained by any spine detection algorithms (Koh et al.2002; Weaver et al. 2004; Bai et al. 2007; Cheng et al.2007; Xu et al. 2006; Zhang et al. 2007).

For real world applications such as dendritic spinedetection, a pure statistically optimized solution is notalways the best choice for the purpose of image analysis.For example, for neuron images, an intensity distributionbased model is likely to segment a dendrite pixel as a spinepixel because of the similar intensity values. To deal withthis issue, more constraints with explicit biological mean-ings must be incorporated into the processing. In our case,these new constraints are not used to modify the energyfunction. Instead, they are employed to develop a new

Neuroinform (2010) 8:157–170 163

strategy of searching for ideal solutions. Unlike the ICMalgorithm, the proposed KICM algorithm will stop at somepoint before reaching the statistically optimal solutionbecause of the biological constraints incorporated in thetransition matrix. The solution, however, is intended topossess more biological meaning. For instance, spines areprotrusions along the boundary of a dendrite. Thus, allpixels inside the dendrite should not be segmented as spinepixels.

To better describe the KICM algorithm, we first rewritethe normal ICM algorithm in a matrix form with thetransition matrix A being defined. For each pixel i, suppose

that sðkÞ ¼ sðkÞ1 sðkÞ2 sðkÞ3

h iTis the segmentation result of the k-

th step. Here sðkÞ ¼ 100½ �T, [010]T or [001]T means thepixel is segmented as the spine, dendrite or backgroundrespectively. Then we have

s kþ1ð Þ ¼ A � sðkÞ ¼að1Þ að1Þ að1Það2Þ að2Þ að2Það3Þ að3Þ að3Þ

24 35 � sðkÞ ð14Þ

Here A is the transition matrix,

aðmÞ ¼ 1; ifem ¼ min e1; e2; e3f g0; ifem 6¼ min e1; e2; e3f g

ð15Þ

and e1, e2,e3 represent the energy if the pixel issegmented as spine, dendrite or background respectively.The energy can be calculated by Eqs. 6 and 7:

e1 ¼ Es ¼ � log p yijxi 2 spineð Þ þ U xi 2 spineð Þ ð16Þ

e2 ¼ Ed ¼ � log p yijxi 2 dendriteð Þ þ U xi 2 dendriteð Þð17Þ

e3¼ Eb¼ � log p yijxi 2 backgroundð Þ þ U xi 2 backgroundð Þð18Þ

With some modification of the transition matrix A, theKICM algorithm is described as

s kþ1ð Þ ¼ A � sðkÞ¼að1Þ a12 að1Þa21 að2Þ að2Það3Þ a32 að3Þ

24 35� sðkÞsðkÞ; otherwise

8>><>>: ; ifA � sðkÞ 6¼ ½000�T

ð19Þ

Here the last column of the transition matrix isunchanged, which means if at the k-th step a pixel issegmented as background, at the next step it will besegmented only based on the lowest energy criterion.However, it is not the case if a pixel is formerly segmentedas dendrite or spine.

The value of a12, a21, and a32 are calculated based on theprior knowledge (assumption) of the neuron image and theinitial segmentation results:

1. In practice incorrectly segmented background pixels aresome ‘detached’ noise blobs, which have relatively highintensity values and thus are likely to be segmented asspine pixels. Thus we assume in the initial segmentationresults that no background pixels are segmented asdendrite. Based on this assumption we have a32=0,which means that initially segmented dendrite pixelscannot be segmented as the background later.

2. The spines are the protrusions along the dendriteboundary. If we can estimate the width of dendrite,then spine pixels can only be those pixels whosedistance to the center of the dendrite (the backbone) isgreater than half of the local dendrite width. Thedetailed algorithm to estimate the width of localdendrite piece is discussed in (Cheng et al. 2007).Based on this assumption, the value of a12 and a21 canbe defined:

a12 ¼ 1; ife1 ¼ min e1; e2f g and di >ti2

0; otherwise

ð20Þ

a21 ¼ 1; ife2 ¼ min e1; e2f g and di <ti2

0; otherwise

ð21Þ

Here di is the distance between pixel i and backbone, ti isthe estimation of the width of local dendrite

Let xi be the segmentation result for pixel i, xs/i be thecurrent segmentation result elsewhere, x∂i is the segmenta-tion result for the 8-neighbor ∂i of pixel i. For MRFmodels, we have

P xjyð Þ ¼ P xijy; xsni� �

P xsnijy� �

¼ P xijy; x@ið ÞP xsnijy� � ð22Þ

We notice that P xijy; x@ið Þnever decreases: xi is eitherupdated when lower energy is found, or is unchanged wheneither (a) lower energy can not be found or (b) whenA � sðkÞ ¼ 000½ �Tas described in Eq. 19. Besides, P xsnijy

� �is

kept constant no matter what value xi is. Thus just like thetraditional ICM algorithm, the KICM algorithm performs ina hill-climbing manner and eventual convergence isassured.

The KICM algorithm is a good complement to MRF-based algorithms, which are based on spatial constraintswithin local neighborhoods. Normally, performanceimproves as the size of the neighborhood increases.However, larger neighborhoods make the model morecomplex and greatly increase the computation complexity.

164 Neuroinform (2010) 8:157–170

That is the reason why 4- or 8- neighborhood models arenormally used for MRFs. The KICM algorithm provides anovel perspective on how to combine more local informa-tion in a much bigger neighborhood without largelyincreasing the computation complexity. Together with theorientation information included in the prior models of theneuron images, the segmentation results can be noticeablyimproved. Figure 4 shows the comparison of segmentationresults between normal MRF-ICM algorithm and theproposed OMRF-KICM algorithm. We can observe fromthe results that the shapes of the dendritic spines are muchbetter represented with the “weak” components (spineneck) being enhanced and the base of the spines accuratelyfound.

In summary, the main frame of the proposed OMRF-KICM algorithm is described below:

1. Obtain region of interest by background compensa-tion algorithm described in Section “OptimizationAlgorithm”

2. Obtain the orientation map described in Section“Oriented Markov Random Field”

3. Set the value of θ1

4. Compute the likelihood probability p yijxið Þat position i5. Initialize the algorithm with the segmentation result

x(0), k=06. Repeat7. Estimate bq2with Eqs. 11 and 128. Repeat with x(k) known9. For each pixel i in the image

10. calculate energy e1, e2, e3 using Eqs. 16–1811. update a(m) using Eq. 1512. s kþ1ð Þ ¼ A � sðkÞ using Eq. 1913. End14. k=k+115. Until the difference between x(k) and x(k−1) is below a

threshold

16. Until N times of iteration17. Return x(k) as the optimal segmentation results

Results and Validation

Validation with existing algorithms

The proposed algorithm is first applied to the initialdetection results obtained by existing detection algorithmsbased on both global thresholding (Koh et al. 2002) andadaptive thresholding (Cheng et al. 2007). We can observefrom Figs. 5 and 6, the OMRF-KICM method canefficiently improve the spine detection results for neuronimages with different pixel sizes.

Figure 5 shows the detection results for neuron imagesacquired with the pixel size of 0.08μm/pixel. From Fig. 5band d, we can observe that the initial detection resultsobtained by global thresholding based algorithm are muchimproved: there are fewer missing spines (especially for theright-upper parts.) Also, the false positives are reduced bycombining the broken spine components. The ‘weak’regions such as the spine neck areas have been effectivelyenhanced and the broken spine components are reduced.Compared with the global thresholding based methods, thedetection results obtained using the adaptive thresholding-based algorithm (Cheng et al. 2007) show fewer missingspines. However, there are still some false positivescaused by the local maximums. The OMRF-KICMmethod can nevertheless remove those positives as shownin Fig. 5c and e.

As shown in Fig. 6, for some neuron images acquiredwith larger pixel size, both global and adaptive thresholdingbased algorithms cannot efficiently detect low intensity,‘weak’ spines. One easy approach is to include anadditional preprocessing step of interpolation so that the

(a) (b) (c)Fig. 4 comparison between MRF-ICM and OMRF-KICM, a original image; b segmentation result with normal MRF-ICM algorithm; csegmentation result with OMRF-KICM algorithm

Neuroinform (2010) 8:157–170 165

(a) original MIP images (b) results using global thresholding (c) results using adaptive thresholding

(d) after applying OMRF-KICM on (b) (e) after applying OMRF-KICM on (c)

Fig. 5 Detection results for images on with pixel size of 0.08 micron/pixel. a MIP images; (b), (d) initial segmentation results using globalthresholding based algorithms and results after applying the proposed

algorithm model respectively; (c), (e) initial segmentation results usingadaptive thresholding based algorithms and results after applying theproposed algorithm

(a) original MIP images (b) results using global thresholding (c) results using adaptive thresholding

(d) after applying OMRF-KICM on (b) (e) after applying OMRF-KICM on (c)

Fig. 6 Detection results for images with pixel size of 0.125 micron/pixel. a original MIP images; (b), (d) initial segmentation results usingglobal thresholding based algorithms and results after applying

OMRF-KICM method respectively; (c), (e) initial segmentation resultsusing adaptive thresholding based algorithms and results afterapplying OMRF-KICM method

166 Neuroinform (2010) 8:157–170

image can be processed with a smaller image pixel size.However, problems exist for this method: besides thereduced detection performance caused by the interpolationdistortion, the processing time also increases with a biggerimage size. For five tested images by using the methoddescribed in (Cheng et al. 2007), the average processingtime for the interpolated images (762×762) is about17’18”, which is more than a seven fold increase comparedwith the average 2’32” processing time for images (512×512) without interpolation. The proposed algorithm cannevertheless solve the missing problem well, with higheraccuracy and much less processing time (average 2’5”processing time for 512×512 images).

Validation with Manual Results

Automatic dendritic spine detection results, with andwithout the processing of the proposed algorithm, are firstcompared with the manual results, which are treated as thegold standard. Initially each automatically detected spineblob is labeled as a spine. However, in some cases a spineseems to be “broken” into several components, which isdue to the limited resolution in the z- direction and thesmall size of spine neck. The broken components of asingle spine will cause errors in the quantification of spinestructures including spine number and length. To addressthis problem, a merging algorithm is performed to combinethose separated parts into single spines. The details of thealgorithm can be found in (Cheng et al. 2007). Basically thecombination is based are two criteria: the broken partsshould be close enough and the orientation of the lineconnecting the centroids of the separated componentsshould be in a certain range of angle. For the examples inSection “Results and Validation”, we use [30°, 30°]. Eachautomatically detected spine component is initially assignedan index number, in an order of the distance to the left edgeof the image. After merging, the broken spine headcomponents are reassigned the same index number as thespine base.

Figure 7 illustrates how the broken spine components arecombined and how the automatic detection results arevalidated using the manual results. Figure 7a is the

segmentation result of part of image 1 by applying method2 as mentioned in Table 1. We can observe that the brokenhead parts are successfully combined with the relative spinebases, e.g., spine number 23 and 27. During validation, thespines are manually marked in 3D. Figure 7b shows howthe manually marked image looks in the 2D projectedimage. For better illustration, the false negatives and falsepositives are marked as yellow and red respectively. As wecan observe in Fig. 7, two manually marked spines can notbe automatically detected; the automatically detected spine32 is actually a false result.

The initial detection results are obtained using bothglobal thresholding (method 1) and adaptive thresholding(method 2) based methods. Five neuron images withdifferent pixel sizes are compared. The manual results areincluded in the braces. The false positive (FP) and falsenegative (FN) values before and after applying theproposed method are listed in Table 1. As we can see fromthe results, the FP (wrong detections) and FN (undetectedforeground) are decreased for both methods after theprocessing of the proposed method. The spine densitymeasurement, however, does not show much improvement.To better illustrate the differences, the precision (P) andrecall (R) rates are also provided. According to the paired t-test, the precision rates of methods with and withoutapplying the proposed algorithm show significant differ-ence with p-value 0.0025. The recall rates also showsignificant difference with p-value 0.001.

Further validation is performed by comparing the spinelength distribution of the proposed method with the manualresults. The details of how we automatically measure thelength of attached and detached spines can be found in(Cheng et al. 2007). Spines in different section of the sameneuron cell under the same condition (shLUCI in hippo-campal pyramidal neurons in rat organotypic slice cultures)are tested. There are altogether 235 spines in three differentimages. The CDFs of the two distributions are plotted inFig. 8, from which we can see that the distributions of spinelength measured manually and automatically are similar toeach other. To quantify the difference, the distributions aretested by two-sample Kolmogorov-Smirnov test. The nullhypothesis that the two distributions are the same is not

(a) (b)

Fig. 7 Comparison of automaticand manual detection results inMIP image a labeled automaticresults by applying method 2 asmentioned in Table 1, withfalsely detected spine markedwith red b manual results, withspine not detected by theautomatic method marked withyellow

Neuroinform (2010) 8:157–170 167

rejected, which means that there is no obvious differencebetween the two distributions. The probability that thedendrite length distributions of manual and automatedresults are the same is 99.13%. The biggest differencebetween these two distributions is 0.075.

The proposed method is also validated using publishedmanually segmented data. This recently published paper(Alvarez et al. 2006) examines the off-target effects ofexpressing short hairpin RNAs (shRNA) in neurons. Thestudy shows that expression of shRNA against luciferase,whose coding region is not found in the rat genome,triggers dramatic loss of dendritic spines and simplificationof dendritic arbors. Figure 9 shows the difference ofaverage spine densities and average spine length for GURand shLUCI neurons at 4DPT. By applying the proposedmethods, a similar reduction of spine density and length byshLUCI as those identified by the manual analysis can bedetected. These results, therefore, validate the proposedalgorithm as a valuable tool for automatic analysis ofneuronal morphology and the identification of biologically

relevant changes in dendritic spine morphology andnumber.

Discussion and Conclusion

In this paper we propose a new OMRF-KICM basedmethod which can efficiently improve the automaticdetection results of existing algorithms. With the intensitydistribution information of background, dendrite and spinepixels, as well as the context and orientation information ofthe neurons, OMRF-KICM method can obviously improvethe detection performance for image with different pixelsizes. Specifically, according to the results in Table 1, wehave demonstrated that the false positive and the falsenegative rates of spine detection can be decreased by morethan 50% after applying the proposed method. There aremainly two motivations to improve the detection perfor-mance for images with larger pixel sizes. First, larger fieldof view can be obtained for images with larger pixel sizes.Second, because the already existed large amounts ofexperimental data with larger image pixel sizes, it wouldbe very time consuming and a waste of resources if all theimages have to be acquired again with smaller image pixelsizes.

The spine density measurement nevertheless showed alittle improvement compared with the adaptive thresholdingalgorithm. This is because the total number of automaticallydetected spines is equal to the number of (actual spines+false positives—false negatives.) We can see from Table 1that although the false negatives are reduced by 50%, thenumbers of total detected spines do not change muchbecause of the simultaneously reduced false positives. Themeasurement of the spine length however is muchimproved for images with large pixel size because of themore accurate detection of small spines and the removal ofthe long false spines.

Furthermore, the approach also holds promise in termsof providing better representation of the true shapes of thespines. The weak parts of the spines, especially the neck

Table 1 Comparison of spine detection results. Method 1 and 2 areglobal and adaptive thresholding based algorithm respectively, aftercombining broken spine parts. Method 1+ and 2+ are the above twomethods with applying the proposed algorithm. Images are acquired

with two different pixel sizes, 0.125 micron/ pixel (Image 1, 3, 4) and0.08 micron/pixel (Image 2, 5). The numbers in the brackets are thetotal number of spines in the respective images

Image 1 (82) Image 2 (67) Image 3 (86) Image 4 (93) Image 5 (36)

FP/P FN/R FP/P FN/R FP/P FN/R FP/P FN/R FP/P FN/R

Method 1 8/90% 19/77% 1/99% 15/78% 12/86% 22/74% 4/96% 11/88% 2/94% 4/89%

Method 1 + 3/96% 9/89% 0/100% 7/90% 5/94% 8/91% 3/97% 6/94% 0/100% 2/94%

Method 2 3/96% 10/88% 2/97% 13/81% 6/93% 15/83% 5/95% 6/94% 0/100% 3/92%

Method 2 + 2/98% 6/93% 0/100% 6/91% 3/97% 5/94% 2/98% 2/98% 0/100% 2/94%

Fig. 8 Comparison of spine length distribution of manual results andresults obtained by the proposed method

168 Neuroinform (2010) 8:157–170

regions which are hard to detect, are better recognized. Wenotice that the dendritic trunks in the segmented images(Figs. 5 and 6) being processed by the proposed method aregenerally thicker than those without the processing. This ispartially because the proposed algorithm combines the localspatial information with the intensity distribution informa-tion for the segmentation. Generally, the neighboring pixelsare prone to be segmented as the same class, althoughdifferent weights are assigned for neighbors with differentorientations. The (2∼3 pixel) thicker dendritic trunks makeit easier to detect the spine necks, however, to a limitedextent: we notice that it is the central part of the spine neck,not the bottom part near to the dendritic trunks, has muchlower intensity compared with other parts of the spines,which is the culprit of the broken spine necks.

We want to point out that the proposed automaticmethod is not designed for providing accurate absolutemeasurements about the spines such as the length. Ourneurobiology collaborators are more interested in thecomparison between the measurements under differenceconditions. As long as the measurements can correctlyreproduce the changes, accurate absolute measurements areless of a concern.

The proposed MAP-OMRF method can efficientlyimprove the automatic detection result. However, itsperformance can be further enhanced. As we observe fromFig. 7a, two segmented spine blobs are attached to eachother. It is a challenging problem to automatically separatethe linked spines. Marker based watershed transformationor the level set algorithm can be the two potential choices tosolve this problem. Also, the efficiency of constraint 2 ofthe KICM algorithm depends on the accuracy of theestimated local dendrite width. In cases where the size ofone spine neck is abnormally large, the small nearby spinescannot be correctly detected. This problem needs to beaddressed in the future work.

Another interesting topic for the future work is theonline calculation of bq1 and bq2, which are the optimalestimation of θ1 and θ2 respectively. Suppose y is theobserved data and x is the distribution of different regions,bq2 can be estimated via the maximization of the posteriorprobability function p q2jy; xð Þ, which is calculated basedon the prior probability function p(θ2) and the conditionalprobability function p y; xjq2ð Þ according to the Bayestheorem. How to obtain p(θ2) is not straightforward,because of the random location of the dendrite and spine,as well as the inhomogeneous intensity in the images. MRFmodel based method is one of the possible approaches.Suppose bx is the optimal segmentation results, bq1 canbe obtained by solving the following MAP estimationproblem:

bx;bq1� �¼ argmax

x;q1P x; q1jbq2; y� �

ð22Þ

Assume that θ1 is uniformly distributed, using the lawsof conditional probabilities, Eq. 22 can be changed to

bx;bq1� �¼ argmax

x;q1P yjx;bq2� �

P xjq1ð ÞP q1ð Þ

¼ argmaxx;q1

P yjx;bq2� �P xjq1ð Þ

ð23Þ

It can be further divided into two sub-problems whichcan be estimated iteratively.

bqðkÞ1 ¼ argmaxq1

P yjbxðk�1Þ;bq2� �P bxðk�1Þjq1� �

ð24Þ

bx kþ1ð Þ ¼ argmaxx

P yjx;bq2� �P xjbqðkÞ1

� �ð25Þ

Many algorithms have been proposed to solve this problem,such as maximum likelihood, coding, pseudo-likelihood,expectation-maximization, mean field approximations, and

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

GUR shLUCI

sp

ine

de

ns

ity

(n

um

be

r o

f s

pin

es

pe

r µ

m)

Manual OMRF

0

0.2

0.4

0.6

0.8

1

1.2

GUR shLUCI

sp

ine l

en

gth

(µ

m)

Manual OMRFa bFig. 9 shRNA expression indu-ces retraction of dendrites.Method 1 and 2 are manual andresults of porposed methodrespectively. a Summary of dataof average spine density at4DPT for GUR and shLUCIneurons. b Summary of data ofaverage spine length at 4DPTfor GUR and shLUCI neurons

Neuroinform (2010) 8:157–170 169

least squares fit etc (Li 2001). However, how to find acomputationally efficient algorithm, which can efficientlycombine the prior knowledge of the neuron images, remainsa question to be investigated.

Information Sharing Statement

The image data used in this work are provided by ProfessorB.L. Sabatini's lab at Harvard Medical School. We areunable to release these data. The algorithm used in thiswork will be included in the next version of NeruonIQ(Neuron Image Quantitator). NeuronIQ is public availableand can be downloaded freely from http://www.cbi-tmhs.org/Neuroniq/index.htm.

Acknowledgments The testing images and manual results areprovided by Bernardo L. Sabatini’s lab. Thanks Dr. Fuhai Li for helptesting the codes. This research is funded by a NIH R01 LM008696Grant to STCW. Finally, the authors want to thank the reviewers fortheir many helpful comments.

References

Alvarez, V. A., Ridenour, D. A., & Sabatini, B. L. (2006). Retractionof synapses and dendritic spines induced by off-target effects ofRNA Interference. Journal of Neuroscience 26(30).

Awate, S. P., & Whitaker, R. T. (2007). Feature-Preserving MRIDenoising: A Nonparametric Empirical Bayes Approach. IEEETrans Med Imag, 26(9), 1242–1255.

Bai, W., Zhou, X., Ji, L., Cheng, J., & Wong, S. T. C. (2007).Automatic dendritic spine analysis in twophoton laser scanningmicroscopy images. Cytometry Part A, 71, 818–826.

Besag, J. (1986). On the Statistical Analysis of Dirty Pictures. Journalof the Royal Statistical Society Series B, 48(3), 259–302.

Besag, J. (1989). Towards Bayesian Image Analysis. Journal ofApplied Statistics, 16(3), 395–407.

Bilousova, T. V., Dansie, L., Ngo, M., Aye, J., Charles, J. R., Ethell,D. W., et al. (2009). Minocycline promotes dendritic spinematuration and improves behavioural performance in the fragileX mouse model. Journal of Medical Genetics, 46, 94–102.

Calhoun, V. D., Adali, T., Pearlson, G. D., & Pekar, J. J. (2001). Spatialand temporal independent component analysis of functional MRIdata containing a pair of task-related waveforms. Human BrainMapping, 13, 43–53.

Cheng, J., Zhou, X., Miller, E., Witt, R. M., Zhu, J., Sabatini, B. L., et al.(2007). A novel computational approach for automatic dendritespines detection in two-photon laser scanning microscopy. Journalof Neuroscience Methods, 165, 122–134.

D’Elia, C., Marrocco, C., Molinara, M., Possi, G., Scarpa, G., &Tortorella, F. (2004). Detection of microcalcifications clusters inmammograms through TS-MRF segmentation and SVM-basedclassification. Proceedings of the 17th International Conferenceon Pattern Recognition, 3, 742–745.

Descombes, X., Kruggel, F., & von Cramon, D. Y. (1998). Spatio-temporal fMRI analysis using Markov random fields. IEEETransactions on Medical Imaging, 17(6), 1028–1039.

Durand, C. M. (2007). Mutations in the gene encoding the synapticscaffolding protein SHANK3 are associated with autism spectrumdisorders. Nature Genetics, 39, 25–27.

Fan, J., Zhou, X., Dy, J. G., Zhang, Y., & Wong, S. T. C. (2009). Anautomated pipeline for dendrite spine detection and tracking of3D optical microscopy neuron images of in vivo mouse models.Neuroinformatics, 7, 113–130.

Fiala, J. C., Spacekb, J., & Harrisa, K. M. (2002). Dendritic SpinePathology: Cause or Consequence of Neurological Disorders.Brain Research Reviews, 39(1), 29–54.

Geman, S., & Geman, D. (1984). Stochastic Relaxation, Gibbs Distribu-tions, and the Bayesian Restoration of Images. IEEE TransactionsPatterns Analysis and Machine Intelligence, 6(6), 721–741.

Hebert, T. J. (1997). Fast iterative segmentation of high resolutionmedical images. IEEE Transactions on Nuclear Science, 44(3),1362–1367.

Juslin, A., & Tohka, J. (2006). Unsupervised segmentation of cardiacPET transmission images for automatic heart volume extraction.Proc. of the 28th IEEE EMBS ann. int. conf. Aug. 1:1077–80.

Knobloch, M., & Mansuy, I. M. (2008). Dendritic spine loss andsynaptic alterations in Alzheimer’s disease. Molecular Neurobi-ology, 37(1), 73–82.

Koh, I. Y., Lindquist, W. B., Zito, K., Nimchinsky, E. A., & Svoboda,K. (2002). An image analysis algorithm for dendritic spines.Neural Computation, 14, 1283–1310.

Li, H. D., Kallergi, M., Clarke, L. P., Jain, V. K., & Clark, R. A. (1995).Markov random field for tumor detection in digital mammography.IEEE Transactions Medical Imaging, 14(3), 565–576.

Li, S. Z. (2001). Markov Random Field Modeling in Image Analysis(2nd ed.). Tokyo, Japan: Springer-Verlag.

Moser, M. B., Trommald, M., & Andersen, P. (1994). An increase indendritic spine density on hippocampal CA1 pyramidal cellsfollowing spatial learning in adult rats suggests the formation ofnew synapses. Proceedings of the National Academy of Science,91, 12673–12675.

Pappas, T. N. (1992). An adaptive clustering algorithm for imagesegmentation. IEEE Transactions on Signal Processing, 40(4),901–914.

Rajapakse, J. C., Wang, Y., Zheng, X., & Zhou, J. (2008).Probabilistic Framework for Brain Connectivity From FunctionalMR Images. IEEE Transactions on Medical Imaging, 27(6),825–833.

Salli, E., Aronen, H. J., Savolainen, S., Korvenoja, A., Visa, A.(2001). Contextual clustering for analysis of functional MRI data.IEEE Transaction on Medical Imaging 20(5):403–14, May.

Tavazoie, S. F., Alvarez, V. A., Ridenour, D. A., Kwiatkowski, D. J.,& Sabatini, B. L. (2005). Regulation of neuronal morphologyand function by the tumor suppressors Tsc1 and Tsc2. NatureNeuroscience, 8(12), 1727–1734.

Wahlby, C., Lindblad, J., Vondrus, M., Bengtsson, E., & Bjorkesten, L.(2002). Algorithms for cytoplasm segmentation of fluorescencelabeled cells. Analytical Cellular Pathology 24.

Weaver, C. M., Hof, P. R., Wearne, S. L., & Lindquist, W. (2004).Automated algorithms for multiscale morphometry of neuronaldendrites, Neural Computation, 16, 1353–1383.

Xiao, G., Brady, M., & Noble, J. A. (2002). Segmentation ofultrasound B-mode images with intensity inhomogeneitycorrection. IEEE Transactions on Medical Imaging, 21(1),48–57.

Xu, X., Cheng, J., Witt, R. M., Sabatini, B. L., & Wong, S. T. C.(2006). A shape analysis method to detect dendritic spine in 3Doptical microscopy image. Biomedal Imaging: Macro Nano, 3 rdIEEE International Symposium, 6, 554–557.

Yuste, R., & Bonhoeffer, T. (2001). Morphological changes indendritic spines associated with long-term synaptic plasticity.Annual Review of Neuroscience, 24, 1071–1089.

Zhang, Y., Zhou, X.,Witt, R.M., Sabatini, B. L., Adjeroh, D., &Wong, S.T. C. (2007). Dendritic spine detection using curvilinear structuredetector and LDA classifier. Neuroimag, 36, 346–360.

170 Neuroinform (2010) 8:157–170