Embed Size (px)
DESCRIPTION
Automated detection of TNT in cell images. Automated detection of TNT in cell images. Automated detection of TNTs( T unnelling N ano T ubes) in cell images. Automated detection of TNTs( T unnelling N ano T ubes) in cell images. Erlend Hodneland, Arvid Lundervold, Xue-Cheng Tai, - PowerPoint PPT Presentation
Citation preview
Erlend Hodneland, University of Bergen
Automated detection of Automated detection of TNT in cell images.TNT in cell images.
Erlend Hodneland, University of Bergen
Automated detection of Automated detection of TNT in cell images.TNT in cell images.
Erlend Hodneland, University of Bergen
Automated detection of Automated detection of TNTs(TNTs(TTunnelling unnelling
NNanoanoTTubes) in cell imagesubes) in cell images
Erlend Hodneland, University of Bergen
Automated detection of Automated detection of TNTs(TNTs(TTunnelling unnelling
NNanoanoTTubes) in cell imagesubes) in cell images
Erlend Hodneland, Arvid Lundervold, Xue-Cheng Tai, Steffen Gurke, Amin Rustom, Hans-Hermann Gerdes.
Erlend Hodneland, University of Bergen
3D session at fluorescence 3D session at fluorescence microscopemicroscope
Dimension : Dimension : 520x688x40520x688x40
Better resolution Better resolution in xy plane than in in xy plane than in z direction.z direction.
Erlend Hodneland, University of Bergen
Two image channelsTwo image channels
The channels appear The channels appear from biological from biological stainings of sample.stainings of sample.
The stainings are The stainings are photo sensible to photo sensible to specific wavelengths specific wavelengths and accumulate in and accumulate in certain certain compartmens of the compartmens of the cells.cells.
100 200 300 400 500 600
50
100
150
200
250
300
350
400
450
500
100 200 300 400 500 600
50
100
150
200
250
300
350
400
450
500
Erlend Hodneland, University of Bergen
First channel displaying First channel displaying cell borders and TNTscell borders and TNTs
Erlend Hodneland, University of Bergen
Gaussian noise and Gaussian noise and undesired structuresundesired structures
Erlend Hodneland, University of Bergen
Video of image stackVideo of image stack
510
1520
2530
3540
100 200 300 400 500 600
Erlend Hodneland, University of Bergen
Second channel displaying Second channel displaying cell cytoplasmacell cytoplasma
Erlend Hodneland, University of Bergen
Biological relevance of Biological relevance of TNTsTNTs
TNTs are until TNTs are until recently unknown recently unknown cell structures.cell structures.
Play a role in cell Play a role in cell to cell to cell communication.communication.
Transport of virus?Transport of virus? Spread of cancer?Spread of cancer?
Virus moving?
Cell 1 Cell 2
Erlend Hodneland, University of Bergen
Automated detection of Automated detection of TNTsTNTs
A very challenging problem due to large A very challenging problem due to large variability between images.variability between images.
The basis methods are built up around The basis methods are built up around Zerocross and Canny edgedetectors.Zerocross and Canny edgedetectors. Morphology incl. Watershed Morphology incl. Watershed
segmentation, binary filling, segmentation, binary filling, dilation, erosion, closing and dilation, erosion, closing and opening.opening.
Erlend Hodneland, University of Bergen
Morhpological Morhpological operators*operators*
*Serra, J 1982, Image analysis and mathematical morphology., Academic Press.
}.Bb allfor ,{
as defined is of Reflection
}.for ,{(A)
as defined
,element by Translate
x
bxxB
B
Aaxacc
xA
r
Erlend Hodneland, University of Bergen
Morhpological Morhpological operators*operators*
*Serra, J 1982, Image analysis and mathematical morphology., Academic Press.
20 40 60 80 100 120 140 160 180 200
20
40
60
80
100
120
140
160
180
200
20 40 60 80 100 120 140 160 180 200
20
40
60
80
100
120
140
160
180
200
}))(({),(:Dilation ØABxBAD xr
}.Bb allfor ,{
as defined is of Reflection
}.for ,{(A)
as defined
,element by Translate
x
bxxB
B
Aaxacc
xA
r
Erlend Hodneland, University of Bergen
Morhpological Morhpological operators*operators*
*Serra, J 1982, Image analysis and mathematical morphology., Academic Press.
}.Bb allfor ,{
as defined is of Reflection
}.for ,{(A)
as defined
,element by Translate
x
bxxB
B
Aaxacc
xA
r
20 40 60 80 100 120 140 160 180 200
20
40
60
80
100
120
140
160
180
200
20 40 60 80 100 120 140 160 180 200
20
40
60
80
100
120
140
160
180
200
})({),(:Erosion ABxBAE x
Erlend Hodneland, University of Bergen
Morhpological Morhpological operators*operators*
*Serra, J 1982, Image analysis and mathematical morphology., Academic Press.
}.Bb allfor ,{
as defined is of Reflection
}.for ,{(A)
as defined
,element by Translate
x
bxxB
B
Aaxacc
xA
r
20 40 60 80 100 120 140 160 180 200
20
40
60
80
100
120
140
160
180
200
20 40 60 80 100 120 140 160 180 200
20
40
60
80
100
120
140
160
180
200
)),,((),( : Opening BBAEDBAO
Erlend Hodneland, University of Bergen
Morhpological Morhpological operators*operators*
*Serra, J 1982, Image analysis and mathematical morphology., Academic Press.
}.Bb allfor ,{
as defined is of Reflection
}.for ,{(A)
as defined
,element by Translate
x
bxxB
B
Aaxacc
xA
r
20 40 60 80 100 120 140 160 180 200
20
40
60
80
100
120
140
160
180
200
20 40 60 80 100 120 140 160 180 200
20
40
60
80
100
120
140
160
180
200
)),,((),(: Closing BBADEBAC
Erlend Hodneland, University of Bergen
Step #1 : Find cellular Step #1 : Find cellular regionsregions
Using canny edge Using canny edge detector to find detector to find borders of cells.borders of cells.
Edge detectors Edge detectors create lots of create lots of broken parts, we broken parts, we need to combine need to combine these parts.these parts.
50 100 150 200 250 300
50
100
150
200
250
Erlend Hodneland, University of Bergen
Step #1 : Find cellular Step #1 : Find cellular regionsregions
Use morphological Use morphological closing and closing and dilation to combine dilation to combine edges into closed edges into closed regions.regions.
50 100 150 200 250 300
50
100
150
200
250
50 100 150 200 250 300
50
100
150
200
250
Dilationand closing
Erlend Hodneland, University of Bergen
Step #1 : Find cellular Step #1 : Find cellular regionsregions
Use morphological filling Use morphological filling to fill closed regions.to fill closed regions.
Cells shown as white, Cells shown as white, filled regions. filled regions.
50 100 150 200 250 300
50
100
150
200
250
50 100 150 200 250 300
50
100
150
200
250
50 100 150 200 250 300
50
100
150
200
250
Filling
Erlend Hodneland, University of Bergen
Step #1 : Find cellular Step #1 : Find cellular regionsregions
3-D representation of 3-D representation of binary cell image. binary cell image.
50 100 150 200 250 300
50
100
150
200
250
CombineAll planes.
Erlend Hodneland, University of Bergen
50 100 150 200 250 300
50
100
150
200
250
Step #2 Find important Step #2 Find important edges in cell border edges in cell border
channelchannel The first channel The first channel
displays TNTs and displays TNTs and cell borders.cell borders.
TNTs have low TNTs have low intensities intensities compared to cell compared to cell borders but they borders but they have a large have a large gradient.gradient.
50 100 150 200 250 300
50
100
150
200
250
Erlend Hodneland, University of Bergen
Step #2 Find important Step #2 Find important edges in cell border edges in cell border
channelchannel Remove edges Remove edges
inside cells.inside cells.
50 100 150 200 250 300
50
100
150
200
250
50 100 150 200 250 300
50
100
150
200
250
50 100 150 200 250 300
50
100
150
200
250
Erlend Hodneland, University of Bergen
Watershed segmentationWatershed segmentation
A segmentation A segmentation procedure specially procedure specially designed for images designed for images with natural minima.with natural minima.
A reliable A reliable segmentation segmentation method, but it needs method, but it needs suitable minima suitable minima regions as input for regions as input for the region growing.the region growing.
100 200 300 400 500 600
50
100
150
200
250
300
350
400
450
500
0 100 200 300 400 500 600 700200
300
400
500
600
700
800
900
1000
1100
Background
Cell Cell
Erlend Hodneland, University of Bergen
Watershed segmentationWatershed segmentation
Pathwise criterion Pathwise criterion of Watershed lines of Watershed lines WW::
For all For all AAii(a,b), (a,b),
minmin(W(a,b)) ≥ (W(a,b)) ≥ minmin(A(Aii(a,b))(a,b))
””Moving on the top of Moving on the top of the hill”the hill”
a
b
Region 1
Region 2
Region 3
min(Ai(a,b))min(Ai(a,b))
min(W(a,b))min(W(a,b))
Erlend Hodneland, University of Bergen
Watershed segmentationWatershed segmentation
The minima The minima seeding regions seeding regions are extremely are extremely important and important and decide where the decide where the watershed lines watershed lines will appear.will appear.
50 100 150 200 250 300
50
100
150
200
250
50 100 150 200 250 300
50
100
150
200
250
50 100 150 200 250 300
50
100
150
200
250
Minima regions Minima imposed on image
Erlend Hodneland, University of Bergen
Watershed segmentationWatershed segmentation
Results improve Results improve when the minima when the minima seeding regions seeding regions are close to the are close to the crest of the desired crest of the desired structures.structures.
50 100 150 200 250 300
50
100
150
200
250
50 100 150 200 250 300
50
100
150
200
250
50 100 150 200 250 300
50
100
150
200
250
1
2
3
4
5
6
7
Watershed image, {1,2 … 7} The boundaries of cells
Erlend Hodneland, University of Bergen
Watershed segmentationWatershed segmentation
Results improve Results improve when the minima when the minima seeding regions seeding regions are close to the are close to the crest of the desired crest of the desired structures.structures.
50 100 150 200 250 300
50
100
150
200
250
50 100 150 200 250 300
50
100
150
200
250
1
2
3
4
5
6
7
Watershed image, {1,2 … 7} The boundaries of cells
50 100 150 200 250 300
50
100
150
200
250
Erlend Hodneland, University of Bergen
Step #3 Watershed Step #3 Watershed segmentation to find crest segmentation to find crest of structures from edgesof structures from edges
TNTs are thin and TNTs are thin and narrow, narrow, approximately 3-4 approximately 3-4 pixles wide (50-pixles wide (50-200nm).200nm). 50 100 150 200 250 300
50
100
150
200
250
10 20 30 40 50 60 70 80
10
20
30
40
50
60
70
80
1
1
TNT
Erlend Hodneland, University of Bergen
Step #3 Watershed Step #3 Watershed segmentation to find crest segmentation to find crest of structures from edgesof structures from edges
Problem : The Problem : The structures from the structures from the edge image are not edge image are not always continuous and always continuous and they are not marking they are not marking the crest of the the crest of the structure.structure.
Solution : Use Solution : Use watershed watershed segmentation to create segmentation to create connected lines on the connected lines on the crest of the high crest of the high intensity structures.intensity structures.
50 100 150 200 250 300
50
100
150
200
250
50 100 150 200 250 300
50
100
150
200
250
Erlend Hodneland, University of Bergen
50 100 150 200 250 300
50
100
150
200
250
Step #3 Watershed Step #3 Watershed segmentation to find crest segmentation to find crest of structures from edgesof structures from edges
Problem : The Problem : The structures from the structures from the edge image are not edge image are not always continuous always continuous and they are not and they are not marking the crest marking the crest of the structure.of the structure.
10 20 30 40 50 60 70 80
10
20
30
40
50
60
70
80
1
10 20 30 40 50 60 70 80
10
20
30
40
50
60
70
80
TNT
1
Erlend Hodneland, University of Bergen
Step #3 Watershed Step #3 Watershed segmentation to find crest segmentation to find crest of structures from edgesof structures from edges
Important: TNTs Important: TNTs can cross several can cross several planes.planes.
Therefore we use a Therefore we use a projection in 3-D projection in 3-D 2-D to include the 2-D to include the whole TNT.whole TNT.
All projections are All projections are ranging over the ranging over the same planes as same planes as the structure we the structure we investigate.investigate.
Cell 2
x
y
z
Cell 1TNT
10
20
Erlend Hodneland, University of Bergen
Step #3 Watershed Step #3 Watershed segmentation to find crest segmentation to find crest of structures from edgesof structures from edges
20 40 60 80 100 120 140
20
40
60
80
100
120
140
20 40 60 80 100 120 140
20
40
60
80
100
120
140
20 40 60 80 100 120 140
20
40
60
80
100
120
140
20 40 60 80 100 120 140
20
40
60
80
100
120
140
Plane 10 Plane 11
Plane 12 Plane 13
Erlend Hodneland, University of Bergen
Step #3 Watershed Step #3 Watershed segmentation to find crest segmentation to find crest of structures from edgesof structures from edges
Using the Using the maximum maximum projectionprojection of the of the structure from the structure from the edge image to take edge image to take advantage of 3-D advantage of 3-D information.information.
10 20 30 40 50 60 70 80
10
20
30
40
50
60
70
80
1
TNT
10 20 30 40 50 60 70 80
10
20
30
40
50
60
70
80
Maximumprojection and closing
10 20 30 40 50 60 70 80
10
20
30
40
50
60
70
80
Erlend Hodneland, University of Bergen
Step #3 Watershed Step #3 Watershed segmentation to find crest segmentation to find crest of structures from edgesof structures from edges
We use We use information from information from the segmentation the segmentation of cells to of cells to construct construct minima minima regionsregions to seed to seed the Watershed the Watershed segmentation.segmentation.
10 20 30 40 50 60 70 80
10
20
30
40
50
60
70
80
10 20 30 40 50 60 70 80
10
20
30
40
50
60
70
80
Background
Cells
TNT
50 100 150 200 250 300
50
100
150
200
250
Erlend Hodneland, University of Bergen
Step #3 Watershed Step #3 Watershed segmentation to find crest segmentation to find crest of structures from edgesof structures from edges
We use We use information from information from the segmentation the segmentation of cells to of cells to construct construct minima minima regionsregions to seed to seed the Watershed the Watershed segmentation.segmentation.
10 20 30 40 50 60 70 80
10
20
30
40
50
60
70
80
TNT
10 20 30 40 50 60 70 80
10
20
30
40
50
60
70
80
10 20 30 40 50 60 70 80
10
20
30
40
50
60
70
80
10 20 30 40 50 60 70 80
10
20
30
40
50
60
70
80
Morphologicalopening
Impose (1) on (2)
1 2Minima regions
Erlend Hodneland, University of Bergen
Step #3 Watershed Step #3 Watershed segmentation to find crest segmentation to find crest of structures from edgesof structures from edges
For Watershed we For Watershed we use the use the sum sum projectionprojection of the of the image to take image to take advantage of 3-D advantage of 3-D information and for information and for Gaussian noise Gaussian noise supression.supression.
10 20 30 40 50 60 70 80
10
20
30
40
50
60
70
80
TNT
Sum projection
10 20 30 40 50 60 70 80
10
20
30
40
50
60
70
80
Image stack
Erlend Hodneland, University of Bergen
10 20 30 40 50 60 70 80
10
20
30
40
50
60
70
80
Step #3 Watershed Step #3 Watershed segmentation to find crest segmentation to find crest of structures from edgesof structures from edges Using Watershed Using Watershed
segmentation to segmentation to achieve a connected achieve a connected line on the crest of the line on the crest of the structure from the structure from the edge image.edge image.
TNT
10 20 30 40 50 60 70 80
10
20
30
40
50
60
70
80
Watershed segmentation
10 20 30 40 50 60 70 80
10
20
30
40
50
60
70
80
TNT
Minimaregions
Erlend Hodneland, University of Bergen
Step #4 Removal of false Step #4 Removal of false TNT candidatesTNT candidates
We end up with We end up with numerous TNT numerous TNT candidates, some candidates, some false and some false and some true.true. 50 100 150 200 250 300
50
100
150
200
250
50 100 150 200 250 300
50
100
150
200
250 TNT candidates
Watershedsegmentation
50 100 150 200 250 300
50
100
150
200
250
Erlend Hodneland, University of Bergen
Step #4 Removal of false Step #4 Removal of false TNT candidatesTNT candidates
Each TNT candidate must undergo an Each TNT candidate must undergo an evaluation of correctedness. Remove evaluation of correctedness. Remove candidatescandidates having low intensities compared to their having low intensities compared to their
surroundings.surroundings. not crossing between two cells.not crossing between two cells. not beeing straigth lines using hough not beeing straigth lines using hough
transformation.transformation. crossing at the nearest distance of the cells.crossing at the nearest distance of the cells.
We are left with ”true” TNT structures after We are left with ”true” TNT structures after the exclusion evaluation.the exclusion evaluation.
Erlend Hodneland, University of Bergen
ResultsResults
We have employed our algorithm to 51 3-We have employed our algorithm to 51 3-D image stacks: D image stacks: Success rate 67%Success rate 67% False positive 50%False positive 50% False negative 33%False negative 33%
compared to manual counting.compared to manual counting. The high number of false positive TNTs is The high number of false positive TNTs is
mostly due to large image variations and mostly due to large image variations and irregularities of the cells.irregularities of the cells.
Erlend Hodneland, University of Bergen
ResultsResults
Large Large irregularites. irregularites.
Main reason for Main reason for false positive or false positive or false negative false negative TNTs.TNTs.
50 100 150 200 250 300
50
100
150
200
250
300
350
Erlend Hodneland, University of Bergen
ResultsResults
20 40 60 80 100 120 140 160 180 200
20
40
60
80
100
120
140
160
180
200
Large Large irregularites. irregularites.
Main reason for Main reason for false positive or false positive or false negative false negative TNTs.TNTs.
Erlend Hodneland, University of Bergen
ResultsResults
Nano experiments Nano experiments to grow the cells to grow the cells on pre-defined on pre-defined matrices.matrices.
This will improve This will improve the automated the automated detection.detection.
Erlend Hodneland, University of Bergen
ConclusionConclusion
We have developed an automated method We have developed an automated method for counting TNTs in cell images. for counting TNTs in cell images.
The method is essentially based on The method is essentially based on existing image processing techniques like existing image processing techniques like edge-detectors, watershed segmentation edge-detectors, watershed segmentation and morphological operators.and morphological operators.
We report a success rate of 67% We report a success rate of 67% compared to manual counting.compared to manual counting.
