Upload
valerie-moore
View
216
Download
1
Tags:
Embed Size (px)
Citation preview
A multispectral image enhancement approach to visualize tissue structures
Pinky A. Bautista1, Tokiya Abe1, Yukako Yagi1, John Gilbertson1, Masahiro Yamaguchi2, and Nagaaki Ohyama2
1 Massachusetts General Hospital 2 Technology Institute of Technology
Multispectral Imaging (MSI)
Originally developed for space-based imaging
Multiple grey-level images are captured at different wavelengths
Allows extraction of additional information which the human eye fails to capture.
Filter sensitivities
3 grey-level images
N>3 grey-level images
RGB imaging Mutispectral imaging
MSI allows greater
flexibility for image analysis as compared to RGB imaging3 broadband
filtersN Narrowband filters
Objectives
To digitally enhance an H&E stained multispectral image such that collagen fiber can easily be differentiated from the rest of the eosin stained tissue components.
Show the capability of multispectral imaging to differentiate tissue structures with minute colorimetric difference.
Multispectral Microscope Imaging system
Olympus BX-62 optical microscope controlled by a PC
16 interference filters
2kx2k pixel CCD camera
*Used in the experiment
Enhancement Method yx,yx,yx,yx, t-tWtte
~
Reference: Masanori Mitsui, Yuri Murakami, Takashi Obi, et.al, “ Color Enhancement in Multispectral Image Using the Karhunen-Loeve Transform,” Optical Review Vol.12, no.2, pp.60-75, 2005
yx,t Original spectral transmittance at location x,y (16-band)
yx,et Enhanced version
estimated spectral transmittance using M (M<N) KL vectors derived from the transmittance data of the selected tissue components
Spectral residual error
W NxN weighting factor Matrix, i.e. N=16
tvrrt
M
1iii~
Controls the color of enhanced areas
Experiment
1. Training Phase 2. Testing Phase
Collection of 16-band transmittance spectra samples of the identified H&E stained tissue components
Derivation of the KL vectors
Identification of the appropriate number of KL vectors, i.e. M-KL vectors
Perform multispectral enhancement on 16-band images using the M-KL vectors derived in the training phase
Transform the multispectral enhanced image into its equivalent RGB format for visualization
Examine the spectral residual error characteristics of the
different tissue components
Derivation of KL vectors
KL vectors were derived from the transmittance of these tissue components
Transmittance spectra of the different tissue components
Training data
fiber
Subject for enhancement
Not Subject for enhancement
RGB format of the 16-band MS image of a
Heart tissue
NucleusCytoplasmRBCs, etc.
Tissue components transmittance spectra
0
0.2
0.4
0.6
0.8
1
1.2
1 3 5 7 9 11 13 15
Band number
Tra
nsm
ittan
ce v
alue Nucleus
Striated Muscle
Red blood cell (RBC)
White area
Collagen fiber
others
Each tissue component is represented with 200 transmittance spectra samples.
structures found in white areas
Spectral Residual Error
-0.03
-0.02
-0.01
0.00
0.01
0.02
0.03
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Band number
Ave
rag
e sp
ectr
al r
esid
ual
err
or
Nucleus
Striated muscle
Red blood cell (RBC)
White area
Collagen fiber
others
The spectral residual error for fiber peaks at band 8
structures found in white areas
Appropriate number of KL vectors was investigated……
5-KL vectors were found to produce distinct peaks on the spectral residual error of collagen fiber
Result (heart tissue)
H&E stained Digitally enhanced
Striated muscle and Collagen fiber which are both stained with Eosin in an H&E stained slide are impressed with different
shades of color when digitally enhanced
Collagen fiber
Striated muscle
Striated Muscle
Collagen fiber
2kx2k pixels 20x
magnification
Results
H&E stainedMT stained Digitally enhanced
Serial Section
referenceTissue areas highlighted in the digitally enhanced image correspond to areas
emphasized by the MT stain
Result (Magnified)
not clearly differentiated
differentiated differentiated
Original H&E stained
Enhanced H&E stained image
MT stained
Tissue structures with minute color difference is differentiated using Multispectral information
reference
RGB and Multispectral
Enhanced using RGB information
Enhanced using Multispectral
information
Original H&E
stained image
MT stained image
Serial Section
RGB and Multispectral Enhanced using RGB
informationEnhanced using
Multispectral information
Original H&E stained image
Not clearly differentiated Clearly differentiated
0
0.2
0.4
0.6
0.8
1
1.2
1 2 3 4 5 6 7 8 9 10 1112 1314 1516
Band number
Spe
ctra
l tra
nsm
ittan
ceStriated muscle
Collagen fiber1
Collagen fiber2
Spectral transmittance
There is a slight difference in the spectral configurations between the labeled fiber1 and fiber2 areas
Conclusion With multispectral imaging it is possible to differentiate tissue structures with minute colorimetric difference
The current enhancement scheme makes it possible to differentiate tissue structures that are less likely differentiated with RGB imaging
Future work
Work with more tissue images to validate the current result
Investigate further the meaning of spectral residual error in relation to tissue differentiation
Investigate possible application of the residual error configurations to select important bands to classify/segment specific tissue structures
We thank CAP foundation for making it possible for us to attend this conference.
THANK YOU….
Weighting matrix Variation
H&E stained Digitally enhanced
Color of target areas
can be varied by
manipulating the
weighting
matrix W
yx,yx,yx,yx, t-tWtte
~Spectral enhancement
Results
H&E stained MT stained Digitally enhanced
Serial Section
reference
Tissue areas highlighted in the digitally enhanced image correspond to areas emphasized by the MT stain
Result (kidney tissue)
Training data were extracted from another MS image of a kidney tissue; training and test images belong to the same slide
H&E stained Digitally enhanced
Result (kidney tissue)
H&E stained Digitally enhanced MT stained