4/11/2016
1
Whole Slide Imaging in imaging research and clinical research
Yukako Yagi, PhD [email protected]
Director of the MGH Pathology Imaging & Communication Technology Center
Assistant Professor of Pathology, Harvard Medical School
Affiliate Faculty, Wellman Center for Photomedicine, MGH
ACCME/DisclosuresThe USCAP requires that anyone in a position to
influence or control the content of CME disclose any relevant financial relationship WITH COMMERCIAL INTERESTS which they or their spouse/partner have, or have had, within the past 12 months, which relates
to the content of this educational activity and creates a conflict of interest.
Dr. Yukako Yagi declares she has no conflict(s) of interest to disclose.
Topics
• WSI Based Image analysis often requestedby pathologists
• IHC analysis• Tissue & tumor detection & measurement• FISH analysis
4/11/2016
2
Backgrounds• Through Imaging Research Core lab work, we are often asked if we could do…. by pathologists.
• Usually it is urgent. • If the solution is clear and simple, we work with them to generate the desired results by developing a small program, macro program, or a manual protocol. It is still helpful for pathologists.
• Sometimes, the scale of the project becomes larger• 3D (100‐1000s slides) imaging are often requested recently for further understanding of disease
• Confocal WSI scanner is available. Single slide 3D analysis is becoming popular
Application: small project• Whole slide imaging (WSI)
• Digitizing conventional glass slides
Image Analysis
Glass slide
WSI scanner
Digital slide
Some commercial software are available but limited or expensive. We often need a quick and less expensive
solution.
• Standardize the color of stain• Standardize the color of images• Improve the image quality• Remove artifacts & ink
• Measure the tumor size and ratio vs. entire tissue• Nuclei counting within tumor• Positive cell counting within tumor • Nuclei density within an area• IHC evaluation (Nuclei staining, membrane staining, cytoplasmic staining)• Double stain of IHC evaluation• FISH analysis
Examples of Requests from pathologists• Standardize the color of stain• Standardize the color of images• Improve the image quality• Remove artifacts & ink
• Measure the tumor size and ratio vs. entire tissue• Nuclei counting within tumor• Positive cell counting within tumor • Nuclei density within an area• IHC evaluation (Nuclei staining, membrane staining, cytoplasmic staining)• Double stain of IHC evaluation• FISH analysis
Examples of Requests from pathologists
4/11/2016
3
Simple Color Normalization
Purposes: to improve the appearance, for image analysis
Simple Color Normalization
Drug images to normalize in the folder “ColorStd.exe” into an arbitrary folder with reference color of image
Results
Color normalized images are saved in “ouputimage” automatically
Any file name is availablelimited to “.tif”, “.Jpg”, “.png” or “.bmp”
Original Standardized image
Reference
4/11/2016
4
Application in Follicular Lymphoma Follicular lymphoma is the most
common of the indolent non‐Hodgkin's lymphomas, and the second‐most‐common form of non‐Hodgkin's lymphomas overall. It is defined as a lymphoma of follicle center B‐cells (centrocytesand centroblasts), which has at least a partially follicular pattern. It is positive for the B‐cell markers CD10, CD19, CD20, and CD22 but almost always negative for CD5.
There are several synonymous and obsolete terms for this disease, such as CB/CC lymphoma (Centroblastic and Centrocyticlymphoma), nodular lymphoma and Brill‐Symmers Disease.
Counting Ki‐67 Positive cell within follicules
BCL‐6: to see Follicules
Ki‐67 H&E
1. Scan 3 stained slides with WSI scanner, 2. Select 20 follicules per case
BCL‐6
Ki‐67 H&E Easier to find follicule with H&E or BCL‐6 WSI. Annotate 20 follicules to export
Counting Ki‐67 Positive cell within follicules
# of negative cell vs # of positive cell
Difficulty:•Staining•Tissue thickness•Area of follicule•Size of follicule
4/11/2016
5
Protocol• Semi‐auto1. Scan Ki‐67 slide
2. Select folliculles (20) and export
3. Color standardization
4. Run “ImmunoRaitio” software*
5. Average
*ImageJ Plug‐Inhttp://jvsmicroscope.uta.fi/sites/def
ault/files/software/immunoratio‐plugin/index.html
We have analyzed many cases and results were approved by pathologists.
Counting Ki‐67 Positive cell for Breast tissue
However, they are non‐specific with respect to cell type and most often skew results by including stromal cells in the index. This problem is particularly relevant in breast cancer, where tumors are often located in a field of fibro‐adipose tissue. We have developed “stromal cell filtration algorithm” in improve the results of immunoratio.
Stromal FiltersCircularity Measure‐ Hu Moment Invariant
• Not dependent on scale, translation, rotation
• Deals better with boundary deformations (blobbier cell nuclei)
4/11/2016
6
Meta‐analysis of results show a better correlation with pathologist scores than without filter (.84 vs .71)
Comparisonwithout stromal filter with stromal filter
Pathologist annotates tumor area on WSI then analyze entire annotation with 10‐20x image
IHC Positive cell density within entire Tumor
One 20x image on a screen
Digitize glass slide with Nanozoomer
Annotate tumor in NDP.view1.2
Export image
Select tumor and measure size in ImageJ
Segment CD8+ TILs with color segmentation* Image has to be
divided into multiple images for analysis
Quantify CD8+ TILs, and calculate density
8.761 mm24579 cells;
522.66 cells/mm2
IHC Positive cell density within TumorManual process with imageJ
Important instruction from the pathologist
4/11/2016
7
Segment the cells by going to “Plugins>Color Segmentation”, select the CD8 cells, nuclei, and background.
IHC Positive cell density within TumorManual process with imageJ
After run “color segmentation”
Convert the above image into 8‐bit by going to “Image>Type>8‐bit”
Next, do threshold segmentation by going to “Image>Adjust>Threshold”;There should be 3 intensity peaks in the histogram, and adjust the threshold slides so that only one peak associated with one specific color is highlighted. Click “Apply”
Separate the touching cells by going to “Process>Binary>Watershed”
Count the cells by going to “Analyze>Analyze Particles”
4/11/2016
8
Follow the same steps to count the total cell number by counting nuclei
Calculate the density of CD8 cells: 203/0.739mm2=274.696 CD8 cells/mm2
Also, it can calculate the ratio of CD8 positive cells : 203/884*100=22.96%
• Based on the annotation and results by imageJ, we are working on developing the automated version.
IHC Positive cell density within TumorAutomated Algorithm Development
Digitize glass slide with Nanozoomer
Annotate tumor in NDP.view1.2
The algorithm looks at the coordinates of annotation
Annotated area is divided into 1024x1024 with 20x (0.46um/pixel)
Segment CD8+ TILs with per piece of image
Quantify CD8+ TILs, and calculate density
result per slide
8.761 mm2
IHC Positive cell density within TumorAutomated Algorithm Development
Segment CD8+ TILs with per piece of image
IHC Positive cell density within TumorAutomated Algorithm Development
Total Results are saved in an excel data
4/11/2016
9
IHC Double Stained WSI analysis
We often received request from researchers to analyze IHC stained WSIs. Most IHC stained slides in research areas are not in very good condition. Especially, double stained slides where we can observe greater staining variability between slides. The method of analysis is also different per project. Pathologist will explain clearly what and how she/he wants to analyze the image. We investigate the best method and develop a protocol then analyze
WSI
Annotation by a pathologist
Analyze
Develop a protocol
Example: Double nuclei stains analysis: positive nuclei density inPeritumoral area vs Tumor area. Pathologist annotates tumor and peritumoral areas.
Selected areas are analyzed by imageJ.
Distribution of tumor in entire tissue
Many requests to calculate % of tumor area within entire tissue.
Distribution of tumor in entire tissue
14.5%
4/11/2016
10
R1R2
R3
Region No. of Pixels
Area in mm2
R1 628960 33.334
R2 504910 26.759
R3 446420 23.660
R1R2
R3
Automated whole tissue measurement & tumor detection using WSI
• Incorporated in a WSI viewer • R1 Largest tissue• Results of entire data set are
in an excel sheet Pixel resolu Total tumore area
mm2265317 18868.49414 14.06137649
• Incorporated in a WSI viewer
• User can edit the area • The purpose is not to
detect tumor accurately.
• The main purpose is to help pathologist to annotate tumor area
Tumor detection is still under development
R1
R2
R3
R4
R5
R6
Region No. of Pixels Area in mm2
R1 1000100 53.004
R2 359880 19.073
R3 152050 8.058
R4 132510 7.023
R5 3412 0.181
R6 3354 0.178
R1
R2
R3
R4
524012 18868.49414 27.77179758
Pixel resolu Total tumore areamm2
4/11/2016
11
R1
R2
R3
R4R5
R6
Region No. of Pixels
Area in mm2
R1 476600 25.259
R2 437480 23.186
R3 332800 17.638
R4 283890 15.046
R5 25559 1.355
R6 3511 0.186
183810 18868.49414 9.741635904
Pixel resolu Total tumore areamm2
FISH Analysis by Confocal WSI scanner
Z Y
X
(Z axis was expanded)
Y Z
X
Y Z
X
Z Y
X
FITC 4
AQUA 2
TRITC 2
CY5 3
4/11/2016
12
Dots quantification of extended focus scanned image with Imaris
FITC dots: 39 TRITC dots: 37
FITC dots: 43
TRITC dots: 38
z‐stack
Extended
Z‐stackExtended focus
Extended focus may decrease spots number due to overlapping
Lateral view
FITC TRITC
Nuclei colocatednon‐colocated Total colocated non‐colocated Total
1 0 0 0 0 0 02 0 0 0 0 1 13 0 0 0 0 0 04 0 3 3 0 3 35 1 3 4 1 2 36 0 3 3 0 3 37 0 0 0 0 2 28 0 2 2 0 0 09 0 2 2 0 3 310 0 1 1 0 2 211 0 2 2 0 1 112 0 1 1 0 0 013 0 1 1 0 0 014 0 3 3 0 3 315 0 2 2 0 2 216 0 0 0 0 0 017 0 3 3 0 5 518 0 1 1 0 1 119 0 2 2 0 1 120 0 1 1 0 1 121 0 1 1 0 1 122 0 0 0 0 0 023 0 1 1 0 0 024 0 1 1 0 1 125 0 1 1 0 1 126 0 2 2 0 1 127 0 0 0 0 1 128 0 3 3 0 2 229 0 3 3 0 0 0
Total 1 42 43 1 37 38
Quantification of dots in different channels in each nucleus (using Z‐stack data)
FITC TRITCNuclei colocated non‐colocated Total colocated non‐colocated Total
1 1 3 4 1 2 32 1 2 3 2 1 33 1 2 3 1 0 14 0 2 2 0 1 15 0 3 3 0 1 16 0 6 6 0 1 17 2 3 5 2 2 48 0 4 4 0 0 09 0 1 1 0 0 0
10 2 3 5 2 2 411 0 0 0 0 0 012 2 4 6 2 3 513 0 1 1 0 0 014 2 4 6 2 5 715 3 3 6 3 5 816 3 3 6 3 1 417 4 1 5 4 2 618 0 2 2 0 1 119 3 3 6 3 4 7
Total 24 50 74 25 31 56
Quantification of dots in different channels in each nucleus (Case 2)
4/11/2016
13
Summary
• Simple, user friendly and affordable image analysis tools for WSI are still unavailable in most situation even at an academic medical center.• Combination of free share software and small programming will have the capabilities to analyze the WSI data easily. Developing the detailed protocol for each project is important to successfully obtain the desired results, especially in research.• Developed protocols are very helpful to develop automated image analysis modules• Currently each algorithm work independently. We could integrate useful algorithm into another algorithm effectively in future.
Acknowledgements• This research was partially supported by 3DHISTECH, BITPLANE, and Hamamatsu Photonics. • Authors acknowledge to all the collaborators, Anthony Bui, Kunal Patel, Joseph Roberto, Michael Senter‐Zapata, Drs. Abner Louissaint, Elena Brachtel, Mari Mino‐Kenudson, Maristela Onozato, John A Iafrate, Veronica Klepies, Gregory Riedlinger, John Iafrate, Pinky Bautista, Noriaki Hashimoto, XiuJun Fu, Bruce Levy.