21
Advances on Digital Image Processing in Pathology Gloria Bueno E.T.S.I. Industriales - UCLM http://visilab.etsii.uclm.es

Advances on Digital Image Processing in Pathology … · •Image Acquisition = Multispectral. A multispectral image is one that captures image data at specific frequencies across

Embed Size (px)

Citation preview

Advances on Digital Image

Processing in Pathology

Gloria Bueno

E.T.S.I. Industriales - UCLM

http://visilab.etsii.uclm.es

• Advances on Hardware and Software

* Image Digitalization and Acquisition:

- Image quality analysis - Multispectral analysis

* Application of Signal Processing Methods in Pathology:

Which Advances on Digital Image?

- Colour Model Analysis - Image Processing:

• ROI Detection – Segmentation • Stitching – Image Registration • 3D Reconstruction • WSI Visualization

- Artificial Intelligence (Classification)

Whole Slide Image (WSI)

nWSI fragments

Parallel, Grid, GPUs..

• Image Digitalization = Digital version of the whole glass slide (Whole Slide Image –WSI).

- Devices: a) motorized microscopes and b) scanners (1).

Image quality metrics development: Quantitative assessment in terms of focusing, contrast, stain, colour, brightness, overexposure and sharpness (2).

* It has been shown that the image quality provided by scanners is slightly better than in motorized microscopes (40x bright field).

* The image quality of both devices is suitable for clinical, educational and research purposes.

(1) ‘Critical Comparison of 31 Commercially Available Digital Slide Systems in Pathology’,

International Journal of Surgical Pathology, Vol. 14 , pp.285–305, 2006. (2) ‘Quality evaluation of microscopy and scanned histological images for diagnostic

purposes’, Micron, Vol. 43 , pp.334–343, 2012.

Advances on Hardware

• Image Acquisition = Multispectral. A multispectral image is one that captures image data at specific

frequencies across the electromagnetic spectrum. Dividing the spectrum into many bands (3).

Spectral imaging can allow extraction of additional information the human eye fails to capture with its receptors for red, green and blue.

Spectral imaging facilitates the use and extension of molecular pathology approaches, by permitting the detection and analysis of multiple probes simultaneously. (3) ‘Spectral Imaging and Pathology: Seeing More’, Laboratory Medicine, Vol. 35(4), pp.

244-251, 2004 .

Advances on Hardware

• Multispectral analysis applications

Advances on Hardware

Spectral unmixing of fluorescent signals from autofluorescence-I. Cells with a hybrid cytoplasmic protein linked to a CFP (cyan fluorescent protein). Spectral stack taken from 500 to 600 nm (in 10–nm steps) using a 40x lens. Spectral information was then used to unmix the CFP signal from the autofluorescence.

A. RGB image B. Autofluorescence signal. C. CFP signal only. D. Combined image of B and C.

• Multispectral analysis applications

Advances on Hardware

Spectral unmixing of fluorescent signals from autofluorescence-I. Tissue section was immunostained using an antibody directed against a vascular antigen and coupled to quantum-dot fluorescing with an emission peak at 640 nm. A spectral stack was taken from 550 to 700 nm (in 10-nm steps) using a 20x lens and spectral information used to unmix tissue auto-fluorescence from the quantum-dot signal.

A. RGB image B. Unmixed autofluorescence signal. C. Unmixed quantum-dot signal. D. Combined image of B and C.

• Multispectral analysis applications

Advances on Hardware

A. RGB image. Notice, orange autofluorescence connective tissue components around the cells.

Spectral multiplexing in FISH. A clinical specimen of breast cancer probed for her2/neu (red) and chromosome 17 (green) with nuclei counterstained with DAPI. Images taken from 450 to 700 nm in 20-nm steps using a 40x lens.

B. Results of spectral unmixing into DAPI, chromosome 17, Her2/neu and autofluorescence channels. The latter is not included in the final composite image, which has improved clarity over the original.

• Multispectral analysis applications – BrightField

Advances on Hardware

A. RGB image

B. Unmixed DAB signal.

C. Hematoxylin signal.

D. Combined image of B and C.

Unmixing DAB from Hematoxylin.Brightfield immunohistochemistry-I. Germinal center of lymph node stained with antibodies to proliferation marker, ki67, and counterstained with hematoxylin. From 440 to 680 nm in 20-nm steps at 20x sand the DAB (brown) stain spectrally unmixed from the blue hematoxylin. Note that the hematoxylin stain should be present in every nucleus.

• Multispectral analysis applications

Advances on Hardware

Nuclei Segmentation. Use of spectral imaging to segment nuclei in H&E-stained specimens. Prostate cancer was spectrally imaged from 450 to 680 nm in 10-nm steps using a 20x lens. The data was converted to absorbance (optical density) units and separated into eosin- and hematoxylin signals.

A. RGB image. B. Eosin signal (present in nuclei as well). C. Hematoxylin signal revealing nuclei isolated from connective tissue

components and suitable for subsequent morphometric analysis, if desired. D. Composite image. This technique can also be used to change color values

for different stained species to increase contrast or “readability.”

• Application of Signal Processing Methods in Pathology

Colour Model Analysis

Advances on Software

Comparative studies between different colour models: RGB, HIS, CMYK, L*a*b* and HSD colour models applied to histological images. This analysis, in turn, allows both: distinguishing possible regions of interest and retrieving their proper colour for further region analysis.

• Colour Model Analysis

Advances on Software

Colour models for a prostate biopsy tissue sample

• Colour Model Analysis

Advances on Software

Colour models for a citology tissue sample Colour models for a citology tissue sample

• Colour Model Analysis

Advances on Software

g) Lab segmentation h) Lab scatter plot

• Colour Model Analysis

Advances on Software

g) Lab segmentation h) Lab scatter plot

• Colour Model Analysis

Advances on Software

g) Lab segmentation h) Lab scatter plot

The results applied to prostate biopsies stained with HE and lung citologies stained with papanicolau show that the vector CIEDE2000 distance for the CIEL*a*b* model reproduces in a better way the original colour.

The comparison of the same processing under different colour model does allow us to choose the best colour model tailored to the microscopic stain and tissue type under consideration to obtain a successful processing.

A compromise between the computational cost and the results focus always to distinguish between different colour detection and colour retrieval for further ROI analysis should be kept.

The colour model should be taken into consideration when defining standards for histological images.

• Application of Signal Processing Methods in Pathology

Advances on Software

- Image Processing:

• ROI Detection – Segmentation • Stitching – Image Registration • 3D Reconstruction • WSI Visualization

- Artificial Intelligence (Classification)

Whole Slide Image (WSI)

nWSI fragments

Parallel, Grid, GPUs..

Feature Extraction

{textural, morphological,

geometrical}

challenge

Image Processing Applications

Advances on Software

•Cell Nuclei Detection and Segmentation on Tissue Microarrays of Renal Clear Cell Carcinoma

a) A detail of a TMA spot. b) The two independent annotations from the pathologists. c) Morphological segmentation. d) Final result after SVM nuclei

classification. Comparing image (b) and (d) it can be seen that the algorithm misses one nucleus in the top right quadrant but segments another nucleus on the left border, which was detected by only one of the two experts.

Image Processing Applications

Advances on Software

• Gleason Grading classification on prostate cancer

Gleason grade (a–c) 3 and (d–f) 4, prostate cancer biopsy specimens. Voronoi (b), (e) and Delaunay (c), (f) graph-based features quantify spatial architecture of the tissues. Final separation between grades 3 (circles) and 4 (squares) by graph embedding.

Image Processing Applications

Advances on Software

• Vessel segmentation using Colour and geometrical Analysis for angiogenesis research.

• Advances on digital image processing for both hardware and software.

• Challenges for WSI proccessing .

• New methods, quality aspects, quantitative validation.

• Changes to carry out diagnosis in Anatomical Pathology.

Conclusion

Thank you for your attention