Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Tissue Counter Analysis of histological images

M. Wiltgen

Institute of Medical Informatics, Statistics and DocumentationMED UNI GRAZ

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Steps in Image Analysis

Preprocessing

Segmentation

Feature Extraction

Classification

Interpretation

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Histological tissue: microscopic views

In histological tissue the structures are arranged in a variety of patterns

Segmentation of different structures is case dependent

Segmentation cannot be done in a general approach

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Study set: 80 cases from benign common nevi and malignant

melanoma

Malignant Melanoma

Benign Common Nevi

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Dissection into square elements

The images are divided into square elements of equal size

For each square element texture features are calculated

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Material and Preparation

Study set:

Learning set (40 images)

Test set (40 images)

Preparation:

Embedded in paraffin and cut at a section thickness

of 4µm

Stained with hematoxylin and eosin

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Tissue Counter Analysis : Overview

Feature analysis and extraction

Classification

Relocation

Feature analysis and extraction

Steps

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

mn gg

The features are based on:

Histogram Co-occurrence matrix

Variance

Shape of the distribution

Mean value

Entropy of the grey level distribution

Average values

Moments

Correlations

Entropies

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Histogram of grey levels

Distribution of grey levels: 2

( )( ) ii

N gh g

L

( ) 1ii

h g

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Features based on the histogram

Mean value

Standard deviation

Skewness and Kurtosis

Entropy of the grey level distribution

i

iiB ghgm )(

i

iBi ghmgF )()( 21

i

iBi ghmgF )(32

i

ii ghghF ))((log)()1( 24

( ) ( )n n i ii

F f g h g

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Co-occurrence matrix

Interpixel relationships:

1

( )

iji j

x y iji ji j k

w

P k w

ijij

iji j

aw

a

mn gg

( , )ij i ja g g

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Features based on the co-occurrence matrix

( ) ( )n n x yk

F f k P k

Angular Second Moment

Summed Average

Sum Variance

Sum Entropy

i j

jiwF 25

k

yx kPkF )(10

k

yx kPFkF )()( 21011

k

yxyx kPkPF ))((log)()1( 213

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Benign Common Nevi

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Malignant Melanoma

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Feature values for Benign Common Nevi and Malignant

MelanomaFeatures Nz Mn

Mean value 127.7 94.3

Standard deviation 35.4 56.1

Skewness -0.75 0.59

Angular second moment

0.00154 0.00115

Sum average 256.5 188.7

Sum variance 276.9 260.3

Sum entropy 6.72 7.77

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Fourier Transform

1 1

0 0

( , ) ( , ) ( , , , )L L

k l

b x y f k l k l x y

( , ) ( , )P k l f k l

Decomposition into frequency components:

Power spectrum:

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Features based on Fourier Transform

( ) 1( ) ( , )

( )f

mk l

F d P k lm d

2 2 1d k l d

Mean of the power spectrum at equidistant radius:

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Benign Common Nevi

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Malignant Melanoma

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Feature Extraction

The images of the 80 cases are divided into square elements with 32x32 pixels.

The data extracted from the single square elements consist of an n-dimensional vector of features:

1( ,..., )nx F F

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Tissue Counter Analysis : Steps

Feature analysis and extraction

Classification

Relocation

Classification

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

CART: Classification and Regression Trees

In CART analysis the set of square elements is split into homogeneous terminal nodes. The method consists of the parts:

selection of splits,

decision if the node is a terminal node or a non-terminal node,

the assignment of a terminal node to a class.

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

CART: Selection of Splits

The set of data

is recursively splitted by use of a threshold value into two sub sets ( ).U

{ | }

{ | }s R l

L l

t x F Ut

t x F U

,R Lt t

{ }X x

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

CART: Selection of Splits

The splitting is chosen in that way that difference between the impurities of the parents and the children sets has maximum value.

*( , ) max ( , )s S

i s t i s t

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

CART: Stop Rule

The decision if the node is a terminal node or a non-terminal node if given by the stop rule:

The threshold is a measure of how homogenous the terminal nodes must be.

max ( , )s S

i s t B

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

CART: Classification Tree

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

CART: Assignment to a Class

All the subsets are collected, assigned to a class and the classes are given by class label attachment:

( )Nevi TT

A p x

( )Mel TT

A p x

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

CART: Splitting Rules

The rules, which split the set of squares into disjunctive nodes are called splitting rules.

The splitting rules are guiding through the tree.

NeviMMeanSSkewnessVVariance )()()(

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Classification Results: Square Elements

Class Total number of elements

ClassificationRate %

Number of elements related to class 1

Number of elements related to class 2

1 benign common nevi

5120 92,734 4748 372

2 malignant melanoma

5120 92,090 405 4715

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Classification Results: Individual Cases

2020N =

MelanomaNevus

"ma

lign

ant

ele

men

ts"

120

100

80

60

40

20

0

-20

21

19

20

Test setLearning set

2020N =

MelanomaNevus"m

alig

nant

ele

men

ts"

120

100

80

60

40

20

0

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Individual Cases of Malignant Melanoma in the Learning Set

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Individual Cases of Benign Common Nevi in the Test Set

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Tissue Counter Analysis : Steps

Feature analysis and extraction

Classification

Relocation

Relocation

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Relocation of the classification results

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Erroneous Classification Results for Benign Common Nevi

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Correct Classification results for malignant melanoma

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Results and Discussion

RESULTS:RESULTS: For the learning set and the test set there is a significant difference between benign nevi and malignant melanoma without overlap. Discriminant analysis based on the percentage of “malignant elements” facilitated a correct classification of all cases.

DISCUSSION:DISCUSSION: Because no image segmentation was needed, problems related to this task were avoided. Though wrong classification of individual elements is unavoidable to some degree, tissue counter analysis shows a good discrimination between benign common nevi and malignant melanoma.

Medizinische Informatik und Dokumentation 2004

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Insti

tut

für

Med

izin

isch

e I

nfo

rmati

k,

Sta

tisti

k u

nd

Doku

men

tati

on

Sta

tisti

k u

nd

Doku

men

tati

on

Conclusion

Tissue counter analysis is a potential diagnostic tool in automatic or semi automatic discrimination of melanocytic skin tumors.