FUNDAMENTALS OF TEXTURE PROCESSING FOR BIOMEDICAL IMAGE ANALYSIS

FUNDAMENTALS OF TEXTURE PROCESSING FOR BIOMEDICAL IMAGE ANALYSIS

Adrien Depeursinge, PhD MICCAI 2015 Tutorial on Biomedical Texture Analysis (BTA), Munich, Oct 5 2015

sampled lattices) is not straightforward and raises several chal-lenges related to translation, scaling and rotation invariances andcovariances that are becoming more complex in 3-D.

1.2. Related work and scope of this survey

Depending on research communities, various taxonomies areused to refer to 3-D texture information. A clarification of the tax-onomy is proposed in this section to accurately define the scope ofthis survey. It is partly based on Toriwaki and Yoshida, 2009. Three-dimensional texture and volumetric texture are both general andidentical terms designing a texture defined in R3 and include:

(1) Volumetric textures existing in ‘‘filled’’ objectsfV : x; y; z 2 Vx;y;z ! R3g that are generated by a volumetricdata acquisition device (e.g., tomography, confocal imaging).

(2) 2.5D textures existing on surfaces of ‘‘hollow’’ objects asfC : x; y; z 2 Cu;v ! R3g,

(3) Dynamic textures in two–dimensional time sequences asfS : x; y; t 2 Sx;y;t ! R3g,

Solid texture refers to category (1) and accounts for textures de-fined in a volume Vx,y,z indexed by three coordinates. Solid textureshave an intrinsic dimension of 3, which means that a number ofvariables equal to the dimensionality of the Euclidean space isneeded to represent the signal (Bennett, 1965; Foncubierta-Rodrí-guez et al., 2013a). Category (2) is designed as textured surface inDana and Nayar (1999) and Cula and Dana (2004), or 2.5-dimen-sional textures in Lu et al. (2006) and Aguet et al. (2008), wheretextures C are existing on the surface of 3-D objects and can be in-dexed uniquely by two coordinates (u,v). (2) is also used in Kajiyaand Kay (1989), Neyret (1995), and Filip and Haindl (2009), where3-D geometries are added onto the surface of objects to create real-istic rendering of virtual scenes. Motion analysis in videos can alsobe considered a multi-dimensional texture analysis problembelonging to category (3) and is designed by ‘‘dynamic texture’’in Bouthemy and Fablet (1998), Chomat and Crowley (1999), andSchödl et al. (2000).

In this survey, a comprehensive review of the literature pub-lished on classification and retrieval of biomedical solid textures(i.e., category (1)) is carried out. The focus of this text is on the fea-ture extraction and not machine learning techniques, since onlyfeature extraction is specific to 3-D solid texture analysis.

1.3. Structure of this article

This survey is structured as follows: The fundamentals of 3-Ddigital texture processing are defined in Section 2. Section 3 de-scribes the reviewmethodology used to systematically retrieve pa-pers dealing with 3-D solid texture classification and retrieval. Theimaging modalities and organs studied in the literature are re-viewed in Sections 4 and 5, respectively to list the various expecta-tions and needs of 3-D image processing. The resulting application-driven techniques are described, organized and grouped togetherin Section 6. A synthesis of the trends and gaps of the various ap-proaches, conclusions and opportunities are given in Section 7,respectively.

2. Fundamentals of solid texture processing

Although several researchers attempted to establish a generalmodel of texture description (Haralick, 1979; Julesz, 1981), it isgenerally recognized that no general mathematical model of tex-ture can be used to solve every image analysis problem (Mallat,1999). In this survey, we compare the various approaches based

on the 3-D geometrical properties of the primitives used, i.e., theelementary building block considered. The set of primitives usedand their assumed interactions define the properties of the textureanalysis approaches, from statistical to structural methods.

In Section 2.1, we define the mathematical framework andnotations considered to describe the content of 3-D digital images.The notion of texture primitives as well as their scales and direc-tions are defined in Section 2.2.

2.1. 3-D digitized images and sampling

In Cartesian coordinates, a generic 3-D continuous image is de-fined by a function of three variables f(x,y,z), where f represents ascalar at a point ðx; y; zÞ 2 R3. A 3-D digital image F(i, j,k) of dimen-sions M $ N $ O is obtained from sampling f at points ði; j; kÞ 2 Z3

of a 3-D ordered array (see Fig. 2). Increments in (i, j,k), correspondto physical displacements in R3 parametrized by the respectivespacings (Dx,Dy,Dz). For every cell of the digitized array, the valueof F(i, j,k) is typically obtained by averaging f in the cuboid domaindefined by (x,y,z) 2 [iDx, (i + 1)Dx]; [jDy, (j + 1)Dy]; [kDz, (k +1)Dz]) (Toriwaki and Yoshida, 2009). This cuboid is called a voxel.The three spherical coordinates (r,h,/) are unevenly sampled to(R,H,U) as shown in Fig. 3.

2.2. Texture primitives

The notion of texture primitive has been widely used in 2-D tex-ture analysis and defines the elementary building block of a giventexture class (Haralick, 1979; Jain et al., 1995; Lin et al., 1999). Alltexture processing approaches aim at modeling a given textureusing sets of prototype primitives. The concept of texture primitiveis naturally extended in 3-D as the geometry of the voxel sequenceused by a given texture analysis method. We consider a primitiveC(i, j,k) centered at a point (i, j,k) that lives on a neighborhood ofthis point. The primitive is constituted by a set of voxels with graytone values that forms a 3-D structure. Typical C neighborhoodsare voxel pairs, linear, planar, spherical or unconstrained. Signalassignment to the primitive can be either binary, categorical orcontinuous. Two example texture primitives are shown in Fig. 4.Texture primitives refer to local processing of 3-D images and localpatterns (see Toriwaki and Yoshida, 2009).

Fig. 2. 3-D digitized images and sampling in Cartesian coordinates.

Fig. 3. 3-D digitized images and sampling in spherical coordinates.

178 A. Depeursinge et al. /Medical Image Analysis 18 (2014) 176–196























sampled

lattices)is

notstraightforw

ardand

raisesseveral

chal-lenges

relatedto

translation,scalingand

rotationinvariances

andcovariances

thatare

becoming

more

complex

in3-D

.

1.2.Relatedwork

andscope

ofthis

survey

Depending

onresearch

communities,

varioustaxonom

iesare

usedto

referto

3-Dtexture

information.A

clarificationof

thetax-

onomyis

proposedin

thissection

toaccurately

definethe

scopeof

thissurvey.Itis

partlybased

onToriw

akiandYoshida,2009.Three-

dimensional

textureand

volumetric

textureare

bothgeneral

andidenticalterm

sdesigning

atexture

definedin

R3and

include:

(1)Volum

etrictextures

existingin

‘‘filled’’objects

fV:x;y

;z2V

x;y;z !

R3g

thatare

generatedby

avolum

etricdata

acquisitiondevice

(e.g.,tomography,confocalim

aging).(2)

2.5Dtextures

existingon

surfacesof

‘‘hollow’’objects

asfC

:x;y;z

2C

u;v!

R3g,

(3)Dynam

ictextures

intw

o–dimensional

time

sequencesas

fS:x;y

;t2Sx;y

;t !R

3g,

Solidtexture

refersto

category(1)

andaccounts

fortextures

de-fined

inavolum

eVx,y,z indexed

bythree

coordinates.Solidtextures

havean

intrinsicdim

ensionof

3,which

means

thatanum

berof

variablesequal

tothe

dimensionality

ofthe

Euclideanspace

isneeded

torepresent

thesignal(Bennett,1965;

Foncubierta-Rodrí-guez

etal.,2013a).Category

(2)is

designedas

texturedsurface

inDana

andNayar

(1999)and

Culaand

Dana

(2004),or2.5-dim

en-sional

texturesin

Luet

al.(2006)and

Aguet

etal.(2008),w

heretextures

Care

existingon

thesurface

of3-Dobjects

andcan

bein-

dexeduniquely

bytw

ocoordinates

(u,v).(2)is

alsoused

inKajiya

andKay

(1989),Neyret

(1995),andFilip

andHaindl(2009),w

here3-D

geometries

areadded

ontothe

surfaceofobjects

tocreate

real-istic

renderingofvirtualscenes.M

otionanalysis

invideos

canalso

beconsidered

amulti-dim

ensionaltexture

analysisproblem

belongingto

category(3)

andis

designedby

‘‘dynamic

texture’’in

Bouthemyand

Fablet(1998),Chom

atand

Crowley

(1999),andSchödlet

al.(2000).In

thissurvey,

acom

prehensivereview

ofthe

literaturepub-

lishedon

classificationand

retrievalof

biomedical

solidtextures

(i.e.,category(1))is

carriedout.The

focusofthis

textis

onthe

fea-ture

extractionand

notmachine

learningtechniques,

sinceonly

featureextraction

isspecific

to3-D

solidtexture

analysis.

1.3.Structureof

thisarticle

Thissurvey

isstructured

asfollow

s:The

fundamentals

of3-D

digitaltexture

processingare

definedin

Section2.

Section3de-

scribesthe

reviewmethodology

usedto

systematically

retrievepa-

persdealing

with

3-Dsolid

textureclassification

andretrieval.The

imaging

modalities

andorgans

studiedin

theliterature

arere-

viewed

inSections

4and

5,respectivelyto

listthe

variousexpecta-

tionsand

needsof3-D

image

processing.Theresulting

application-driven

techniquesare

described,organizedand

groupedtogether

inSection

6.Asynthesis

ofthe

trendsand

gapsof

thevarious

ap-proaches,

conclusionsand

opportunitiesare

givenin

Section7,

respectively.

2.Fundam

entals

ofsolid

texture

processing

Although

severalresearchers

attempted

toestablish

ageneral

model

oftexture

description(H

aralick,1979;

Julesz,1981),

itis

generallyrecognized

thatno

generalmathem

aticalmodel

oftex-

turecan

beused

tosolve

everyim

ageanalysis

problem(M

allat,1999).

Inthis

survey,wecom

parethe

variousapproaches

based

onthe

3-Dgeom

etricalproperties

ofthe

primitives

used,i.e.,theelem

entarybuilding

blockconsidered.The

setof

primitives

usedand

theirassum

edinteractions

definethe

propertiesofthe

textureanalysis

approaches,fromstatisticalto

structuralmethods.

InSection

2.1,we

definethe

mathem

aticalfram

ework

andnotations

consideredto

describethe

contentof3-D

digitalimages.

Thenotion

oftexture

primitives

aswell

astheir

scalesand

direc-tions

aredefined

inSection

2.2.

2.1.3-Ddigitized

images

andsam

pling

InCartesian

coordinates,ageneric

3-Dcontinuous

image

isde-

finedby

afunction

ofthree

variablesf(x,y,z),w

herefrepresents

ascalar

atapointðx;y

;zÞ2R

3.A3-D

digitalimage

F(i,j,k)ofdim

en-sions

M$

N$

Ois

obtainedfrom

sampling

fat

pointsði;j;kÞ

2Z

3

ofa3-D

orderedarray

(seeFig.2).Increm

entsin

(i,j,k),correspondto

physicaldisplacem

entsin

R3param

etrizedby

therespective

spacings(D

x,Dy,D

z).Forevery

cellofthedigitized

array,thevalue

ofF(i,j,k)is

typicallyobtained

byaveraging

finthe

cuboiddom

aindefined

by(x,y,z)2

[iDx,(i+

1)Dx];

[jDy,(j+

1)Dy];

[kDz,(k

+1)D

z])(Toriw

akiandYoshida,2009).This

cuboidis

calledavoxel.

Thethree

sphericalcoordinates

(r,h,/)are

unevenlysam

pledto

(R,H,U

)as

shownin

Fig.3.

2.2.Textureprim

itives

Thenotion

oftextureprim

itivehas

beenwidely

usedin

2-Dtex-

tureanalysis

anddefines

theelem

entarybuilding

blockof

agiven

textureclass

(Haralick,1979;

Jainet

al.,1995;Lin

etal.,1999).A

lltexture

processingapproaches

aimat

modeling

agiven

textureusing

setsofprototype

primitives.The

conceptoftextureprim

itiveis

naturallyextended

in3-D

asthe

geometry

ofthevoxelsequence

usedby

agiven

textureanalysis

method.W

econsider

aprim

itiveC(i,j,k)

centeredat

apoint

(i,j,k)that

liveson

aneighborhood

ofthis

point.Theprim

itiveis

constitutedby

aset

ofvoxelswith

graytone

valuesthat

formsa3-D

structure.TypicalC

neighborhoodsare

voxelpairs,

linear,planar,

sphericalor

unconstrained.Signal

assignment

tothe

primitive

canbe

eitherbinary,

categoricalor

continuous.Tw

oexam

pletexture

primitives

areshow

nin

Fig.4.

Textureprim

itivesrefer

tolocalprocessing

of3-Dim

agesand

localpatterns

(seeToriw

akiandYoshida,2009).

Fig.2.3-D

digitizedim

agesand

sampling

inCartesian

coordinates.

Fig.3.3-D

digitizedim

agesand

sampling

insphericalcoordinates.

178A.D

epeursingeet

al./MedicalIm

ageAnalysis

18(2014)

176–196

OUTLINE

• Biomedical texture analysis: background

• Defining texture processes

• Notations, sampling and texture functions

• Texture operators, primitives and invariances

• Multiscale analysis

• Operator scale and uncertainty principle

• Region of interest and operator aggregation

• Multidirectional analysis

• Isotropic versus directional operators

• Importance of the local organization of image directions

• Conclusions

• References

REFERENCES

66

[?] Texture in Biomedical Images, Petrou M.,

L1

L2

M1

M2

M

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. XX, NO. XX, XX 2015 1

Steerable Wavelet Machines (SWM): LearningMoving Frames for Texture Classification

Adrien Depeursinge, Zsuzsanna Püspöki, John Paul Ward, and Michael Unser

Abstract—We present texture operators encoding class-specific local organizations of image directions (LOID) in arotation-invariant fashion. The operators are based on steerablecircular harmonic wavelets (CHW), offering a rich and yet com-pact initial representation for characterizing natural textures.We enforce the preservation of the joint location and orientationimage structure by building Cartan moving frames (MF) fromlocally-steered multi-order CHWs. In a second step, we usesupport vector machines (SVM) to learn a multi-class shapingmatrix of the initial CHW representations, yielding data-drivenMFs. Intuitively, the learned MFs can be seen as class-specificforged detectors that are applied and rotated at each point ofthe image to evaluate the magnitude of their responses, i.e., thepresence of the corresponding texture class. We experimentallydemonstrate the effectiveness of the proposed operators forclassifying natural textures, which outperformed the reportedperformance of recent approaches on several test suites of theOutex database.

Index Terms—Texture classification, feature learning, mov-ing frames, support vector machines, steerability, rotation-invariance, illumination-invariance, wavelet analysis.

I. INTRODUCTION

THe wealth of digital texture patterns is tightly relatedto the size of the observation window on a discrete

lattice. In an extreme case, an image region composedof one pixel cannot form geometrical structures. There-fore, the characterization of textural patterns often requiresintegrating image operators g (x), x 2 R2 over an imagedomain X . The latter raises two major challenges. First,the action of the integrated operators becomes diffuseover X , which hinders the spatial precision of texturesegmentation approaches. Second, the effect of integrationbecomes even more destructive when unidirectional opera-tors are jointly used to characterize the local organization ofimage directions (LOID) (e.g., directional filterbanks [1, 2],curvelets [3], histogram of gradients (HOG) [4], Haralick [5]).When separately integrated, the responses of unidirectionalindividual operators are not local anymore and their jointresponses become only sensitive to the global amount ofimage directions in the region X . For instance, the joint

responses of image gradients g1,2(x) =≥ØØ @ f (x)

@x1

ØØ,ØØ @ f (x)@x2

ØØ¥

arenot able to discriminate between the two textures classesf1(x) and f2(x) shown in Fig. 1 when integrated over theimage domain X .

This loss of information is detailed by Sifre et al. interms of separable group invariants [6]. When integrated

The authors are with the Biomedical Imaging Group, École Poly-technique Fédérale de Lausanne, Lausanne 1015, Switzerland (e-mail:[email protected]; [email protected]). A. Depeursinge isalso with the MedGIFT group, Institute of Information Systems, Universityof Applied Sciences Western Switzerland, Sierre (HES-SO).

of1(x) : of2(x) :

RX

ØØ @ f (x)@x1

ØØdx

R XØ Ø@

f(x)

@x 2

Ø Ø dx

image gradients

Fig. 1. The joint responses of image gradients≥| @ f (x)@x1

|, | @ f (x)@x2

|¥

are notable to discriminate between the two textures classes f1(x) and f2(x)when integrated over the image domain X . One circle in the gradientrepresentation corresponds to one realization (i.e., full image) of f1,2.

separately, the responses of unidirectional operators be-come invariant to a larger family of roto-translations wheredifferent orientations are translated by different values. Forinstance, it can be observed in Fig. 1 that f2 can be obtainedfrom f1 by vertically translating horizontal bars only andhorizontally translating vertical bars only.

The LOID are key for visual understanding [7] and is theonly property able to differentiate between f1 and f2 inFig. 1. It relates to the joint information between positionsand orientations in images, which is discarded by operatorsthat are invariant to separable groups. The LOID has beenleveraged in the literature to define [8] and discriminatetexture classes [9–12]. It is also at the origin of the successof methods based on local binary patterns (LBP) [9] andits extensions [13–19], which specifically encodes the LOIDwith uniform circular pixel sequences. Although classicaldeep learning approaches are not enforcing the characteri-zation of the LOID [20], deep convolutional networks wererecently proposed by Oyallon et al. to specifically preservethe structure of the roto-translation group [21].

An even bigger challenge is to design texture operatorsthat can characterize the LOID in a rotation-invariantfashion [9, 11]. The latter is very important to recognizeimage structures independently from both their local ori-entations and the global orientation of the image. Exam-ples of such structures are vascular, bronchial or dendritictrees in biomedical images, river deltas or urban areas insatellite images, or even more complex biomedical tissuearchitectures or crystals in petrographic analysis. Since mostof the above-mentioned imaging modalities yield imageswith pixel sizes defined in physical dimensions, we arenot interested in invariance to image scale. We aim todesign texture operators that are invariant to the family ofEuclidean transforms (also called rigid), i.e.,

g (x) = g (Rµ0 x °x0), 8x , x0,µ0 2R2,

·m

BACKGROUND – RADIOMICS - HISTOPATHOLOMICS

• Personalized medicine aims at enhancing the patient’s quality of life and prognosis

• Tailored treatment and medical decisions based on the molecular composition of diseased tissue

• Current limitations [Gerlinger2012]

• Molecular analysis of tissue composition is invasive (biopsy), slow and costly

• Cannot capture molecular heterogeneity

3

Intr atumor Heterogeneity Revealed by multiregion Sequencing

n engl j med 366;10 nejm.org march 8, 2012 887

tion through loss of SETD2 methyltransferase func-tion driven by three distinct, regionally separated mutations on a background of ubiquitous loss of the other SETD2 allele on chromosome 3p.

Convergent evolution was observed for the X-chromosome–encoded histone H3K4 demeth-ylase KDM5C, harboring disruptive mutations in R1 through R3, R5, and R8 through R9 (missense

and frameshift deletion) and a splice-site mutation in the metastases (Fig. 2B and 2C).

mTOR Functional Intratumor HeterogeneityThe mammalian target of rapamycin (mTOR) ki-nase carried a kinase-domain missense mutation (L2431P) in all primary tumor regions except R4. All tumor regions harboring mTOR (L2431P) had

B Regional Distribution of Mutations

C Phylogenetic Relationships of Tumor Regions D Ploidy Profiling

A Biopsy Sites

R2 R4

DI=1.43

DI=1.81

M2bR9

Tetraploid

R4b

R9 R8R5

R4a

R1R3R2

M1M2b

M2a

VHL

KDM5C (missense and frameshift)mTOR (missense)

SETD2 (missense)KDM5C (splice site)

SETD2 (splice site)

?

SETD2 (frameshift)

PreP

PreM

Normal tissue

PrePPreMR1R2R3R5R8R9R4M1M2aM2b

C2o

rf85

WD

R7SU

PT6H

CD

H19

LAM

A3

DIX

DC

1H

PS5

NRA

PKI

AA

1524

SETD

2PL

CL1

BCL1

1AIF

NA

R1$

DA

MTS

10

C3

KIA

A12

67.

RT4

CD

44A

NKR

D26

TM7S

F4SL

C2A

1D

AC

H2

MM

AB

ZN

F521

HM

G20

AD

NM

T3A

RLF

MA

MLD

1M

AP3

K6H

DA

C6

PHF2

1BFA

M12

9BRP

S8C

IB2

RAB2

7ASL

C2A

12D

USP

12A

DA

MTS

L4N

AP1

L3U

SP51

KDM

5CSB

F1TO

M1

MYH

8W

DR2

4IT

IH5

AKA

P9FB

XO1

LIA

STN

IKSE

TD2

C3o

rf20

MR1

PIA

S3D

IO1

ERC

C5

KLALK

BH8

DA

PK1

DD

X58

SPA

TA21

ZN

F493

NG

EFD

IRA

S3LA

TS2

ITG

B3FL

NA

SATL

1KD

M5C

KDM

5CRB

FOX2

NPH

S1SO

X9C

ENPN

PSM

D7

RIM

BP2

GA

LNT1

1A

BHD

11U

GT2

A1

MTO

RPP

P6R2

ZN

F780

AW

SCD

2C

DKN

1BPP

FIA

1THSS

NA

1C

ASP

2PL

RG1

SETD

2C

CBL

2SE

SN2

MA

GEB

16N

LRP7

IGLO

N5

KLK4

WD

R62

KIA

A03

55C

YP4F

3A

KAP8

ZN

F519

DD

X52

ZC

3H18

TCF1

2N

USA

P172

X4KD

M2B

MRP

L51

C11

orf6

8A

NO

5EI

F4G

2M

SRB2

RALG

DS

EXT1

ZC

3HC

1PT

PRZ

1IN

TS1

CC

R6D

OPE

Y1A

TXN

1W

HSC

1C

LCN

2SS

R3KL

HL1

8SG

OL1

VHL

C2o

rf21

ALS

2CR1

2PL

B1FC

AM

RIF

I16

BCA

S2IL

12RB

2

PrivateUbiquitous Shared primary Shared metastasis

Private

Ubiquitous

Lungmetastases

Chest-wallmetastasis

Perinephricmetastasis

M110 cm

R7 (G4)

R5 (G4)

R9

R3 (G4)

R1 (G3) R2 (G3)

R4 (G1)

R6 (G1)

Hilu

m

R8 (G4)

Primarytumor

Shared primaryShared metastasis

M2b

M2a

Propidium Iodide Staining

No.

of C

ells

The New England Journal of Medicine Downloaded from nejm.org at UNIVERSITE DE GENEVE on June 2, 2014. For personal use only. No other uses without permission.

Copyright © 2012 Massachusetts Medical Society. All rights reserved.









A Biopsy Sites

R2 R4

DI=1.43

DI=1.81

M2bR9

Tetraploid

R4b

R9 R8R5

R4a

R1R3R2

M1M2b

M2a

VHL



SETD2 (splice site)

?

SETD2 (frameshift)

PreP

PreM

Normal tissue


C2o

rf85

WD

R7SU

PT6H

CD

H19

LAM

A3

DIX

DC

1H

PS5

NRA

PKI

AA

1524

SETD

2PL

CL1

BCL1

1AIF

NA

R1$

DA

MTS

10

C3

KIA

A12

67.

RT4

CD

44A

NKR

D26

TM7S

F4SL

C2A

1D

AC

H2

MM

AB

ZN

F521

HM

G20

AD

NM

T3A

RLF

MA

MLD

1M

AP3

K6H

DA

C6

PHF2

1BFA

M12

9BRP

S8C

IB2

RAB2

7ASL

C2A

12D

USP

12A

DA

MTS

L4N

AP1

L3U

SP51

KDM

5CSB

F1TO

M1

MYH

8W

DR2

4IT

IH5

AKA

P9FB

XO1

LIA

STN

IKSE

TD2

C3o

rf20

MR1

PIA

S3D

IO1

ERC

C5

KLALK

BH8

DA

PK1

DD

X58

SPA

TA21

ZN

F493

NG

EFD

IRA

S3LA

TS2

ITG

B3FL

NA

SATL

1KD

M5C

KDM

5CRB

FOX2

NPH

S1SO

X9C

ENPN

PSM

D7

RIM

BP2

GA

LNT1

1A

BHD

11U

GT2

A1

MTO

RPP

P6R2

ZN

F780

AW

SCD

2C

DKN

1BPP

FIA

1THSS

NA

1C

ASP

2PL

RG1

SETD

2C

CBL

2SE

SN2

MA

GEB

16N

LRP7

IGLO

N5

KLK4

WD

R62

KIA

A03

55C

YP4F

3A

KAP8

ZN

F519

DD

X52

ZC

3H18

TCF1

2N

USA

P172

X4KD

M2B

MRP

L51

C11

orf6

8A

NO

5EI

F4G

2M

SRB2

RALG

DS

EXT1

ZC

3HC

1PT

PRZ

1IN

TS1

CC

R6D

OPE

Y1A

TXN

1W

HSC

1C

LCN

2SS

R3KL

HL1

8SG

OL1

VHL

C2o

rf21

ALS

2CR1

2PL

B1FC

AM

RIF

I16

BCA

S2IL

12RB

2


Private

Ubiquitous

Lungmetastases



M110 cm

R7 (G4)

R5 (G4)

R9

R3 (G4)

R1 (G3) R2 (G3)

R4 (G1)

R6 (G1)

Hilu

mR8 (G4)

Primarytumor


M2b

M2a


No.

of C

ells











A Biopsy Sites

R2 R4

DI=1.43

DI=1.81

M2bR9

Tetraploid

R4b

R9 R8R5

R4a

R1R3R2

M1M2b

M2a

VHL



SETD2 (splice site)

?

SETD2 (frameshift)

PreP

PreM

Normal tissue


C2o

rf85

WD

R7SU

PT6H

CD

H19

LAM

A3

DIX

DC

1H

PS5

NRA

PKI

AA

1524

SETD

2PL

CL1

BCL1

1AIF

NA

R1$

DA

MTS

10

C3

KIA

A12

67.

RT4

CD

44A

NKR

D26

TM7S

F4SL

C2A

1D

AC

H2

MM

AB

ZN

F521

HM

G20

AD

NM

T3A

RLF

MA

MLD

1M

AP3

K6H

DA

C6

PHF2

1BFA

M12

9BRP

S8C

IB2

RAB2

7ASL

C2A

12D

USP

12A

DA

MTS

L4N

AP1

L3U

SP51

KDM

5CSB

F1TO

M1

MYH

8W

DR2

4IT

IH5

AKA

P9FB

XO1

LIA

STN

IKSE

TD2

C3o

rf20

MR1

PIA

S3D

IO1

ERC

C5

KLALK

BH8

DA

PK1

DD

X58

SPA

TA21

ZN

F493

NG

EFD

IRA

S3LA

TS2

ITG

B3FL

NA

SATL

1KD

M5C

KDM

5CRB

FOX2

NPH

S1SO

X9C

ENPN

PSM

D7

RIM

BP2

GA

LNT1

1A

BHD

11U

GT2

A1

MTO

RPP

P6R2

ZN

F780

AW

SCD

2C

DKN

1BPP

FIA

1THSS

NA

1C

ASP

2PL

RG1

SETD

2C

CBL

2SE

SN2

MA

GEB

16N

LRP7

IGLO

N5

KLK4

WD

R62

KIA

A03

55C

YP4F

3A

KAP8

ZN

F519

DD

X52

ZC

3H18

TCF1

2N

USA

P172

X4KD

M2B

MRP

L51

C11

orf6

8A

NO

5EI

F4G

2M

SRB2

RALG

DS

EXT1

ZC

3HC

1PT

PRZ

1IN

TS1

CC

R6D

OPE

Y1A

TXN

1W

HSC

1C

LCN

2SS

R3KL

HL1

8SG

OL1

VHL

C2o

rf21

ALS

2CR1

2PL

B1FC

AM

RIF

I16

BCA

S2IL

12RB

2


Private

Ubiquitous

Lungmetastases



M110 cm

R7 (G4)

R5 (G4)

R9

R3 (G4)

R1 (G3) R2 (G3)

R4 (G1)

R6 (G1)H

ilum

R8 (G4)

Primarytumor


M2b

M2a


No.

of C

ells




• Huge potential for computerized medical image analysis

• Explore and reveal tissue structures related to tissue composition, function, ….

• Local quantitative image feature extraction

• Supervised and unsupervised machine learning

4

malignant, nonresponder

malignant, responder

benign

pre-malignant

undefined

quant. feat. #1

quan

t. fe

at. #

2

Supervised learning, big data



• Create imaging biomarkers to predict diagnosis, prognosis, treatment response [Aerts2014]

5

Radiomics [Kumar2012] “Histopatholomics” [Gurcan2009]

Reuse existing diagnostic images ✓ radiology data1 ✓ digital pathology

Capture tissue heterogeneity

✓ 3D neighborhoods(e.g., 512x512x512)

✓ large 2D regions(e.g., 15,000x15,000)

Analytic power beyond naked eyes

✓ complex 3D tissue morphology

✓exhaustive characterization of 2D tissue structures

Non-invasive ✓ x

1e.g., X-ray, Ultrasound, CT, MRI, PET, OCT, …



• Explore and reveal tissue structures related to tissue composition, function, ….

• Local quantitative image feature extraction

• Supervised and unsupervised machine learning

6



benign

pre-malignant

undefined

quant. feat. #1

quan

t. fe

at. #

2

Supervised learning, big data

Specific to texture!

OUTLINE











• Conclusions

• References

REFERENCES

66


L1

L2

M1

M2

M






I. INTRODUCTION




@x1


ØØ¥




of1(x) : of2(x) :

RX

ØØ @ f (x)@x1

ØØdx

R XØ Ø@

f(x)

@x 2

Ø Ø dx

image gradients


|, | @ f (x)@x2

|¥





g (x) = g (Rµ0 x °x0), 8x , x0,µ0 2R2,

·m

• Definition of texture

• Everybody agrees that nobody agrees on the definition of “texture” (context-dependent)

• “coarse”, “edgy”, “directional”, “repetitive”, “random”, …

• Oxford dictionary: “the feel, appearance, or consistency of a surface or a substance”

• [Haidekker2011]: “Texture is a systematic local variation of the image values”

• [Petrou2011]: “The most important characteristic of texture is that it is scale dependent. Different types of texture are visible at different scales”

COMPUTERIZED TEXTURE ANALYSIS

8

resolution, allowing to characterize structural properties of bio-medical tissue.1 Tissue anomalies are well characterized by localizedtexture properties in most imaging modalities (Tourassi, 1999;Kovalev and Petrou, 2000; Castellano et al., 2004; Depeursinge andMüller, 2011). This calls for scientific contributions on computerizedanalysis of 3-D texture in biomedical images, which engendered ma-jor scientific breakthroughs in 3-D solid texture analysis during thepast 20 years (Blot and Zwiggelaar, 2002; Kovalev and Petrou,2009; Foncubierta-Rodríguez et al., 2013a).

1.1. Biomedical volumetric solid texture

A uniform textured volume in a 3-D biomedical image is consid-ered to be composed of homogeneous tissue properties. The con-cept of organs or organelles was invented by human observersfor efficient understanding of anatomy. The latter can be definedas an organized cluster of one or several tissue types (i.e., definingsolid textures). Fig. 1 illustrates that, at various scales, everything istexture in biomedical images starting from the cell level to the or-gan level. The scaling parameter of textures is thus fundamentaland it is often used in computerized texture analysis approaches(Yeshurun and Carrasco, 2000).

According to the Oxford Dictionaries,2 texture is defined as ‘‘thefeel, appearance, or consistency of a surface or a substance’’, which re-lates to the surface structureor the internal structureof the consideredmatter in the context of 2-D or 3-D textures, respectively. The defini-tion of 3-D texture is not equivalent to 2-D surface texture since opa-que 3-D textures cannot be described in terms of reflectivity or albedoof a matter, which are often used to characterize textured surfaces(Dana et al., 1999). Haralick et al. (1973) also define texture as being‘‘an innate property of virtually all surfaces’’ and stipulates thattexture ‘‘contains important informationabout the structural arrange-ment of surfaces and their relationship to the surrounding environ-ment’’, which is also formally limited to textured surfaces.

Textured surfaces are central to human vision, because they areimportant visual cues about surface property, scenic depth, surfaceorientation, and texture information is used in pre-attentive visionfor identifying objects and understanding scenes (Julesz, 1962).The human visual cortex is sensitive to the orientation and spatialfrequencies (i.e., repetitiveness) of patterns (Blakemore and

Campbell, 1969; Maffei and Fiorentini, 1973), which relates to tex-ture properties. It is only in a second step that regions of homoge-neous textures are aggregated to constitute objects (e.g., organs) ata higher level of scene interpretation. However, the human com-prehension of the three-dimensional environment relies on ob-jects. The concept of three-dimensional texture is little used,because texture existing in more than two dimensions cannot befully visualized by humans (Toriwaki and Yoshida, 2009). Only vir-tual navigation in MPR or semi-transparent visualizations aremade available by computer graphics and allow observing 2-D pro-jections of opaque textures.

In a concern of sparsity and synthesis, 3-D computer graphicshave been focusing on objects. Shape-based methods allow encap-sulating essential properties of objects and thus provide approxi-mations of the real world that are corresponding to humanunderstanding and abstraction level. Recently, data acquisitiontechniques in medical imaging (e.g., tomographic, confocal, echo-graphic) as well as recent computing and storage infrastructuresallow computer vision and graphics to go beyond shape-basedmethods and towards three-dimensional solid texture-baseddescription of the visual information. 3-D solid textures encompassrich information of the internal structures of objects because theyare defined for each coordinate x; y; z 2 Vx;y;z ! R3, whereas shape-based descriptions are defined on surfaces x; y; z 2 Cu;v ! R3.jVj" jCj because every point of C can be uniquely indexed by onlytwo coordinates (u,v). Texture- and shape-based approaches arecomplementary and their success depends on the applicationneeds. While several papers on shape-based methods for classifica-tion and retrieval of organs and biomedical structures have beenpublished during the past 15 years (McInerney and Terzopoulos,1996; Metaxas, 1996; Beichel et al., 2001; Heimann and Meinzer,2009), 3-D biomedical solid texture analysis is still an emerging re-search field (Blot and Zwiggelaar, 2002). The most common ap-proach to 3-D solid texture analysis is to use 2-D texture in slices(Castellano et al., 2004; Depeursinge et al., 2007; Sørensen et al.,2010) or by projecting volumetric data on a plane (Chan et al.,2008), which does not allow exploiting the wealth of 3-D textureinformation. Based on the success and attention that 2-D textureanalysis obtained in the biomedical computer vision communityas well as the observed improved performance of 3-D techniquesover 2-D approaches in several application domains (Ranguelovaand Quinn, 1999; Mahmoud-Ghoneim et al., 2003; Xu et al.,2006b; Chen et al., 2007), 3-D biomedical solid texture analysisis expected to be a major research field in computer vision in thecoming years. The extension of 2-D approaches to R3 (or Z3 for

Fig. 1. 3-D biomedical tissue defines solid texture at multiple scales.

1 Biomedical tissue is considered in a broad meaning including connective, muscle,and nervous tissue.

2 http://oxforddictionaries.com/definition/texture, as of 9 October2013.

A. Depeursinge et al. /Medical Image Analysis 18 (2014) 176–196 177

[Depeursinge2014a]


directions

9

scales

• Spatial scales and directions in images are fundamental for visual texture discrimination [Blakemore1969, Romeny2011]

• Relating to directional frequencies (shown in Fourier)


10

directionsscales

• Spatial scales and directions in images are fundamental for visual texture discrimination [Blakemore1969, Romeny2011]

• Most approaches are leveraging these two properties

• Explicitly: Gray-level co-occurrence matrices (GLCM), run-length matrices (RLE), directional filterbanks and wavelets, Fourier, histograms of gradients (HOG), local binary patterns (LBP)

• Implicitly: Convolutional neural networks (CNN), scattering transform, topographic independant component analysis (TICA)

NOTATIONS, SAMPLING AND TEXTURE FUNCTIONS

11

• 2-D continuous texture functions in space and Fourier

• Cartesian coordinates:

• Polar coordinates:

• 2-D digital texture functions



• Sampling (Cartesian)

• Increments in corresponds to physical displacements in as

f(k), k =

✓k1k2

◆2 Z2

f(R,⇥), R 2 Z+,⇥ 2 [0, 2⇡)

(k1, k2) R2

✓x1

x2

◆=

✓�x1 · k1�x2 · k2

◆

x1

x2 k2

k1

)

�x1

�x2

R2 Z2

·f(x) f(k)

f(x), x =

✓x1

x2

◆2 R2

, f(x)F�! f(!) =

Z

R2

f(x)e�jh!,xidx, ! 2 R2

f(r, ✓), r 2 R+, ✓ 2 [0, 2⇡), f(r, ✓)F�! f(⇢,�), ⇢ 2 R+,� 2 [0, 2⇡)


12

• 3-D continuous texture functions in space and Fourier



• 3-D digital texture functions



• Sampling (Cartesian)

• Increments in corresponds to physical displacements in as

x1

x2

k2k1

)�x1�x2

·f(x) f(k)

f(k), k =

0

@k1k2k3

1

A 2 Z3

f(r, ✓,�), r 2 R+, ✓ 2 [0, 2⇡),� 2 [0, 2⇡)

f(R,⇥,�), R 2 Z+,⇥ 2 [0, 2⇡),� 2 [0, 2⇡)

(k1, k2, k3) R3

0

@x1

x2

x3

1

A =

0

@�x1 · k1�x2 · k2�x3 · k3

1

A �x3

x3 k3R3 Z3

f(x), x =

0

@x1

x2

x3

1

A 2 R3, f(x)

F�! f(!) =

Z

R3

f(x)e�jh!,xidx, ! 2 R3

• We consider a texture function as a realization of a spatial stochastic process of where is the value at the spatial position indexed by

• The values of follow one or several probability density functions

• Examples


13

f(x)

Rd

{Xm,m 2 RM1⇥···⇥Md}

Xm m

moving average Gaussian pointwise Poisson biomedical: lung fibrosis in CT

m 2 R128⇥128 m 2 R32⇥32 m 2 R84⇥84

Xm fXm(q)


14

• Stationarity of spatial stochastic processes

• A spatial process is stationary if the probability density functions are equivalent for all

• Example: heteroscedastic moving average Gaussian process

{Xm,m 2 RM1⇥···⇥Md}mfXm(q)

stationarynon-stationary (strict sense)

fb,Xm(q) =1

3p2⇡

e�(q�0)2

2 32

fa fb

fa,Xm(q) =1

1p2⇡

e�(q�0)2

2 12


15

• Stationarity of textures and human perception / tissue biology

• Strict/weak process stationarity and texture class definition is not equivalent

• Image analysis tasks when textures are considered as

• Stationary (wide sense): texture classification

• Non-stationary: texture segmentation

Outex “canvas039”: stationary? brain glioblastoma in T1-MRI: stationary?

Fig. 3: Top: Texture mosaics. Bottom: Our method using γ = 0.14, 0.13, 0.12, and 0.8, respectively. Our method segments theleft and the right image almost perfectly. Even on the challenging central images, the major structures are segmented well.

ages2. We observe that the differently textured regions arenicely separated.

5. CONCLUSION

We have presented a novel approach to the segmentation oftextured images. We used feature vectors based on the am-plitude of monogenic curvelets. For the segmentation of thehigh-dimensional feature images, we used a fast computa-tional strategy for the Potts model. Tests carried out on syn-thetic texture images as well as on real color images show thepotential of our approach.

6. REFERENCES

[1] M. Yang, W. Moon, Y. Wang, M. Bae, C. Huang, J. Chen,and R. Chang, “Robust texture analysis using multi-resolutiongray-scale invariant features for breast sonographic tumor di-agnosis,” IEEE Transactions on Medical Imaging, vol. 32, no.12, pp. 2262–2273, 2013.

[2] A. Depeursinge, A. Foncubierta-Rodriguez, D. Van De Ville,and H. Müller, “Three-dimensional solid texture analysis inbiomedical imaging: Review and opportunities,” Medical Im-age Analysis, vol. 18, no. 1, pp. 176 – 196, 2014.

[3] J. Shotton, J. Winn, C. Rother, and A. Criminisi, “Textonboostfor image understanding: Multi-class object recognition andsegmentation by jointly modeling texture, layout, and context,”

2The images were taken from titanic-magazin.de, zastavki.com,123rf.com.

International Journal of Computer Vision, vol. 81, no. 1, pp. 2–23, 2009.

[4] M. Pietikäinen, Computer vision using local binary patterns,Springer London, 2011.

[5] R. Haralick, “Statistical and structural approaches to texture,”Proceedings of the IEEE, vol. 67, no. 5, pp. 786–804, 1979.

[6] A. Jain and F. Farrokhnia, “Unsupervised texture segmentationusing Gabor filters,” Pattern Recognition, vol. 24, no. 12, pp.1167–1186, 1991.

[7] M. Unser, “Texture classification and segmentation usingwavelet frames,” IEEE Transactions on Image Processing, vol.4, no. 11, pp. 1549–1560, 1995.

[8] T. Randen and J. Husoy, “Filtering for texture classification:A comparative study,” IEEE Transactions on Pattern Analysisand Machine Intelligence, vol. 21, no. 4, pp. 291–310, 1999.

[9] M. Rousson, T. Brox, and R. Deriche, “Active unsupervisedtexture segmentation on a diffusion based feature space,” inProceedings of the IEEE Conference on Computer Vision andPattern Recognition, Madison, 2003, vol. 2, pp. II–699.

[10] M. Ozden and E. Polat, “Image segmentation using color andtexture features,” in Proceedings of the 13th European SignalProcessing Conference, Antalya, 2005, pp. 2226–2229.

[11] J. Santner, M. Unger, T. Pock, C. Leistner, A. Saffari, andH. Bischof, “Interactive texture segmentation using randomforests and total variation,” in British Machine Vision Confer-ence, 2009, pp. 1–12.

[12] S. Geman and D. Geman, “Stochastic relaxation, Gibbs distri-butions, and the Bayesian restoration of images,” IEEE Trans-actions on Pattern Analysis and Machine Intelligence, vol. 6,no. 6, pp. 721–741, 1984.

ICIP 20144351

Fig. 3: Top: Texture mosaics. Bottom: Our method using γ = 0.14, 0.13, 0.12, and 0.8, respectively. Our method segments theleft and the right image almost perfectly. Even on the challenging central images, the major structures are segmented well.

ages2. We observe that the differently textured regions arenicely separated.

5. CONCLUSION

We have presented a novel approach to the segmentation oftextured images. We used feature vectors based on the am-plitude of monogenic curvelets. For the segmentation of thehigh-dimensional feature images, we used a fast computa-tional strategy for the Potts model. Tests carried out on syn-thetic texture images as well as on real color images show thepotential of our approach.

6. REFERENCES

[1] M. Yang, W. Moon, Y. Wang, M. Bae, C. Huang, J. Chen,and R. Chang, “Robust texture analysis using multi-resolutiongray-scale invariant features for breast sonographic tumor di-agnosis,” IEEE Transactions on Medical Imaging, vol. 32, no.12, pp. 2262–2273, 2013.

[2] A. Depeursinge, A. Foncubierta-Rodriguez, D. Van De Ville,and H. Müller, “Three-dimensional solid texture analysis inbiomedical imaging: Review and opportunities,” Medical Im-age Analysis, vol. 18, no. 1, pp. 176 – 196, 2014.

[3] J. Shotton, J. Winn, C. Rother, and A. Criminisi, “Textonboostfor image understanding: Multi-class object recognition andsegmentation by jointly modeling texture, layout, and context,”

2The images were taken from titanic-magazin.de, zastavki.com,123rf.com.

International Journal of Computer Vision, vol. 81, no. 1, pp. 2–23, 2009.

[4] M. Pietikäinen, Computer vision using local binary patterns,Springer London, 2011.

[5] R. Haralick, “Statistical and structural approaches to texture,”Proceedings of the IEEE, vol. 67, no. 5, pp. 786–804, 1979.

[6] A. Jain and F. Farrokhnia, “Unsupervised texture segmentationusing Gabor filters,” Pattern Recognition, vol. 24, no. 12, pp.1167–1186, 1991.

[7] M. Unser, “Texture classification and segmentation usingwavelet frames,” IEEE Transactions on Image Processing, vol.4, no. 11, pp. 1549–1560, 1995.

[8] T. Randen and J. Husoy, “Filtering for texture classification:A comparative study,” IEEE Transactions on Pattern Analysisand Machine Intelligence, vol. 21, no. 4, pp. 291–310, 1999.

[9] M. Rousson, T. Brox, and R. Deriche, “Active unsupervisedtexture segmentation on a diffusion based feature space,” inProceedings of the IEEE Conference on Computer Vision andPattern Recognition, Madison, 2003, vol. 2, pp. II–699.

[10] M. Ozden and E. Polat, “Image segmentation using color andtexture features,” in Proceedings of the 13th European SignalProcessing Conference, Antalya, 2005, pp. 2226–2229.

[11] J. Santner, M. Unger, T. Pock, C. Leistner, A. Saffari, andH. Bischof, “Interactive texture segmentation using randomforests and total variation,” in British Machine Vision Confer-ence, 2009, pp. 1–12.

[12] S. Geman and D. Geman, “Stochastic relaxation, Gibbs distri-butions, and the Bayesian restoration of images,” IEEE Trans-actions on Pattern Analysis and Machine Intelligence, vol. 6,no. 6, pp. 721–741, 1984.

ICIP 20144351

)[Storath2014]

OUTLINE











• Conclusions

• References

REFERENCES

66


L1

L2

M1

M2

M






I. INTRODUCTION




@x1


ØØ¥




of1(x) : of2(x) :

RX

ØØ @ f (x)@x1

ØØdx

R XØ Ø@

f(x)

@x 2

Ø Ø dx

image gradients


|, | @ f (x)@x2

|¥





g (x) = g (Rµ0 x °x0), 8x , x0,µ0 2R2,

·m

TEXTURE OPERATORS AND PRIMITIVES

17

• Texture operators

• A -dimensional texture analysis approach is characterized by a set of local operators centered at the position

• is local in the sense that each element only depends on a subregion of

• The subregion is the support of

• can be linear (e.g., wavelets) or non-linear (e.g., median, GLCMs, LBPs)

• For each position , maps the texture function into a -dimensional space

Nd

f(x)

xL1 ⇥ · · ·⇥ Ld

L1

L2

M1

M2

·

m

gn(x,m) : RM1⇥···⇥Md 7! RP , n = 1, . . . , N�gn(x,m)

�p=1,...,P

gn

m

gn

m gnP

gn(f(x),m) : RM1⇥···⇥Md 7! RP

L1 ⇥ · · ·⇥ Ld gn


18

• From texture operators to texture measurements (i.e., features)

• The operator is typically applied to all positions of the image by “sliding” its window over the image

• Regional texture measurements can be obtained from the aggregation of over a region of interest

• For instance, integration can be used to aggregate over

• e.g., average:

L1

L2M1

M2

L1 ⇥ · · ·⇥ Ld

·

gn(x,m) m

µ 2 RP

gn(f(x),m) M

M

m

gn(f(x),m) M

µ =

0

B@µ1...µP

1

CA =1

|M |

Z

M

�gn(f(x),m)

�p=1,...,P

dm


• Texture primitives

• A “texture primitive” (also called “texel”) is a fundamental elementary unit (i.e., a building block) of a texture class [Haralick1979, Petrou2006]

• Intuitively, given a collection of texture functions , an appropriate set of texture operators must be able to:

(i) Detect and quantify the presence of all distinct primitives in

(ii) Characterize the spatial relationships between the primitives (e.g., geometric transformations, density) when aggregated

texture primitives

,

19

texture primitive

,

fj

�

f1 f2 primitive

,

primitive

,

fj=1,...,J

gn


• General-purpose texture operators

• In general, the texture primitives are neither well-defined, nor known in advance (e.g., biomedical tissue)

• General-purpose operator sets are useful to estimate the primitives

• How to build such operator sets?

20

tissue in HRCT data with optimized SVMs. Lung tissue texture classificationusing co-occurence matrices, Gabor filters and Tamura texture features was in-vestigated in [15]. The classification of regions of interest (ROIs) delineated bythe user consitutes the intial steps towards automatic detection of abnormal lungtissue patterns in the whole HRCT volume.

2 Methods

The dataset used is part of an internal multimedia database of ILD cases con-taining HRCT images with annotated ROIs created in the Talisman project1.843 ROIs from healthy and five pathologic lung tissue patterns are selected fortraining and testing the classifiers selecting classes with sufficiently high repre-sentation (see Table 1).

The wavelet frame decompositions with dyadic and quincunx subsamplingare implemented in Java [11, 16] as well as optimization of SVMs. The basicimplementation of the SVMs is taken from the open source Java library Weka2.

Table 1. Visual aspect and distribution of the ROIs per class of lung tissue pattern.

visualaspect

class healthy emphysema ground glass fibrosis micronodules macronodules

# of ROIs 113 93 148 312 155 22# of patients 11 6 14 28 5 5

3 Results

3.1 Isotropic Polyharmonic B–Spline Wavelets

As mentioned in Section 1.1, isotropic analysis is preferable for lung texturecharacterization. The Laplacian operator plays an important role in image pro-cessing and is clearly isotropic. Indeed, ∆ = ∂2

∂x2

1

+ ∂2

∂x2

2

, is rotationally invariant.

The polyharmonic B–spline wavelets implement a multiscale smoothed versionof the Laplacian [16]. This wavelet, at the first decomposition level, can be char-acterized as

ψγ(D−1x) = ∆γ

2 {φ} (x), (1)

1 TALISMAN: Texture Analysis of Lung ImageS for Medical diagnostic AssistaNce,http://www.sim.hcuge.ch/medgift/01 Talisman EN.htm

2 http://www.cs.waikato.ac.nz/ml/weka/

2-D lung tissue in CT images [Depeursinge2012a]

3-D normal and osteoporotic bone in CT [Dumas2009]µ

• General-purpose texture operators

• The exhaustive analysis of spatial scales and directions is computationally expensive when the support of the operators is large: choices are required

• Directions (e.g., GLCMs, RLE, HOG)

• Scales

the classification accuracy of RLE versus GLCMs for categorizinglung tissue patterns associated with diffuse lung disease in HRCT.Using identical choices of the directions for RLE and GLCMs, theyfound found no statistical differences between the classificationperformance. Gao et al. (2010) and Qian et al. (2011) comparedthe performance of three-dimensional GLCM, LBP, Gabor filtersand WT in retrieving similar MR images of the brain. They ob-served a small increase in retrieval performance for LBP and GLCMwhen compared to Gabor filters and WT. However, the databaseused is rather small and the results might not be statisticallysignificant.

Several papers compared the performance of texture analysisalgorithms in their 2-D versus 3-D forms. As expected, 2-D textureanalysis is most often less discriminative than 3-D, which was ob-served for various applications and techniques, such as:

! GLCMs, RLE and fractal dimension for the classification of lungtissue types in HRCT in Xu et al. (2005, 2006b).

! GLCMs for the classification of brain tumors in MRI in Mah-moud-Ghoneim et al. (2003) and Allin Christe et al. (2012).

! GLCMs for the classification of breast in contrast–enhancedMRI, where statistical significance was assessed in Chen et al.(2007).

! GMRF for the segmentation of gray matter in MRI in Ranguelovaand Quinn (1999).

! LBP for synthetic texture classification in Paulhac et al. (2008).

This demonstrates that 2-D slice-based discrimination of 3-Dnative texture does not allow fully exploiting the informationavailable in 3-D datasets. An exception was observed with 2-D ver-sus 3-D WTs in Jafari-Khouzani et al. (2004), where the 2-D ap-proach showed a small increase in classification performance ofabnormal regions responsible for temporal lobe epilepsy. A separa-ble 3-D WT was used, which did not allow to adequately exploitthe 3-D texture information available and may explain the ob-served results.

7. Discussion

In the preceding sections, we have reviewed the current state-of-the-art in 3-D biomedical texture analysis. The papers were cat-egorized in terms of imaging modality used, organ studied and im-age processing techniques. The increasing number of papers overthe past 32 years clearly shows a growing interest in computerizedcharacterization of three-dimensional texture information (seeFig. 5). This is a consequence of increasingly available 3-D dataacquisition devices that are reaching high spatial resolutionsallowing to capture tissue properties in its natural space.

The analysis of the medical applications in 100 papers in Sec-tion 5 shows the diversity of 3-D biomedical textures. The variousgeometrical properties of the textures are summarized in Table 2,which defines the multiple challenges of 3-D texture analysis.The need for methods able to characterize structural scales and

Fig. 15. Amounts of pixels/voxels covered by the main 13 directions. The number N of discarded pixels/voxels (gray regions in (a) and (b)) according to the neighborhood sizeR is shown in (c).


the classification accuracy of RLE versus GLCMs for categorizinglung tissue patterns associated with diffuse lung disease in HRCT.Using identical choices of the directions for RLE and GLCMs, theyfound found no statistical differences between the classificationperformance. Gao et al. (2010) and Qian et al. (2011) comparedthe performance of three-dimensional GLCM, LBP, Gabor filtersand WT in retrieving similar MR images of the brain. They ob-served a small increase in retrieval performance for LBP and GLCMwhen compared to Gabor filters and WT. However, the databaseused is rather small and the results might not be statisticallysignificant.

Several papers compared the performance of texture analysisalgorithms in their 2-D versus 3-D forms. As expected, 2-D textureanalysis is most often less discriminative than 3-D, which was ob-served for various applications and techniques, such as:

! GLCMs, RLE and fractal dimension for the classification of lungtissue types in HRCT in Xu et al. (2005, 2006b).

! GLCMs for the classification of brain tumors in MRI in Mah-moud-Ghoneim et al. (2003) and Allin Christe et al. (2012).

! GLCMs for the classification of breast in contrast–enhancedMRI, where statistical significance was assessed in Chen et al.(2007).

! GMRF for the segmentation of gray matter in MRI in Ranguelovaand Quinn (1999).

! LBP for synthetic texture classification in Paulhac et al. (2008).

This demonstrates that 2-D slice-based discrimination of 3-Dnative texture does not allow fully exploiting the informationavailable in 3-D datasets. An exception was observed with 2-D ver-sus 3-D WTs in Jafari-Khouzani et al. (2004), where the 2-D ap-proach showed a small increase in classification performance ofabnormal regions responsible for temporal lobe epilepsy. A separa-ble 3-D WT was used, which did not allow to adequately exploitthe 3-D texture information available and may explain the ob-served results.

7. Discussion

In the preceding sections, we have reviewed the current state-of-the-art in 3-D biomedical texture analysis. The papers were cat-egorized in terms of imaging modality used, organ studied and im-age processing techniques. The increasing number of papers overthe past 32 years clearly shows a growing interest in computerizedcharacterization of three-dimensional texture information (seeFig. 5). This is a consequence of increasingly available 3-D dataacquisition devices that are reaching high spatial resolutionsallowing to capture tissue properties in its natural space.

The analysis of the medical applications in 100 papers in Sec-tion 5 shows the diversity of 3-D biomedical textures. The variousgeometrical properties of the textures are summarized in Table 2,which defines the multiple challenges of 3-D texture analysis.The need for methods able to characterize structural scales and

Fig. 15. Amounts of pixels/voxels covered by the main 13 directions. The number N of discarded pixels/voxels (gray regions in (a) and (b)) according to the neighborhood sizeR is shown in (c).



21

2-D 3-D (one quadrant) such that

num

ber o

f dis

card

ed

pixe

ls/v

oxel

sof order −1/2 (an isotropic smoothing operator) of f : Rf =−∇∆−1/2f . Let’s indeed recall the Fourier-domain definition ofthese operators: ∇ F←→ jω and ∆−1/2 F←→ ||ω||−1. Unlike theusual gradient ∇, the Riesz transform is self-reversible

!R⋆Rf(ω) =(jω)∗(jω)

||ω||2 f(ω) = f(ω).

This allows us to define a self-invertible wavelet frame of L2(R3)(tight frame). We however see that there exists a singularity for thefrequency (0, 0, 0). This issue will be fixed later, thanks to the van-ishing moments of the primary wavelet transform.

2.2. Steerability

The interpretation of the Riesz transform as being a directionalderivative filterbank makes its steerability easy to understand: itbehaves similarly to a steerable gradient filterbank, with the addedcrucial property of perfect reconstruction. We parameterize any ro-tation in 3D with a real and unique 3 by 3 matrix U which is unitary(UTU = I). Let us consider the Fourier transform of the impulseresponse of the Riesz transform after a rotation by U as

!R{δ}(Ux)(ω) = −jUω

||Uω|| = U

!−j

ω||ω||

"= U !R{δ}(x)(ω),

with δ the Dirac distribution. The rotated Riesz transform of f there-fore corresponds to the multiplication by U of the non-rotated Rieszcoefficients

RUf(x) = URf(x), (2)

which demonstrates the 3D steerability of the Riesz transform.

2.3. Riesz-Wavelet Pyramid

One crucial property of the Riesz transform is its ability to map anyframe of L2(R3) (in particular wavelet frames) into L2(R3) since itpreserves the inner product of L2(R3) [3, 4]. Following the previousRiesz-wavelet constructions [3, 4], we propose to apply the 3D Riesztransform to the coefficients of a wavelet pyramid to build a steerablewavelet transform in 3D.

2.3.1. Primary Wavelet pyramid

A primary isotropic wavelet pyramid is required in order to pre-serve the relevance of the directional analysis performed by the Riesztransform. Moreover, the bandlimitedness of the wavelet bands mustbe enforced to ensure the isotropy of the primary wavelet togetherwith the possibility of down-sampling [1, 8]. A conventional or-thogonal and separable wavelet transform fulfills none of these con-ditions. In [3], a 2D spline-based wavelet transform was used asthe primary transform. However, while low-order spline waveletsare fast to compute, they are not truly isotropic. We thus proposeinstead a 3D non-separable wavelet with an isotropic wavelet func-tion, as done in 2D in [2]. To achieve bandlimitedness of the waveletbands it is more convenient to design the wavelet transform directlyin the 3D Fourier domain. Moreover, the isotropy constraint im-poses a purely radial wavelet function (i.e., it depends on ||ω|| andnot on the individual frequency components ωi in the Fourier do-main). Among all possible wavelet functions, two are of particularinterest: the Shannon’s wavelet

ψsha(ω) =

#1, π

2 ≤ ||ω|| ≤ π0, otherwise

Fig. 1. Frequency tiling with the Shannon’s wavelet. Each waveletscale is obtained by a bandpass filter of support [π/2k+1,π/2k]. Thespace-domain subsampling operations, which restrict the frequencyplane to the support of each wavelet function, are shown with boxes.

� �

(a) Filterbank implementation of the isotropic wavelet transform. A cas-cade of low-pass filters (Li(ω)) and high-pass filters (H0(ω) and B(ω))is applied. The filterbank is self-reversible.

(b) Self-reversible Riesz-wavelet filterbank.

Fig. 2. The Riesz-wavelet transform filterbank implementation.

and the Simoncelli’s wavelet used for the 2D steerable pyramid

ψsim(ω) =

$cos

%π2 log2

%2||ω||

π

&&, π

4 < ||ω|| ≤ π

0, otherwise.

The Shannon’s wavelet function is a radial step function which corre-sponds to the frequency-domain tiling shown in Fig. 1. This wavelettransform decomposes the signal spectrum with isotropic and non-overlapping tiles. Using the Simoncelli’s wavelet function would re-sult in a smooth frequency partitioning with overlapping tiles, whichis less prone to reconstruction artifacts after coefficient processing.The decomposition shown in Figure 1 can be efficiently achievedby a succession of filtering and downsampling operations, the high-pass coefficient remaining non-subsampled to alleviate aliasing, asopposed to the orthogonal wavelet transform. The wavelet decom-position cascade is illustrated in Figure 2(a).

2.3.2. Riesz-Wavelet Pyramid

We build a Riesz-wavelet frame by applying the Riesz transform toeach scale of the isotropic pyramid defined by the wavelet functionand its dual {ψ, ψ}. The continuous version of the Riesz-wavelettransform prior subsampling is

qk(x) = R{ψk ∗ f}(x)

2133

2-D GLCMs with various spatial distances [Haralick1979]2-D isotropic dyadic wavelets

in Fourier [Chenouard2011]

d = 1 (�k1 = 1,�k2 = 0) d = 2 (�k1 = 2,�k2 = 0) d = 2p2 (�k1 = 2,�k2 = 2)

r

L1 ⇥ · · ·⇥ Ld

L1 = L2 = L3 = 2r + 1

d

g(f(x),m) = g(f(R✓0x� x0),m), 8x0 2 R2, ✓0 2 [0, 2⇡)

• Invariances of the texture operators to geometric transformations can be desirable

• E.g., scaling, rotations and translations

• Invariances of texture operators can be enforced

• Example with 2-D Euclidean transforms (i.e., rotation and translation)with the rotation matrix

INVARIANCE OF TEXTURE OPERATORS

22

R✓0 =

✓cos ✓0 � sin ✓0sin ✓0 cos ✓0

◆

INVARIANCE OF TEXTURE OPERATORS

23

• Computer vision versus biomedical imaging

Computer vision Biomedical image analysis

translation translation-invariant translation-invariant

rotation rotation-invariant rotation-invariant

scale scale-invariant multi-scale

160 IEEE REVIEWS IN BIOMEDICAL ENGINEERING, VOL. 2, 2009

Fig. 10. (a) A digitized histopathology image of Grade 4 CaP and different graph-based representations of tissue architecture via Delaunay Triangulation, VoronoiDiagram, and Minimum Spanning tree.

Fig. 11. Digitized histological image at successively higher scales (magnifica-tions) yields incrementally more discriminatory information in order to detectsuspicious regions.

or resolution. For instance at low or coarse scales color or tex-ture cues are commonly used and at medium scales architec-tural arrangement of individual histological structures (glandsand nuclei) start to become resolvable. It is only at higher res-olutions that morphology of specific histological structures canbe discerned.

In [93], [94], a multiresolution approach has been used for theclassification of high-resolution whole-slide histopathology im-ages. The proposed multiresolution approach mimics the eval-uation of a pathologist such that image analysis starts from thelowest resolution, which corresponds to the lower magnificationlevels in a microscope and uses the higher resolution represen-tations for the regions requiring more detailed information fora classification decision. To achieve this, images were decom-posed into multiresolution representations using the Gaussianpyramid approach [95]. This is followed by color space con-version and feature construction followed by feature extractionand feature selection at each resolution level. Once the classifieris confident enough at a particular resolution level, the systemassigns a classification label (e.g., stroma-rich, stroma-poor orundifferentiated, poorly differentiating, differentiating) to theimage tile. The resulting classification map from all image tilesforms the final classification map. The classification of a whole-slide image is achieved by dividing into smaller image tiles andprocessing each image tile independently in parallel on a clusterof computer nodes.

As an example, refer to Fig. 11, showing a hierarchicalcascaded scheme for detecting suspicious areas on digitizedprostate histopathology slides as presented in [96].

Fig. 12 shows the results of a hierarchical classifier for detec-tion of prostate cancer from digitized histopathology. Fig. 12(a)

Fig. 12. Results from the hierarchical machine learning classifier. (a) Originalimage with the tumor region (ground truth) in black contour, (b) results at scale1, (c) results at scale 2, and (d) results at scale 3. Note that only areas determinedas suspicious at lower scales are considered for further analysis at higher scales.

shows the original image with tumor outlined in black. The nextthree columns show the classifier results at increasing analysisscales. Pixels classified as “nontumor” at a lower magnification(scale) are discarded at the subsequent higher scale, reducingthe number of pixels needed for analysis at higher scales. Ad-ditionally, the presence of more discriminating information athigher scales allows the classifier to better distinguish betweentumor and nontumor pixels.

At lower resolutions of histological imagery, textural analysisis commonly used to capture tissue architecture, i.e., the overallpattern of glands, stroma and organ organization. For each digi-tized histological image several hundred corresponding featurescenes can be generated. Texture feature values are assignedto every pixel in the corresponding image. 3-D statistical, gra-dient, and Gabor filters can be extracted in order to analyzethe scale, orientation, and anisotropic information of the re-gion of interest. Filter operators are applied in order to extractfeatures within local neighborhoods centered at every spatiallocation. At medium resolution, architectural arrangement ofnuclei within each cancer grade can be described via severalgraph-based algorithms. At higher resolutions, nuclei and themargin and boundary appearance of ductal and glandular struc-tures have proved to be of discriminatory importance. Many ofthese features are summarized in Tables I and II.

D. Feature Selection, Dimensionality Reduction,and Manifold Learning

1) Feature Selection: While humans have innate abilities toprocess and understand imagery, they do not tend to excel at


7

• Invariances: computer vision versus biomedical imaging

Computer vision Biomedical image analysis

scale scale-invariant multi-scale

rotation rotation-invariant rotation-invariant

[4] Histopathological image analysis: a review, Gurcan et al., IEEE Reviews in Biomed Eng, 2:147-71, 2009

160 IEEE REVIEWS IN BIOMEDICAL ENGINEERING, VOL. 2, 2009

Fig. 10. (a) A digitized histopathology image of Grade 4 CaP and different graph-based representations of tissue architecture via Delaunay Triangulation, VoronoiDiagram, and Minimum Spanning tree.

Fig. 11. Digitized histological image at successively higher scales (magnifica-tions) yields incrementally more discriminatory information in order to detectsuspicious regions.

or resolution. For instance at low or coarse scales color or tex-ture cues are commonly used and at medium scales architec-tural arrangement of individual histological structures (glandsand nuclei) start to become resolvable. It is only at higher res-olutions that morphology of specific histological structures canbe discerned.

In [93], [94], a multiresolution approach has been used for theclassification of high-resolution whole-slide histopathology im-ages. The proposed multiresolution approach mimics the eval-uation of a pathologist such that image analysis starts from thelowest resolution, which corresponds to the lower magnificationlevels in a microscope and uses the higher resolution represen-tations for the regions requiring more detailed information fora classification decision. To achieve this, images were decom-posed into multiresolution representations using the Gaussianpyramid approach [95]. This is followed by color space con-version and feature construction followed by feature extractionand feature selection at each resolution level. Once the classifieris confident enough at a particular resolution level, the systemassigns a classification label (e.g., stroma-rich, stroma-poor orundifferentiated, poorly differentiating, differentiating) to theimage tile. The resulting classification map from all image tilesforms the final classification map. The classification of a whole-slide image is achieved by dividing into smaller image tiles andprocessing each image tile independently in parallel on a clusterof computer nodes.

As an example, refer to Fig. 11, showing a hierarchicalcascaded scheme for detecting suspicious areas on digitizedprostate histopathology slides as presented in [96].

Fig. 12 shows the results of a hierarchical classifier for detec-tion of prostate cancer from digitized histopathology. Fig. 12(a)

Fig. 12. Results from the hierarchical machine learning classifier. (a) Originalimage with the tumor region (ground truth) in black contour, (b) results at scale1, (c) results at scale 2, and (d) results at scale 3. Note that only areas determinedas suspicious at lower scales are considered for further analysis at higher scales.

shows the original image with tumor outlined in black. The nextthree columns show the classifier results at increasing analysisscales. Pixels classified as “nontumor” at a lower magnification(scale) are discarded at the subsequent higher scale, reducingthe number of pixels needed for analysis at higher scales. Ad-ditionally, the presence of more discriminating information athigher scales allows the classifier to better distinguish betweentumor and nontumor pixels.

At lower resolutions of histological imagery, textural analysisis commonly used to capture tissue architecture, i.e., the overallpattern of glands, stroma and organ organization. For each digi-tized histological image several hundred corresponding featurescenes can be generated. Texture feature values are assignedto every pixel in the corresponding image. 3-D statistical, gra-dient, and Gabor filters can be extracted in order to analyzethe scale, orientation, and anisotropic information of the re-gion of interest. Filter operators are applied in order to extractfeatures within local neighborhoods centered at every spatiallocation. At medium resolution, architectural arrangement ofnuclei within each cancer grade can be described via severalgraph-based algorithms. At higher resolutions, nuclei and themargin and boundary appearance of ductal and glandular struc-tures have proved to be of discriminatory importance. Many ofthese features are summarized in Tables I and II.

D. Feature Selection, Dimensionality Reduction,and Manifold Learning

1) Feature Selection: While humans have innate abilities toprocess and understand imagery, they do not tend to excel at

[Gurcan2009][Lazebnik2005]

OUTLINE











• Conclusions

• References

REFERENCES

66


L1

L2

M1

M2

M






I. INTRODUCTION




@x1


ØØ¥




of1(x) : of2(x) :

RX

ØØ @ f (x)@x1

ØØdx

R XØ Ø@

f(x)

@x 2

Ø Ø dx

image gradients


|, | @ f (x)@x2

|¥





g (x) = g (Rµ0 x °x0), 8x , x0,µ0 2R2,

·m

k2k1

�x1�x2f(k)�x3

k3

MULTISCALE TEXTURE OPERATORS

25

• Inter-patient and inter-dimension scale normalization

• Most medical imaging protocols yield images with various sampling steps

• Inter-patient scale normalization is required to ensure the correspondance of spatial frequencies

• Inter-dimension scale normalization is required to ensure isotropic scale/directions definition

(�x1,�x2,�x3)

�x1 = �x2 = 0.4mm �x1 = �x2 = 1.6mm

spatial Fourier Fourierspatial

0 0

0 0

⇡ ⇡�⇡ �⇡

�⇡ �⇡

�x1�x2

�x3 )�x

01�x

02

�x

03

d = 1 (�k1 = 1,�k2 = 0)

GLCMs

0.4mm

1.6mm


26

• Which scales for texture measurements?

• Two aspects: A. How to define the size(s) of the operator(s) ?

B. How to define the size of the region of interest ?


19

• From texture operators to texture measurements


• Regional texture measurements can be obtained from the aggregation of over a region


• e.g., average:

L1

L2M1

M2

L1 ⇥ · · ·⇥ Ld

·

gn(x,m) m

µ 2 RP

gn(f(x),m) M

M

m

gn(f(x),m) M

µ =

0

B@µ1...µP

1

CA =1

|M |

Z

M

�gn(f(x),m)

�p=1,...,P

dm

M

original image d

increasingly small

operator sizeL1 > L2 > · · · > LN

0

BBBBBBBBBBBBBBBBBBBBBBBB@

µ11...µ1P

µ21...µ2P

...

µN1...

µNP

1

CCCCCCCCCCCCCCCCCCCCCCCCA

= µM: concatenated measurements from multiscale operators aggregated over

Ln1 ⇥ · · ·⇥ Ln

d

gn=1,...,N (f(x),m)

M

f(x)

M

[Depeursinge2012b]


27



B. How to define the size of the region of interest ?


19



• Regional texture measurements can be obtained from the aggregation of over a region


• e.g., average:

L1

L2M1

M2

L1 ⇥ · · ·⇥ Ld

·

gn(x,m) m

µ 2 RP

gn(f(x),m) M

M

m

gn(f(x),m) M

µ =

0

B@µ1...µP

1

CA =1

|M |

Z

M

�gn(f(x),m)

�p=1,...,P

dm

M

original image d

increasingly small


0


µ11...µ1P

µ21...µ2P

...

µN1...

µNP

1



Ln1 ⇥ · · ·⇥ Ln

d

gn=1,...,N (f(x),m)

M

f(x)

M

[Depeursinge2012b]


28

• Size and spectral coverage of operators

• Uncertainty principle: operators cannot be well located both in space and Fourier [Petrou2006]

• In 2D, the trade-off between the spatial support (i.e., ) and frequency support (i.e., ) of the operators is given by

L1 ⇥ L2

⌦1 ⇥ ⌦2

F�!

f(!)f(x)

L21 ⌦

21 L

22 ⌦

22 � 1

16


29


• Becomes a problem in the case of non-stationary texture:

,accurate spatial localization

poor spectrum characterization

Gaussian window:

� = 3.2mm

0 ⇡|!1|

F�!


30





Gaussian window:

� = 3.2mm

Gaussian window:

� = 38.4mm

0 ⇡|!1|

F�!


31





Gaussian window:

� = 3.2mm

Gaussian window:

� = 38.4mm

0 ⇡|!1|

F�!

The spatial support should have the minimum size that allows rich enough texture-specific spectral characterization


32

• Other consequence:

• Large influence of proximal objects when the support of operators is larger than the region of interest

• Example with band-limited operators (2D isotropic wavelets) and lung boundary [Ward2015, Depeursinge2015a]

• Tuning the shape/bandwidth was found to have a strong influence on lung tissue classification accuracy

M

0

0.05

0.1

0.15

0.2

0.25

0.3

0

0.2

0.4

0.6

0.8

1

0

0.2

0.4

0.6

0.8

1

0

5

10

15

20

25

30

35

0

0.05

0.1

0.15

0.2

0

0.5

1

1.5

2

0 ⇡|!1|0 0 ⇡|!1||!1| ⇡x1 x1 x1

f(!)f(x) f(x) f(x)f(!) f(!)


31

• Other consequence:

• Large influence of proximal objects when the support of operators is larger than the region of interest:

• Example with band-limited operators (2D isotropic wavelets) and lung boundary [DPC2015,WPU2015]

• Tuning the shape/bandwidth was found to have a strong influence on lung tissue classification accuracy

L1

L2M1

M2

·

M

m

M

0

0.05

0.1

0.15

0.2

0.25

0.3

0

0.2

0.4

0.6

0.8

1

0

0.2

0.4

0.6

0.8

1

0

5

10

15

20

25

30

35

0

0.05

0.1

0.15

0.2

0

0.5

1

1.5

2

0 ⇡|!1|0 0 ⇡|!1||!1| ⇡x1 x1 x1

f(!)f(x) f(x) f(x)f(!) f(!)

better spatial localizationworse spectral localization


19





• e.g., average:

L1

L2M1

M2

L1 ⇥ · · ·⇥ Ld

·

gn(x,m) m

µ 2 RP

gn(f(x),m) M

m

gn(f(x),m) M

µ =

0

B@µ1...µP

1

CA =1

|M |

Z

M

�gn(f(x),m)

�p=1,...,P

dm


33



B. How to define the size of the region of interest ? M

original image d

increasingly small


0


µ11...µ1P

µ21...µ2P

...

µN1...

µNP

1



Ln1 ⇥ · · ·⇥ Ln

d

gn=1,...,N (f(x),m)

M

f(x)

M

M

[Depeursinge2012b]


• How large must be the region of interest ?

• No more than enough to evaluate texture stationarity in terms of human perception / tissue biology

• Example with operator: undecimated isotropic Simoncelli’s dyadic wavelets [Portilla2000] applied to all image positions

• Operators’ responses are averaged over

M


19





• e.g., average:

L1

L2M1

M2

L1 ⇥ · · ·⇥ Ld

·

gn(x,m) m

µ 2 RP

gn(f(x),m) M

m

gn(f(x),m) M

µ =

0

B@µ1...µP

1

CA =1

|M |

Z

M

�gn(f(x),m)

�p=1,...,P

dm

M

f(x) g1(f(x),m) g2(f(x),m)

original image with regions I

1

|M |

Z

M|g1(f(x),m)|dm

M

feature space

1 |M|Z M

|g2(f(x

),m

)|dm

f(x)

Ma,M b,M c

The averaged responses over the entire image does not correspond to anything visually!

g1(⇢) =

⇢cos

�⇡2 log2

� 2⇢⇡

��, ⇡

4 < ⇢ ⇡0, otherwise.

g2(⇢) =

⇢cos

�⇡2 log2

� 4⇢⇡

��, ⇡

8 < ⇢ ⇡2

0, otherwise.

\g1,2�f(⇢,�)

�= g1,2(⇢,�) · f(⇢,�)

m 2 RM1⇥M2






M


19





• e.g., average:

L1

L2M1

M2

L1 ⇥ · · ·⇥ Ld

·

gn(x,m) m

µ 2 RP

gn(f(x),m) M

m

gn(f(x),m) M

µ =

0

B@µ1...µP

1

CA =1

|M |

Z

M

�gn(f(x),m)

�p=1,...,P

dm

M

f(x) g1(f(x),m)

m 2 RM1⇥M2

g2(f(x),m)


1

|M |

Z

M|g1(f(x),m)|dm

M

feature space

1 |M|Z M

|g2(f(x

),m

)|dm

f(x)

Ma,M b,M c


g1(⇢) =

⇢cos

�⇡2 log2

� 2⇢⇡

��, ⇡


g2(⇢) =

⇢cos

�⇡2 log2

� 4⇢⇡

��, ⇡

8 < ⇢ ⇡2

0, otherwise.

\g1,2�f(⇢,�)

�= g1,2(⇢,�) · f(⇢,�)

Nor biologically!






M


19





• e.g., average:

L1

L2M1

M2

L1 ⇥ · · ·⇥ Ld

·

gn(x,m) m

µ 2 RP

gn(f(x),m) M

m

gn(f(x),m) M

µ =

0

B@µ1...µP

1

CA =1

|M |

Z

M

�gn(f(x),m)

�p=1,...,P

dm

M

f(x) g1(f(x),m)

m 2 RM1⇥M2

g2(f(x),m)


1

|M |

Z

M|g1(f(x),m)|dm

M

feature space

1 |M|Z M

|g2(f(x

),m

)|dm

f(x)

Ma,M b,M c


g1(⇢) =

⇢cos

�⇡2 log2

� 2⇢⇡

��, ⇡


g2(⇢) =

⇢cos

�⇡2 log2

� 4⇢⇡

��, ⇡

8 < ⇢ ⇡2

0, otherwise.

\g1,2�f(⇢,�)

�= g1,2(⇢,�) · f(⇢,�)

Nor biologically!

Define regions that are homogeneous in terms of operators’ responses

(e.g., pixelwise clustering, graph cuts [Malik2001],

Pott’s model [Storath2014])


37


• Example: supervised texture segmentation with undecimated isotropic Simoncelli’s wavelets and linear support vector machines (SVM)

M

feature space (training)

predicted labels

1

|M |

Z

M|g1(f(x),m)|dm

1 |M|Z M

|g2(f(x

),m

)|dm

decision values

-10

-8

-6

-4

-2

0

2

4

6

8

train class 1 (128x128) test image (256x256)train class 2 (128x128)

segmentation error=0.05127


38



M

predicted labels

decision values

-10

-8

-6

-4

-2

0

2

4

6

8

train class 1 (128x128) test image (256x256)train class 2 (128x128)


patch radius0 20 40 60 80 100 120

segm

enta

tion

erro

r

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

0.22

0.24

: circular patch with Mr = 30

: circular patch with


39



M

patch radius0 20 40 60 80 100 120

segm

enta

tion

erro

r

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

0.22

0.24

train class 1 (128x128) train class 2 (128x128) test image (256x256)

r = 8

decision values

-2

-1

0

1

2

3

predicted labels


M


40



M

patch radius0 20 40 60 80 100 120

segm

enta

tion

erro

r

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

0.22

0.24

: circular patch with r = 128

test image (256x256)train class 1 (128x128) train class 2 (128x128) predicted labels

decision values

-1.5

-1

-0.5

0

0.5

1


M


41


• Tissue properties are not homogeneous (i.e., non-stationary) over the entire organ

• Importance of building tissue atlases and digital phenotypes [Depeursinge2015b]

M

the maximum of the score provided by the SVMs. A maximum areaunder the ROC curve (AUC) of 0.81 was obtained with the regionalRiesz attributes, which suggests that prediction was correct for morethan 4 of 5 patients. The performance of HU and GLCM attributeswas close to random (0.54 and 0.6 for HU and GLCMs, respectively).On the other hand, predictive SVM models based on the responses ofthe Riesz filters, averaged over the entire lungs, had an AUC of 0.72.

Our system's performance was also compared with the interpreta-tions of 2 fellowship-trained cardiothoracic fellows, each having 1 yearof experience. Interobserver agreement was assessed with the Cohenκ statistics30 and the percentage of agreement (ie, number of times the2 observer agreed). The comparisons are detailed in Tables 3 and 4.The operating points of the 2 independent observers are reported inFigure 4 (top right). A detailed analysis of the 6 cases that were mis-classified by our system is shown in Table 5 with representative CTimages, including predictions from the computer and the 2 fellows com-pared with the consensus classification. The system predicted 2 classicUIP cases as atypical UIP and 3 atypical UIP cases as classic UIP. A com-prehensive analysis of all 33 cases is illustrated in the SupplementalTable, Supplemental Digital Content 1, http://links.lww.com/RLI/A189.

Overall, 7 incorrect predictions were made by the fellows and 6 incor-rect predictions by the computer. The fellows and the computer madeonly 2 common errors (cases 1 and 13).

DISCUSSIONWe developed a novel computational method for the automated

classification of classic versus atypical UIP based on regional volumet-ric texture analysis. This constitutes, to the best of our knowledge, a firstattempt to automatically differentiate the UIP subtypes with computa-tional methods. An SVM classifier yielded a score that predicts if theUIP is classic or atypical. The classifier was based on a group of attri-butes that characterize the radiological phenotype of the lung paren-chyma, specifically the morphological properties (ie, texture) of theparenchyma. Because diffuse lung diseases can vary in the distributionand severity of abnormalities throughout the lungs, we extracted ourquantiative image features from 36 anatomical regions of the lung. Toour knowledge, adding this spatial characterization to the computationalmodel is also innovative, and it is particularly relevant for assessingdiffuse lung disease.

FIGURE 3. The 36 subregions of the lungs localized the prototype regional distributions of the texture properties. Figure 3 can be viewed online in colorat www.investigativeradiology.com.

FIGURE 4. The ROC analysis of the system's performance. Classic UIP is the positive class. Left, Comparison of various feature groups using the digitallung tissue atlas. Three-dimensional Riesz wavelets provide a superior AUC of 0.81. Right, Importance of the anatomical atlas when comparedwith anapproach based on the global tissue properties and comparison of the computer's and cardiothoracic fellows' performance. Bottom, Probability densityfunctions of the computer score for classic (red) and atypical UIP (blue) based on regional Riesz texture analysis and the computer's operating pointhighlighted in the upper right subfigure. Atypical UIP is associated with a negative score, which implies that positive scores predict classic UIPs with highspecificity. Figure 4 can be viewed online in color at www.investigativeradiology.com.

Depeursinge et al Investigative Radiology • Volume 00, Number 00, Month 2015

4 www.investigativeradiology.com © 2014 Wolters Kluwer Health, Inc. All rights reserved.

Copyright © 2014 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.

M1

M2

= µM1,...,S

OUTLINE











• Conclusions

• References

REFERENCES

66


L1

L2

M1

M2

M






I. INTRODUCTION




@x1


ØØ¥




of1(x) : of2(x) :

RX

ØØ @ f (x)@x1

ØØdx

R XØ Ø@

f(x)

@x 2

Ø Ø dx

image gradients


|, | @ f (x)@x2

|¥





g (x) = g (Rµ0 x °x0), 8x , x0,µ0 2R2,

·m

43

• Which directions for texture measurements?

• Isotropic operators: insensitive to image directions

• Linear:

• Non-linear: e.g., median filter

• Directional operators

• Linear:

Other: e.g., Fourier, circular and spherical harmonics [Unser2013, Ward2014], MR8 [Varma2005], HOG [Dalal2005], Simoncelli’s pyramid [Simoncelli1995], curvelets [Candes2000]

• Non-linear:

Other: e.g., RLE [Galloway1975]

2D Gaussian filter 2D isotropic wavelets [Portilla2000]

MULTIDIRECTIONAL TEXTURE OPERATORS

2D Gaussian derivatives (e.g., Riesz wavelets [Unser2011])@

@x1

@

@x2

@

2

@x

22

@

2

@x

21

@

2

@x1@x2

d = 2 (�k1 = 2,�k2 = 0)

2D GLCMs

d = 2p2 (�k1 = 2,�k2 = 2)

2D LBPs [Ojala2002]

!"#$%& "'(%)! *+ # $',$)% *- ,#&'.! ! !! " !" /0#/ -*,1 #$',$.)#,)2 !211%/,'$ +%'304*, !%/5

6- /0% $**,&'+#/%! *- #$ #,% 7!% !89 /0%+ /0% $**,&'+#/%!*- #& #,% 3':%+ 42 !#! "#$!%'&() "% ! &'"!%'&() ""5 ;'35 <')).!/,#/%! $',$.)#,)2 !211%/,'$ +%'304*, !%/! -*, :#,'*.!7)%!85 =0% 3,#2 :#).%! *- +%'304*,! >0'$0 &* +*/ -#))%(#$/)2 '+ /0% $%+/%, *- "'(%)! #,% %!/'1#/%& 42'+/%,"*)#/'*+5

!"# $%&'()'*+ ,-./01%.2( 3*).-'.*%(

?! /0% -',!/ !/%" /*>#,& 3,#2@!$#)% '+:#,'#+$%9 >% !.4/,#$/9>'/0*./ )*!'+3 '+-*,1#/'*+9 /0% 3,#2 :#).% *- /0% $%+/%,"'(%) 7#$8 -,*1 /0% 3,#2 :#).%! *- /0% $',$.)#,)2 !211%/,'$+%'304*,0**& #& !& $ !% ( ( ( % ) # )"9 3':'+3A

* $ +!#$% #! # #$% #) # #$% ( ( ( % #)#) # #$", !%"

B%(/9 >% #!!.1% /0#/ &'--%,%+$%! #& # #$ #,% '+&%"%+&%+/*- #$9 >0'$0 #))*>! .! /* -#$/*,'C% 7D8A

* % +!#$"+!#! # #$% #) # #$% ( ( ( % #)#) # #$", !*"

6+ ",#$/'$%9 #+ %(#$/ '+&%"%+&%+$% '! +*/ >#,,#+/%&E0%+$%9 /0% -#$/*,'C%& &'!/,'4./'*+ '! *+)2 #+ #"",*('1#/'*+*- /0% F*'+/ &'!/,'4./'*+5 G*>%:%,9 >% #,% >'))'+3 /* #$$%"//0% "*!!'4)% !1#)) )*!! '+ '+-*,1#/'*+ #! '/ #))*>! .! /*#$0'%:% '+:#,'#+$% >'/0 ,%!"%$/ /* !0'-/! '+ 3,#2 !$#)%5B#1%)29 /0% &'!/,'4./'*+ +!#$" '+ 7H8 &%!$,'4%! /0% *:%,#))).1'+#+$% *- /0% '1#3%9 >0'$0 '! .+,%)#/%& /* )*$#) '1#3%/%(/.,% #+&9 $*+!%I.%+/)29 &*%! +*/ ",*:'&% .!%-.) '+@-*,1#/'*+ -*, /%(/.,% #+#)2!'!5 G%+$%9 1.$0 *- /0% '+-*,1#@/'*+ '+ /0% *,'3'+#) F*'+/ 3,#2 )%:%) &'!/,'4./'*+ 7<8 #4*./ /0%/%(/.,#) $0#,#$/%,'!/'$! '! $*+:%2%& 42 /0% F*'+/ &'--%,%+$%&'!/,'4./'*+ JDKLA

* % +!#! # #$% #) # #$% ( ( ( % #&#) # #$", !+"

=0'! '! # 0'30)2 &'!$,'1'+#/':% /%(/.,% *"%,#/*,5 6/,%$*,&! /0% *$$.,,%+$%! *- :#,'*.! "#//%,+! '+ /0% +%'304*,@0**& *- %#$0 "'(%) '+ # ) ,-#./$"#'$01 0'!/*3,#15 ;*,$*+!/#+/ ,%3'*+!9 /0% &'--%,%+$%! #,% C%,* '+ #)) &',%$/'*+!5M+ # !)*>)2 !)*"%& %&3%9 /0% *"%,#/*, ,%$*,&! /0% 0'30%!/&'--%,%+$% '+ /0% 3,#&'%+/ &',%$/'*+ #+& C%,* :#).%! #)*+3/0% %&3% #+&9 -*, # !"*/9 /0% &'--%,%+$%! #,% 0'30 '+ #))&',%$/'*+!5

N'3+%& &'--%,%+$%! #& # #$ #,% +*/ #--%$/%& 42 $0#+3%! '+1%#+ ).1'+#+$%E 0%+$%9 /0% F*'+/ &'--%,%+$% &'!/,'4./'*+ '!'+:#,'#+/ #3#'+!/ 3,#2@!$#)% !0'-/!5 O% #$0'%:% '+:#,'#+$%>'/0 ,%!"%$/ /* /0% !$#)'+3 *- /0% 3,#2 !$#)% 42 $*+!'&%,'+3F.!/ /0% !'3+! *- /0% &'--%,%+$%! '+!/%#& *- /0%', %(#$/ :#).%!A

* % +!-!#! # #$"% -!#) # #$"% ( ( ( % -!#)#) # #$""% !2"

>0%,%

-!." $ )% . & !!% . / !,

!!3"

P2 #!!'3+'+3 # 4'+*1'#) -#$/*, %& -*, %#$0 !'3+ -!#& # #$"9>% /,#+!-*,1 7Q8 '+/* # .+'I.% 01))%! +.14%, /0#/$0#,#$/%,'C%! /0% !"#/'#) !/,.$/.,% *- /0% )*$#) '1#3% /%(/.,%A

01))%! $")#)

&$!

-!#& # #$"%&, !4"

=0% +#1% RS*$#) P'+#,2 T#//%,+U ,%-)%$/! /0% -.+$/'*+@#)'/2 *- /0% *"%,#/*,9 '5%59 # )*$#) +%'304*,0**& '!/0,%!0*)&%& #/ /0% 3,#2 :#).% *- /0% $%+/%, "'(%) '+/* #4'+#,2 "#//%,+5 01))%! *"%,#/*, '! 42 &%-'+'/'*+ '+:#,'#+/#3#'+!/ #+2 1*+*/*+'$ /,#+!-*,1#/'*+ *- /0% 3,#2 !$#)%9'5%59 #! )*+3 #! /0% *,&%, *- /0% 3,#2 :#).%! '+ /0% '1#3%!/#2! /0% !#1%9 /0% *./"./ *- /0% 01))%! *"%,#/*,,%1#'+! $*+!/#+/5

6- >% !%/ 7) $ 5% ! $ )89 >% *4/#'+ 01)5%)9 >0'$0 '!!'1')#, /* /0% 01) *"%,#/*, >% ",*"*!%& '+ JDVL5 =0% />*&'--%,%+$%! 4%/>%%+ 01)5%) #+& 01) #,%A <8 =0% "'(%)! '+/0% +%'304*, !%/ #,% '+&%(%& !* /0#/ /0%2 -*,1 # $',$.)#,$0#'+ #+& D8 /0% 3,#2 :#).%! *- /0% &'#3*+#) "'(%)! #,%&%/%,1'+%& 42 '+/%,"*)#/'*+5 P*/0 1*&'-'$#/'*+! #,% +%$%!@!#,2 /* *4/#'+ /0% $',$.)#,)2 !211%/,'$ +%'304*, !%/9 >0'$0#))*>! -*, &%,':'+3 # ,*/#/'*+ '+:#,'#+/ :%,!'*+ *- 01))%!5

!"! $%&'()'*+ 456.6'5* 3*).-'.*%(

=0% 01))%! *"%,#/*, ",*&.$%! %) &'--%,%+/ *./"./ :#).%!9$*,,%!"*+&'+3 /* /0% %) &'--%,%+/ 4'+#,2 "#//%,+! /0#/ $#+ 4%-*,1%& 42 /0% ) "'(%)! '+ /0% +%'304*, !%/5 O0%+ /0% '1#3%'! ,*/#/%&9 /0% 3,#2 :#).%! #& >')) $*,,%!"*+&'+3)2 1*:%#)*+3 /0% "%,'1%/%, *- /0% $',$)% #,*.+& #!5 N'+$% #! '!#)>#2! #!!'3+%& /* 4% /0% 3,#2 :#).% *- %)%1%+/ 7!% !8 /* /0%,'30/ *- #$ ,*/#/'+3 # "#,/'$.)#, 4'+#,2 "#//%,+ +#/.,#))2,%!.)/! '+ # &'--%,%+/ 01))%! :#).%5 =0'! &*%! +*/ #"")2 /*"#//%,+! $*1",'!'+3 *- *+)2 W! 7*, <!8 >0'$0 ,%1#'+$*+!/#+/ #/ #)) ,*/#/'*+ #+3)%!5 =* ,%1*:% /0% %--%$/ *-,*/#/'*+9 '5%59 /* #!!'3+ # .+'I.% '&%+/'-'%, /* %#$0 ,*/#/'*+'+:#,'#+/ )*$#) 4'+#,2 "#//%,+ >% &%-'+%A

01)23)%! $ 435'!6!!01))%!% 3" ( 3 $ !% )% ( ( ( %6# ))%

!5"

>0%,% !6!!.% 3" "%,-*,1! # $',$.)#, 4'/@>'!% ,'30/ !0'-/ *+/0% ) ,7#8 +.14%, . 3 /'1%!5 6+ /%,1! *- '1#3% "'(%)!9 7K8

!"#$# %& #$'( )*$&+,%-!$*&+!. /,#01-2#$% #.3 ,!&#&+!. +.4#,+#.& &%5&*,% 2$#--+6+2#&+!. 7+&8 $!2#$ 9+.#,0 :#&&%,.- ;<=

6>?' @' 2>ABCDEADF GFHHIJA>B KI>?LMNA GIJG ONA P>OOIAIKJ Q)%!R'

!"#$%& "'(%)! *+ # $',$)% *- ,#&'.! ! !! " !" /0#/ -*,1 #$',$.)#,)2 !211%/,'$ +%'304*, !%/5

6- /0% $**,&'+#/%! *- #$ #,% 7!% !89 /0%+ /0% $**,&'+#/%!*- #& #,% 3':%+ 42 !#! "#$!%'&() "% ! &'"!%'&() ""5 ;'35 <')).!/,#/%! $',$.)#,)2 !211%/,'$ +%'304*, !%/! -*, :#,'*.!7)%!85 =0% 3,#2 :#).%! *- +%'304*,! >0'$0 &* +*/ -#))%(#$/)2 '+ /0% $%+/%, *- "'(%)! #,% %!/'1#/%& 42'+/%,"*)#/'*+5

!"# $%&'()'*+ ,-./01%.2( 3*).-'.*%(

?! /0% -',!/ !/%" /*>#,& 3,#2@!$#)% '+:#,'#+$%9 >% !.4/,#$/9>'/0*./ )*!'+3 '+-*,1#/'*+9 /0% 3,#2 :#).% *- /0% $%+/%,"'(%) 7#$8 -,*1 /0% 3,#2 :#).%! *- /0% $',$.)#,)2 !211%/,'$+%'304*,0**& #& !& $ !% ( ( ( % ) # )"9 3':'+3A

* $ +!#$% #! # #$% #) # #$% ( ( ( % #)#) # #$", !%"

B%(/9 >% #!!.1% /0#/ &'--%,%+$%! #& # #$ #,% '+&%"%+&%+/*- #$9 >0'$0 #))*>! .! /* -#$/*,'C% 7D8A

* % +!#$"+!#! # #$% #) # #$% ( ( ( % #)#) # #$", !*"

6+ ",#$/'$%9 #+ %(#$/ '+&%"%+&%+$% '! +*/ >#,,#+/%&E0%+$%9 /0% -#$/*,'C%& &'!/,'4./'*+ '! *+)2 #+ #"",*('1#/'*+*- /0% F*'+/ &'!/,'4./'*+5 G*>%:%,9 >% #,% >'))'+3 /* #$$%"//0% "*!!'4)% !1#)) )*!! '+ '+-*,1#/'*+ #! '/ #))*>! .! /*#$0'%:% '+:#,'#+$% >'/0 ,%!"%$/ /* !0'-/! '+ 3,#2 !$#)%5B#1%)29 /0% &'!/,'4./'*+ +!#$" '+ 7H8 &%!$,'4%! /0% *:%,#))).1'+#+$% *- /0% '1#3%9 >0'$0 '! .+,%)#/%& /* )*$#) '1#3%/%(/.,% #+&9 $*+!%I.%+/)29 &*%! +*/ ",*:'&% .!%-.) '+@-*,1#/'*+ -*, /%(/.,% #+#)2!'!5 G%+$%9 1.$0 *- /0% '+-*,1#@/'*+ '+ /0% *,'3'+#) F*'+/ 3,#2 )%:%) &'!/,'4./'*+ 7<8 #4*./ /0%/%(/.,#) $0#,#$/%,'!/'$! '! $*+:%2%& 42 /0% F*'+/ &'--%,%+$%&'!/,'4./'*+ JDKLA

* % +!#! # #$% #) # #$% ( ( ( % #&#) # #$", !+"

=0'! '! # 0'30)2 &'!$,'1'+#/':% /%(/.,% *"%,#/*,5 6/,%$*,&! /0% *$$.,,%+$%! *- :#,'*.! "#//%,+! '+ /0% +%'304*,@0**& *- %#$0 "'(%) '+ # ) ,-#./$"#'$01 0'!/*3,#15 ;*,$*+!/#+/ ,%3'*+!9 /0% &'--%,%+$%! #,% C%,* '+ #)) &',%$/'*+!5M+ # !)*>)2 !)*"%& %&3%9 /0% *"%,#/*, ,%$*,&! /0% 0'30%!/&'--%,%+$% '+ /0% 3,#&'%+/ &',%$/'*+ #+& C%,* :#).%! #)*+3/0% %&3% #+&9 -*, # !"*/9 /0% &'--%,%+$%! #,% 0'30 '+ #))&',%$/'*+!5

N'3+%& &'--%,%+$%! #& # #$ #,% +*/ #--%$/%& 42 $0#+3%! '+1%#+ ).1'+#+$%E 0%+$%9 /0% F*'+/ &'--%,%+$% &'!/,'4./'*+ '!'+:#,'#+/ #3#'+!/ 3,#2@!$#)% !0'-/!5 O% #$0'%:% '+:#,'#+$%>'/0 ,%!"%$/ /* /0% !$#)'+3 *- /0% 3,#2 !$#)% 42 $*+!'&%,'+3F.!/ /0% !'3+! *- /0% &'--%,%+$%! '+!/%#& *- /0%', %(#$/ :#).%!A

* % +!-!#! # #$"% -!#) # #$"% ( ( ( % -!#)#) # #$""% !2"

>0%,%

-!." $ )% . & !!% . / !,

!!3"

P2 #!!'3+'+3 # 4'+*1'#) -#$/*, %& -*, %#$0 !'3+ -!#& # #$"9>% /,#+!-*,1 7Q8 '+/* # .+'I.% 01))%! +.14%, /0#/$0#,#$/%,'C%! /0% !"#/'#) !/,.$/.,% *- /0% )*$#) '1#3% /%(/.,%A

01))%! $")#)

&$!

-!#& # #$"%&, !4"

=0% +#1% RS*$#) P'+#,2 T#//%,+U ,%-)%$/! /0% -.+$/'*+@#)'/2 *- /0% *"%,#/*,9 '5%59 # )*$#) +%'304*,0**& '!/0,%!0*)&%& #/ /0% 3,#2 :#).% *- /0% $%+/%, "'(%) '+/* #4'+#,2 "#//%,+5 01))%! *"%,#/*, '! 42 &%-'+'/'*+ '+:#,'#+/#3#'+!/ #+2 1*+*/*+'$ /,#+!-*,1#/'*+ *- /0% 3,#2 !$#)%9'5%59 #! )*+3 #! /0% *,&%, *- /0% 3,#2 :#).%! '+ /0% '1#3%!/#2! /0% !#1%9 /0% *./"./ *- /0% 01))%! *"%,#/*,,%1#'+! $*+!/#+/5

6- >% !%/ 7) $ 5% ! $ )89 >% *4/#'+ 01)5%)9 >0'$0 '!!'1')#, /* /0% 01) *"%,#/*, >% ",*"*!%& '+ JDVL5 =0% />*&'--%,%+$%! 4%/>%%+ 01)5%) #+& 01) #,%A <8 =0% "'(%)! '+/0% +%'304*, !%/ #,% '+&%(%& !* /0#/ /0%2 -*,1 # $',$.)#,$0#'+ #+& D8 /0% 3,#2 :#).%! *- /0% &'#3*+#) "'(%)! #,%&%/%,1'+%& 42 '+/%,"*)#/'*+5 P*/0 1*&'-'$#/'*+! #,% +%$%!@!#,2 /* *4/#'+ /0% $',$.)#,)2 !211%/,'$ +%'304*, !%/9 >0'$0#))*>! -*, &%,':'+3 # ,*/#/'*+ '+:#,'#+/ :%,!'*+ *- 01))%!5

!"! $%&'()'*+ 456.6'5* 3*).-'.*%(

=0% 01))%! *"%,#/*, ",*&.$%! %) &'--%,%+/ *./"./ :#).%!9$*,,%!"*+&'+3 /* /0% %) &'--%,%+/ 4'+#,2 "#//%,+! /0#/ $#+ 4%-*,1%& 42 /0% ) "'(%)! '+ /0% +%'304*, !%/5 O0%+ /0% '1#3%'! ,*/#/%&9 /0% 3,#2 :#).%! #& >')) $*,,%!"*+&'+3)2 1*:%#)*+3 /0% "%,'1%/%, *- /0% $',$)% #,*.+& #!5 N'+$% #! '!#)>#2! #!!'3+%& /* 4% /0% 3,#2 :#).% *- %)%1%+/ 7!% !8 /* /0%,'30/ *- #$ ,*/#/'+3 # "#,/'$.)#, 4'+#,2 "#//%,+ +#/.,#))2,%!.)/! '+ # &'--%,%+/ 01))%! :#).%5 =0'! &*%! +*/ #"")2 /*"#//%,+! $*1",'!'+3 *- *+)2 W! 7*, <!8 >0'$0 ,%1#'+$*+!/#+/ #/ #)) ,*/#/'*+ #+3)%!5 =* ,%1*:% /0% %--%$/ *-,*/#/'*+9 '5%59 /* #!!'3+ # .+'I.% '&%+/'-'%, /* %#$0 ,*/#/'*+'+:#,'#+/ )*$#) 4'+#,2 "#//%,+ >% &%-'+%A

01)23)%! $ 435'!6!!01))%!% 3" ( 3 $ !% )% ( ( ( %6# ))%

!5"

>0%,% !6!!.% 3" "%,-*,1! # $',$.)#, 4'/@>'!% ,'30/ !0'-/ *+/0% ) ,7#8 +.14%, . 3 /'1%!5 6+ /%,1! *- '1#3% "'(%)!9 7K8

!"#$# %& #$'( )*$&+,%-!$*&+!. /,#01-2#$% #.3 ,!&#&+!. +.4#,+#.& &%5&*,% 2$#--+6+2#&+!. 7+&8 $!2#$ 9+.#,0 :#&&%,.- ;<=

6>?' @' 2>ABCDEADF GFHHIJA>B KI>?LMNA GIJG ONA P>OOIAIKJ Q)%!R'

REFERENCES

66


L1

L2

M1

M2

M






I. INTRODUCTION




@x1


ØØ¥




of1(x) : of2(x) :

RX

ØØ @ f (x)@x1

ØØdx

R XØ Ø@

f(x)

@x 2

Ø Ø dx

image gradients


|, | @ f (x)@x2

|¥





g (x) = g (Rµ0 x °x0), 8x , x0,µ0 2R2,

·m

gn(r, ✓,m) 7! gn(r,m)x


44


• Is directional information important for texture discrimination?

F�!

f(!)f(x)

F�!

f(!)f(x)


45


• Importance of the local organization of image directions (LOID)

• i.e., how directional structures intersect

�


46


• Isotropic and unidirectional operators can hardly characterize the LOIDs, especially when aggregated over a region [Sifre2014, Depeursinge2014b]

• Example of feature representation when integrated over entire image

M

isotropic Simoncelli wavelets

scale 1

scal

e 2

o o

GLCM contrast

GLC

M c

ontra

st

d = 1 (�k1 = 1,�k2 = 0)

d=

1(�

k1=

0,�

k2=

1)

GLCMsgradients along and

1

|M |

Z

M

✓@f(x)

@x1

◆2

dx

1 |M|Z M

✓@f(x

)

@x

2

◆2

dx

x1 x2

M ⌘

L1

L2M1

M2

·

M

m L1

L2M1

M2

·

M

m


47


• Isotropic and unidirectional operators can hardly characterize the LOIDs, especially when aggregated over a region [Sifre2014, Depeursinge2014b]

• Example of feature representation when integrated over entire image

M

isotropic Simoncelli wavelets

scale 1

scal

e 2

o o

GLCM contrast

GLC

M c

ontra

st

d = 1 (�k1 = 1,�k2 = 0)

d=

1(�

k1=

0,�

k2=

1)

GLCMsgradients along and

1

|M |

Z

M

✓@f(x)

@x1

◆2

dx

1 |M|Z M

✓@f(x

)

@x

2

◆2

dx

x1 x2

M ⌘

L1

L2M1

M2

·

M

m L1

L2M1

M2

·

M

m

Very poor discrimination! Solutions proposed in a few slides…


48

• Locally rotation-invariant operators over

• Isotropic operators:

• By definition

• Directional:

• Averaging operators’ responses over all directions:

2D GLCMs

µ1 (e.g., GLCM contrast)

No characterization of image directions!

�µ⇡/21µ⇡/4

1 µ3⇡/41µ0

1

L1 ⇥ L2

g(f(x),m) = g(f(R✓0x),m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2

REFERENCES

68


L1

L2

M1

M2

M


49


• Locally “aligning” directional operators

• MR8 filterbank [Varma2005]

• Rotation-invariant LBP [Ojala2002, Ahonen2009]

• Steerable Riesz wavelets [Unser2013, Depeursinge2014b]

L1 ⇥ L2

g(f(x),m) = g(f(R✓0x),m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2


50



• Maximum response 8 (MR8) filterbank [Varma2005]

• Filter responses are obtained for each pixel from the convolution of the filter and the image

• For each position , only the maximum responses among gradient and Laplacian filters are kept

isotropic mutliscale oriented gradients multiscale oriented Laplacians

m

L1 ⇥ L2

g(f(x),m) = g(f(R✓0x),m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2


51



• Maximum response 8 (MR8) filterbank [Varma2005]

• Filter responses are obtained for each pixel from the convolution of the filter and the image

• For each position , only the maximum responses among gradient and Laplacian filters are kept

isotropic mutliscale oriented gradients multiscale oriented Laplacians

m

L1 ⇥ L2

Yields approximate local rotation invariance

Poor characterization of the LOIDs

g(f(x),m) = g(f(R✓0x),m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2


52



• Local binary patterns (LBP) [Ojala2002]

• Rotation-invariant LBP [Ahonen2009]

L1 ⇥ L2

1) define a circular neighborhood 2) Binarize and build a number that encode the LOIDs

)

3) Aggregate over the entire image and count code occurrences

0 170

⌘

4) make codes invariant to circular shifts

U8(1, 0) = 10101010 = 170

U8(1, 0) = 10101010U8(1, 1) = 01010101

m

rotation r

discrete Fourier transform

The new measures are independent of the rotation ) r

H8(1, u) =7X

r=0

hI(U8(1, r))e�j2⇡ur/8

µp = |H8(1, u)|

µ0,p = hI(U8(1, 0))

g(f(x),m) = g(f(R✓0x),m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2


53



• Local binary patterns (LBP) [Ojala2002]

• Rotation-invariant LBP [Ahonen2009]

L1 ⇥ L2

1) define a circular neighborhood 2) Binarize and build a number that encode the LOIDs

)

3) Aggregate over the entire image and count code occurrences

0 170

⌘

4) make codes invariant to circular shifts

U8(1, 0) = 10101010 = 170

U8(1, 0) = 10101010U8(1, 1) = 01010101

m

rotation r

discrete Fourier transform

The new measures are independent of the rotation ) r

H8(1, u) =7X

r=0

hI(U8(1, r))e�j2⇡ur/8

µp = |H8(1, u)|

µ0,p = hI(U8(1, 0))

Encodes the LOIDs independently from their local orientations!

Requires binarization…

Spherical sequences are undefined in 3D…

g(f(x),m) = g(f(R✓0x),m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2


54




• Operators: th-order multi-scale image derivatives

L1 ⇥ L2

input imageN4 A. Depeursinge et al.

N = 1 N = 2

N = 3

Fig. 1. Templates corresponding to the Riesz kernels convolved with a Gaussiansmoother for N=1,2,3.

is determined by the partial derivatives in Eq. (1). Whereas 2N Riesz filtersare generated by (1), only N + 1 components have distinct properties due tocommutativity of the convolution operators in (2) (e.g., ∂2/∂x∂y is equivalentto ∂2/∂y∂x). The Riesz components yield a steerable filterbank [15] allowingto analyze textures in any direction, which is an advantage when compared toclassical Gaussian derivatives or Gabor filters. Qualitatively, the first Riesz com-ponent of even order corresponds to a ridge profile whereas for odd ones we obtainan edge profile, but much richer profiles can be obtained by linear combinationsof the different components. The templates of h1,2(x) convolved with Gaussiankernels for N=1,2,3 are depicted in Fig. 1. The Nth–order Riesz transform canbe coupled with an isotropic multiresolution decomposition (e.g., Laplacian ofGaussian (LoG)) to obtain rotation–covariant (steerable) basis functions [15].

The main idea of the proposed approach is to derive texture signatures frommultiscale Riesz coefficients. An example showing healthy and fibrosis tissuerepresented in terms of their Riesz components with N=2 is depicted in Fig. 2 a).In order to provide a local categorization of the lung parenchyma, lung regionsin 2D axial slices are divided into 32×32 overlapping blocks with a distancebetween contiguous block centers of 16. The Riesz transform is applied to eachblock, and every Riesz component n = 1, . . . , N+1 is mapped to a multiscalerepresentation by convolving them with four LoG filters of scales s = 1, . . . , 4with a dyadic scale progression. In a total of (N+1)×4 subbands, the variancesσn,s of the coefficients are used as texture features along with 22 grey levelhistogram (GLH) bins in [-1050;600] Hounsfield Units (HU). The percentageof air pixels with values ≤ −1000 HU completes the feature space learned bysupport vector machines (SVM) with a Gaussian kernel.

The local dominant texture orientations have an influence on the reparti-tion of respective responses of the Riesz components, which is not desirable forcreating robust features with well–defined clusters of instances. For example, arotation of π/2 will switch the responses of h1 and h2 for N=1. To ensure thatthe repartitions of σn,s are comparable for two similar textures having distinct

N = 1g(1,0)(x,m)

f(x)

g(1,0)(f(x),m)

g(0,1)(x,m)

g(0,1)(f(x),m)

g(f(x),m) = g(f(R✓0x),m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2


55




• Operators: th-order multi-scale image derivatives

L1 ⇥ L2

N4 A. Depeursinge et al.

N = 1 N = 2

N = 3

Fig. 1. Templates corresponding to the Riesz kernels convolved with a Gaussiansmoother for N=1,2,3.

is determined by the partial derivatives in Eq. (1). Whereas 2N Riesz filtersare generated by (1), only N + 1 components have distinct properties due tocommutativity of the convolution operators in (2) (e.g., ∂2/∂x∂y is equivalentto ∂2/∂y∂x). The Riesz components yield a steerable filterbank [15] allowingto analyze textures in any direction, which is an advantage when compared toclassical Gaussian derivatives or Gabor filters. Qualitatively, the first Riesz com-ponent of even order corresponds to a ridge profile whereas for odd ones we obtainan edge profile, but much richer profiles can be obtained by linear combinationsof the different components. The templates of h1,2(x) convolved with Gaussiankernels for N=1,2,3 are depicted in Fig. 1. The Nth–order Riesz transform canbe coupled with an isotropic multiresolution decomposition (e.g., Laplacian ofGaussian (LoG)) to obtain rotation–covariant (steerable) basis functions [15].

The main idea of the proposed approach is to derive texture signatures frommultiscale Riesz coefficients. An example showing healthy and fibrosis tissuerepresented in terms of their Riesz components with N=2 is depicted in Fig. 2 a).In order to provide a local categorization of the lung parenchyma, lung regionsin 2D axial slices are divided into 32×32 overlapping blocks with a distancebetween contiguous block centers of 16. The Riesz transform is applied to eachblock, and every Riesz component n = 1, . . . , N+1 is mapped to a multiscalerepresentation by convolving them with four LoG filters of scales s = 1, . . . , 4with a dyadic scale progression. In a total of (N+1)×4 subbands, the variancesσn,s of the coefficients are used as texture features along with 22 grey levelhistogram (GLH) bins in [-1050;600] Hounsfield Units (HU). The percentageof air pixels with values ≤ −1000 HU completes the feature space learned bysupport vector machines (SVM) with a Gaussian kernel.

The local dominant texture orientations have an influence on the reparti-tion of respective responses of the Riesz components, which is not desirable forcreating robust features with well–defined clusters of instances. For example, arotation of π/2 will switch the responses of h1 and h2 for N=1. To ensure thatthe repartitions of σn,s are comparable for two similar textures having distinct

N = 1 N = 2

N = 3

g(1,0)(x,m) g(2,0)(x,m)g(0,1)(x,m) g(0,2)(x,m)

g(0,3)(x,m)

g(1,1)(x,m)

g(3,0)(x,m) g(2,1)(x,m) g(1,2)(x,m)

g(f(x),m) = g(f(R✓0x),m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2


56




• Steerability:

L1 ⇥ L2

g(1,0)(R✓0x,0) = cos ✓0 g(1,0)(x,0) + sin ✓0 g(0,1)(x,0)

g(f(x),m) = g(f(R✓0x),m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2


57




• Local rotation-invariance:

L1 ⇥ L2

✓max

(m) := argmax

✓02[0,2⇡)

✓cos ✓

0

g(1,0)(f(x),m) + sin ✓

0

g(0,1)(f(x),m)

◆

µM =1

|M |

Z

M

✓g(1,0)(f(R✓

max

(m) x),m)

◆2

dm)

g(f(x),m) = g(f(R✓0x),m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2


58



• Steerable Riesz wavelets [Depeursinge2014b, Unser2013]

• Local rotation-invariance:

L1 ⇥ L2

✓max

(m) := argmax

✓02[0,2⇡)

✓cos ✓

0

g(1,0)(f(x),m) + sin ✓

0

g(0,1)(f(x),m)

◆

µM =1

|M |

Z

M

✓g(1,0)(f(R✓

max

(m) x),m)

◆2

dm)

Encodes the LOIDs independently from their local orientations!

No binarization required!

Available in 3D [Chenouard2012, Depeursinge2015a], and combined with feature learning [Depeursinge2014b].

g(f(x),m) = g(f(R✓0x),m), 8✓0 2 [0, 2⇡), 8x 2 RL1⇥L2

• Operators characterizing the LOIDs


GLCMs Riesz wavelets ( )

59

GLCM contrast

GLC

M c

ontra

st

d = 1 (�k1 = 1,�k2 = 0)

d=

1(�

k1=

0,�

k2=

1)

N = 2

1 |M|Z M

✓g (

2,0)(f(x

),m

)◆2

dm

1

|M |

Z

M

✓g(0,2)(f(x),m)

◆2

dm

aligned Riesz wavelets ( )N = 2

1

|M |

Z

M

✓g(0,2)(f(R✓

max

(m) x),m)

◆2

dm1 |M|Z M

✓g (

2,0)(f(R

✓m

ax

(m)x),m

)◆2

dm

• Operators characterizing the LOIDs


60

GLCMs

GLCM contrast

GLC

M c

ontra

st

d = 1 (�k1 = 1,�k2 = 0)

d=

1(�

k1=

0,�

k2=

1)

Riesz wavelets ( )

1 |M|Z M

✓g (

2,0)(f(x

),m

)◆2

dm

1

|M |

Z

M

✓g(0,2)(f(x),m)

◆2

dm

N = 2

aligned Riesz wavelets ( )N = 2

1

|M |

Z

M

✓g(0,2)(f(R✓

max

(m) x),m)

◆2

dm1 |M|Z M

✓g (

2,0)(f(R

✓m

ax

(m)x),m

)◆2

dm


61

• Isotropic or directional analysis? [Depeursinge2014b]

• Outex [Ojala2002]: 24 classes, 180 images/class, 9 rotation angles in

• Texture classification: linear SVMs trained with unrotated images only


1) canvas001 2) canvas002 3) canvas003 4) canvas005 5) canvas006 6) canvas009 7) canvas011 8) canvas021


17) canvas038 18) canvas039 19) tile005 20) tile006 21) carpet002 22) carpet004 23) carpet005 24) carpet009

Fig. 5. 128⇥ 128 blocks from the 24 texture classes of the Outex database.

1) canvas 2) cloth 3) cotton 4) grass 5) leather 6) matting 7) paper 8) pigskin

9) raffia 10) rattan 11) reptile 12) sand 13) straw 14) weave 15) wood 16) wool

Fig. 6. 16 Brodatz texture classes of the Contrib TC 00000 test suite.

180 ⇥ 180 images from rotation angles 20

�, 70�, 90�, 120�,135

� and 150

� of the other seven Brodatz images for eachclass. The total number of images in the test set is 672.

G. Experimental setupOVA SVM models using Gaussian kernels as K(x

i

,x

j

) =

exp(

�||xi�xj ||22�

2k

) are used both to learn texture signatures andto classify the texture instances in the final feature spaceobtained after k iterations. A number of scales J = 6

was used to cover the whole spectrum of the 128 ⇥ 128

subimages in Outex and J = 3 for covering the spectrum of16 ⇥ 16 subimages in Contrib TC 00000. The angle matrixthat maximizes the response of the texture signature at thesmallest scale ⇥

1

(x) (see Eq. (11)) is used to steer Riesztemplates from all scales. The dimensionality of the initialfeature space is J(N + 1). Every texture signature �

N

c,K

iscomputed using the texture instances from the training set.The coefficients from all instances are rotated to locally aligneach signature �N

c,K

and are concatenated to constitute the finalfeature space. The dimensionality of the final feature space isJ ⇥ (N + 1) ⇥ N

c

. OVA SVM models are trained in thisfinal feature space using the training instances. The remainingtest instances obtained are used to evaluate the generalizationperformance. All data processing was performed using MAT-LAB R2012b (8.0.0.783) 64–bit (glnxa64), The MathWorks

Inc., 2012. The computational complexity is dominated by thelocal orientation of �N

c

in Eq. 11, which consists of finding theroots of the polynomials defined by the steering matrix A

✓.It is therefore NP–hard (Non–deterministic Polynomial–timehard), where the order of the polynomials is controlled by theorder of the Riesz transform N .

III. RESULTS

The performance of our approach is demonstrated withthe Outex and the Brodatz databases. The performance oftexture classification is first investigated in Section III-A.The evolution and the convergence of the texture signatures�

N

c,k

through iterations k = 1, . . . , 10 is then studied inSection III-B for the Outex TC 00010 test suite.

A. Rotation–covariant texture classification

The rotation–covariant properties of our approach are eval-uated using Outex TC 00010, Outex TC 00012 and Con-trib TC 00000 test suites. The classification performanceof the proposed approach after the initial iteration (k=1)is compared with two other approaches that are based onmultiscale Riesz filterbanks. As a baseline, the classificationperformance using the energy of the coefficients of the initialRiesz templates was evaluated. Since the cardinality of the

0"

0.1"

0.2"

0.3"

0.4"

0.5"

0.6"

0.7"

0.8"

0.9"

1"

0" 1" 2" 3" 4" 5" 6" 7" 8" 9" 10"

ini/al"coeffs"

aligned"1st"template"

order of the Riesz transformN

clas

sific

atio

n ac

cura

cy

Riesz waveletsaligned Riesz wavelets


62

• Isotropic or directional analysis? [Depeursinge2014b]

• Outex [Ojala2002]: 24 classes, 180 images/class, 9 rotation angles in











�, 70�, 90�, 120�,135

� and 150



i

,x

j

) =

exp(

�||xi�xj ||22�

2k




1


N

c,K


c,K


c



c



III. RESULTS


N

c,k




0"

0.1"

0.2"

0.3"

0.4"

0.5"

0.6"

0.7"

0.8"

0.9"

1"

0" 1" 2" 3" 4" 5" 6" 7" 8" 9" 10"

ini/al"coeffs"



clas

sific

atio

n ac

cura

cy


Isotropic operators (i.e., ) perform best when not aligned!

N = 0

OUTLINE











• Conclusions

• References

REFERENCES

66


L1

L2

M1

M2

M






I. INTRODUCTION




@x1


ØØ¥




of1(x) : of2(x) :

RX

ØØ @ f (x)@x1

ØØdx

R XØ Ø@

f(x)

@x 2

Ø Ø dx

image gradients


|, | @ f (x)@x2

|¥





g (x) = g (Rµ0 x °x0), 8x , x0,µ0 2R2,

·m

CONCLUSIONS

• We presented a general framework to describe and analyse texture information in 2D and 3D

• Tissue structures in 2D/3D medical images contain extremely rich and valuable information to optimize personalized medicine in a non-invasive way

• Invisible to the naked eye!

64

REFERENCES

66


L1

L2

M1

M2

M






I. INTRODUCTION




@x1


ØØ¥




of1(x) : of2(x) :

RX

ØØ @ f (x)@x1

ØØdx

R XØ Ø@

f(x)

@x 2

Ø Ø dx

image gradients


|, | @ f (x)@x2

|¥





g (x) = g (Rµ0 x °x0), 8x , x0,µ0 2R2,

·m



benign

pre-malignant

undefined

quant. feat. #1

quan

t. fe

at. #

2



• Enough to evaluate texture stationarity in terms of human perception / tissue biology

• Example with operator: undecimated isotropic Simoncelli’s dyadic wavelets [PoS2000] applied to all image positions


M


19





• e.g., average:

L1

L2M1

M2

L1 ⇥ · · ·⇥ Ld

·

gn(x,m) m

µ 2 RP

gn(f(x),m) M

m

gn(f(x),m) M

µ =

0

B@µ1...µP

1

CA =1

|M |

Z

M

�gn(f(x),m)

�p=1,...,P

dm

M

f(x) g1(f(x),m)

m 2 RM1⇥M2

g2(f(x),m)


1

|M |

Z

M|g1(f(x),m)|dm

M

feature space

1 |M|Z M

|g2(f(x

),m

)|dm

f(x)

Ma,M b,M c


g1(⇢) =

⇢cos

�⇡2 log2

� 2⇢⇡

��, ⇡


g2(⇢) =

⇢cos

�⇡2 log2

� 4⇢⇡

��, ⇡

8 < ⇢ ⇡2

0, otherwise.

\g1,2�f(⇢,�)

�= g1,2(⇢,�) · f(⇢,�)

Nor biologically!

BoVW can be used to first reveal the intra-class visual diversity

texture operators region of interestand aggregation

scales

uncertainty principle averaging operators’ responses

directions

isotropic versus directional importance of LOIDs

CONCLUSIONS

• Biomedical textures are realizations of complex non-stationary spatial stochastic processes

• General-purpose image operators are necessary to identify data-specific discriminative scales and directions

65



• Enough to evaluate texture stationarity in terms of human perception / tissue biology

• Example with operator: undecimated isotropic Simoncelli’s dyadic wavelets [PoS2000] applied to all image positions


M


19





• e.g., average:

L1

L2M1

M2

L1 ⇥ · · ·⇥ Ld

·

gn(x,m) m

µ 2 RP

gn(f(x),m) M

m

gn(f(x),m) M

µ =

0

B@µ1...µP

1

CA =1

|M |

Z

M

�gn(f(x),m)

�p=1,...,P

dm

M

f(x) g1(f(x),m)

m 2 RM1⇥M2

g2(f(x),m)


1

|M |

Z

M|g1(f(x),m)|dm

M

feature space

1 |M|Z M

|g2(f(x

),m

)|dm

f(x)

Ma,M b,M c


g1(⇢) =

⇢cos

�⇡2 log2

� 2⇢⇡

��, ⇡


g2(⇢) =

⇢cos

�⇡2 log2

� 4⇢⇡

��, ⇡

8 < ⇢ ⇡2

0, otherwise.

\g1,2�f(⇢,�)

�= g1,2(⇢,�) · f(⇢,�)

Nor biologically!

BoVW can be used to first reveal the intra-class visual diversity

F�!

f(!)f(x)

)TextureQbased'biomarkers:'current'limitaGons'

x  Assume'homogeneous'texture'properGes'over'the'enGre'lesion'[5]'

'

x NonQspecific'features'x  Global'vs'local'characterizaGon'of'image'direcGons'[6]'

Radiology: Volume 269: Number 1—October 2013 n radiology.rsna.org 13

REVIEW: Quantitative Imaging in Cancer Evolution and Ecology Gatenby et al

with the mean signal value. By using just two sequences, a contrast-enhanced T1 sequence and a fluid-attenuated inver-sion-recovery sequence, we can define four habitats: high or low postgadolini-um T1 divided into high or low fluid-at-tenuated inversion recovery. When these voxel habitats are projected into the tu-mor volume, we find they cluster into spatially distinct regions. These habitats can be evaluated both in terms of their relative contributions to the total tumor volume and in terms of their interactions with each other, based on the imaging characteristics at the interfaces between regions. Similar spatially explicit analysis can be performed with CT scans (Fig 5).

Analysis of spatial patterns in cross-sectional images will ultimately re-quire methods that bridge spatial scales from microns to millimeters. One possi-ble method is a general class of numeric tools that is already widely used in ter-restrial and marine ecology research to link species occurrence or abundance with environmental parameters. Species distribution models (48–51) are used to gain ecologic and evolutionary insights and to predict distributions of species or morphs across landscapes, sometimes extrapolating in space and time. They can easily be used to link the environ-mental selection forces in MR imaging-defined habitats to the evolutionary dy-namics of cancer cells.

Summary

Imaging can have an enormous role in the development and implementation of patient-specific therapies in cancer. The achievement of this goal will require new methods that expand and ultimately re-place the current subjective qualitative assessments of tumor characteristics. The need for quantitative imaging has been clearly recognized by the National Cancer Institute and has resulted in for-mation of the Quantitative Imaging Net-work. A critical objective of this imaging consortium is to use objective, repro-ducible, and quantitative feature metrics extracted from clinical images to develop patient-specific imaging-based prog-nostic models and personalized cancer therapies.

rise to local-regional phenotypic adap-tations. Phenotypic alterations can re-sult from epigenetic, genetic, or chro-mosomal rearrangements, and these in turn will affect prognosis and response to therapy. Changes in habitats or the relative abundance of specific ecologic communities over time and in response to therapy may be a valuable metric with which to measure treatment efficacy and emergence of resistant populations.

Emerging Strategies for Tumor Habitat Characterization

A method for converting images to spa-tially explicit tumor habitats is shown in Figure 4. Here, three-dimensional MR imaging data sets from a glioblastoma are segmented. Each voxel in the tumor is defined by a scale that includes its image intensity in different sequences. In this case, the imaging sets are from (a) a contrast-enhanced T1 sequence, (b) a fast spin-echo T2 sequence, and (c) a fluid-attenuated inversion-recov-ery (or FLAIR) sequence. Voxels in each sequence can be defined as high or low based on their value compared

microenvironment can be rewarded by increased proliferation. This evolution-ary dynamic may contribute to distinct differences between the tumor edges and the tumor cores, which frequently can be seen at analysis of cross-sec-tional images (Fig 5).

Interpretation of the subsegmenta-tion of tumors will require computa-tional models to understand and predict the complex nonlinear dynamics that lead to heterogeneous combinations of radiographic features. We have ex-ploited ecologic methods and models to investigate regional variations in cancer environmental and cellular properties that lead to specific imaging character-istics. Conceptually, this approach as-sumes that regional variations in tumors can be viewed as a coalition of distinct ecologic communities or habitats of cells in which the environment is governed, at least to first order, by variations in vascular density and blood flow. The environmental conditions that result from alterations in blood flow, such as hypoxia, acidosis, immune response, growth factors, and glucose, represent evolutionary selection forces that give

Figure 4

Figure 4: Left: Contrast-enhanced T1 image from subject TCGA-02-0034 in The Cancer Genome Atlas–Glioblastoma Multiforme repository of MR volumes of glioblastoma multiforme cases. Right: Spatial distribution of MR imaging–defined habitats within the tumor. The blue region (low T1 postgadolinium, low fluid-attenuated inversion recovery) is particularly notable because it presumably represents a habitat with low blood flow but high cell density, indicating a population presumably adapted to hypoxic acidic conditions.





Summary







Figure 4


[5]'QuanGtaGve'imaging'in'cancer'evoluGon'and'ecology,'Gatenby'et'al.,'Radiology,'269(1):8Q15,'2013'

5'

global'direcGonal'operators:' local'grouped'steering:'

[6]'RotaGonQcovariant'texture'learning'using'steerable'Riesz'wavelets,'Depeursinge'et'al.,'IEEE'Trans'Imag'Proc.,'23(2):898Q908,'2014.'

TextureQbased'biomarkers:'current'limitaGons'

x  Assume'homogeneous'texture'properGes'over'the'enGre'lesion'[5]'

'

x NonQspecific'features'x  Global'vs'local'characterizaGon'of'image'direcGons'[6]'





Summary







Figure 4






Summary







Figure 4


[5]'QuanGtaGve'imaging'in'cancer'evoluGon'and'ecology,'Gatenby'et'al.,'Radiology,'269(1):8Q15,'2013'

5'

global'direcGonal'operators:' local'grouped'steering:'

[6]'RotaGonQcovariant'texture'learning'using'steerable'Riesz'wavelets,'Depeursinge'et'al.,'IEEE'Trans'Imag'Proc.,'23(2):898Q908,'2014.'


58

• Isotropic or directional analysis? [DFV2014]

• Outex [OPM2002]: 24 classes, 180 images/class, 9 rotation angles in











�, 70�, 90�, 120�,135

� and 150



i

,x

j

) =

exp(

�||xi�xj ||22�

2k




1


N

c,K


c,K


c



c



III. RESULTS


N

c,k




0"

0.1"

0.2"

0.3"

0.4"

0.5"

0.6"

0.7"

0.8"

0.9"

1"

0" 1" 2" 3" 4" 5" 6" 7" 8" 9" 10"

ini/al"coeffs"



clas

sifica

tion

accu

racy



N = 0


58

• Isotropic or directional analysis? [DFV2014]

• Outex [OPM2002]: 24 classes, 180 images/class, 9 rotation angles in











�, 70�, 90�, 120�,135

� and 150



i

,x

j

) =

exp(

�||xi�xj ||22�

2k




1


N

c,K


c,K


c



c



III. RESULTS


N

c,k




0"

0.1"

0.2"

0.3"

0.4"

0.5"

0.6"

0.7"

0.8"

0.9"

1"

0" 1" 2" 3" 4" 5" 6" 7" 8" 9" 10"

ini/al"coeffs"



clas

sifica

tion

accu

racy



N = 0

THANKS ! BIG @ EPFLMichael UnserJulien FageotArash Aminiand all members

MedGIFT @ HES-SO Henning MüllerYashin DicenteRoger SchaerRanveer Joyseree Oscar JimenezManfredo Atzori

66

Source code and data available!https://sites.google.com/site/btamiccai2015/

[email protected]

FUNDAMENTALS OF DIGITAL TEXTURE PROCESSING FOR BIOMEDICAL IMAGE ANALYSIS

Adrien Depeursinge, PhD MICCAI 2015 Tutorial on Biomedical Texture Analysis (BTA), Munich, Oct 5 2015













































sampled

lattices)is

notstraightforw

ardand

raisesseveral

chal-lenges

relatedto

translation,scalingand

rotationinvariances

andcovariances

thatare

becoming

more

complex

in3-D

.

1.2.Relatedwork

andscope

ofthis

survey

Depending

onresearch

communities,

varioustaxonom

iesare

usedto

referto

3-Dtexture

information.A

clarificationof

thetax-

onomyis

proposedin

thissection

toaccurately

definethe

scopeof

thissurvey.Itis

partlybased

onToriw

akiandYoshida,2009.Three-

dimensional

textureand

volumetric

textureare

bothgeneral

andidenticalterm

sdesigning

atexture

definedin

R3and

include:

(1)Volum

etrictextures

existingin

‘‘filled’’objects

fV:x;y

;z2V

x;y;z !

R3g

thatare

generatedby

avolum

etricdata

acquisitiondevice

(e.g.,tomography,confocalim

aging).(2)

2.5Dtextures

existingon

surfacesof

‘‘hollow’’objects

asfC

:x;y;z

2C

u;v!

R3g,

(3)Dynam

ictextures

intw

o–dimensional

time

sequencesas

fS:x;y

;t2Sx;y

;t !R

3g,

Solidtexture

refersto

category(1)

andaccounts

fortextures

de-fined

inavolum

eVx,y,z indexed

bythree

coordinates.Solidtextures

havean

intrinsicdim

ensionof

3,which

means

thatanum

berof

variablesequal

tothe

dimensionality

ofthe

Euclideanspace

isneeded

torepresent

thesignal(Bennett,1965;

Foncubierta-Rodrí-guez

etal.,2013a).Category

(2)is

designedas

texturedsurface

inDana

andNayar

(1999)and

Culaand

Dana

(2004),or2.5-dim

en-sional

texturesin

Luet

al.(2006)and

Aguet

etal.(2008),w

heretextures

Care

existingon

thesurface

of3-Dobjects

andcan

bein-

dexeduniquely

bytw

ocoordinates

(u,v).(2)is

alsoused

inKajiya

andKay

(1989),Neyret

(1995),andFilip

andHaindl(2009),w

here3-D

geometries

areadded

ontothe

surfaceofobjects

tocreate

real-istic

renderingofvirtualscenes.M

otionanalysis

invideos

canalso

beconsidered

amulti-dim

ensionaltexture

analysisproblem

belongingto

category(3)

andis

designedby

‘‘dynamic

texture’’in

Bouthemyand

Fablet(1998),Chom

atand

Crowley

(1999),andSchödlet

al.(2000).In

thissurvey,

acom

prehensivereview

ofthe

literaturepub-

lishedon

classificationand

retrievalof

biomedical

solidtextures

(i.e.,category(1))is

carriedout.The

focusofthis

textis

onthe

fea-ture

extractionand

notmachine

learningtechniques,

sinceonly

featureextraction

isspecific

to3-D

solidtexture

analysis.

1.3.Structureof

thisarticle

Thissurvey

isstructured

asfollow

s:The

fundamentals

of3-D

digitaltexture

processingare

definedin

Section2.

Section3de-

scribesthe

reviewmethodology

usedto

systematically

retrievepa-

persdealing

with

3-Dsolid

textureclassification

andretrieval.The

imaging

modalities

andorgans

studiedin

theliterature

arere-

viewed

inSections

4and

5,respectivelyto

listthe

variousexpecta-

tionsand

needsof3-D

image

processing.Theresulting

application-driven

techniquesare

described,organizedand

groupedtogether

inSection

6.Asynthesis

ofthe

trendsand

gapsof

thevarious

ap-proaches,

conclusionsand

opportunitiesare

givenin

Section7,

respectively.

2.Fundam

entals

ofsolid

texture

processing

Although

severalresearchers

attempted

toestablish

ageneral

model

oftexture

description(H

aralick,1979;

Julesz,1981),

itis

generallyrecognized

thatno

generalmathem

aticalmodel

oftex-

turecan

beused

tosolve

everyim

ageanalysis

problem(M

allat,1999).

Inthis

survey,wecom

parethe

variousapproaches

based

onthe

3-Dgeom

etricalproperties

ofthe

primitives

used,i.e.,theelem

entarybuilding

blockconsidered.The

setof

primitives

usedand

theirassum

edinteractions

definethe

propertiesofthe

textureanalysis

approaches,fromstatisticalto

structuralmethods.

InSection

2.1,we

definethe

mathem

aticalfram

ework

andnotations

consideredto

describethe

contentof3-D

digitalimages.

Thenotion

oftexture

primitives

aswell

astheir

scalesand

direc-tions

aredefined

inSection

2.2.

2.1.3-Ddigitized

images

andsam

pling

InCartesian

coordinates,ageneric

3-Dcontinuous

image

isde-

finedby

afunction

ofthree

variablesf(x,y,z),w

herefrepresents

ascalar

atapointðx;y

;zÞ2R

3.A3-D

digitalimage

F(i,j,k)ofdim

en-sions

M$

N$

Ois

obtainedfrom

sampling

fat

pointsði;j;kÞ2

Z3

ofa3-D

orderedarray

(seeFig.2).Increm

entsin

(i,j,k),correspondto

physicaldisplacem

entsin

R3param

etrizedby

therespective

spacings(D

x,Dy,D

z).Forevery

cellofthedigitized

array,thevalue

ofF(i,j,k)is

typicallyobtained

byaveraging

finthe

cuboiddom

aindefined

by(x,y,z)2

[iDx,(i+

1)Dx];

[jDy,(j+

1)Dy];

[kDz,(k

+1)D

z])(Toriw

akiandYoshida,2009).This

cuboidis

calledavoxel.

Thethree

sphericalcoordinates

(r,h,/)are

unevenlysam

pledto

(R,H,U

)as

shownin

Fig.3.

2.2.Textureprim

itives

Thenotion

oftextureprim

itivehas

beenwidely

usedin

2-Dtex-

tureanalysis

anddefines

theelem

entarybuilding

blockof

agiven

textureclass

(Haralick,1979;

Jainet

al.,1995;Lin

etal.,1999).A

lltexture

processingapproaches

aimat

modeling

agiven

textureusing

setsofprototype

primitives.The

conceptoftextureprim

itiveis

naturallyextended

in3-D

asthe

geometry

ofthevoxelsequence

usedby

agiven

textureanalysis

method.W

econsider

aprim

itiveC(i,j,k)

centeredat

apoint

(i,j,k)that

liveson

aneighborhood

ofthis

point.Theprim

itiveis

constitutedby

aset

ofvoxelswith

graytone

valuesthat

formsa3-D

structure.TypicalC

neighborhoodsare

voxelpairs,

linear,planar,

sphericalor

unconstrained.Signal

assignment

tothe

primitive

canbe

eitherbinary,

categoricalor

continuous.Tw

oexam

pletexture

primitives

areshow

nin

Fig.4.Texture

primitives

referto

localprocessingof3-D

images

andlocal

patterns(see

Toriwakiand

Yoshida,2009).

Fig.2.3-D

digitizedim

agesand

sampling

inCartesian

coordinates.

Fig.3.3-D

digitizedim

agesand

sampling

insphericalcoordinates.

178A.D

epeursingeet

al./MedicalIm

ageAnalysis

18(2014)

176–196

Stanford UniversityDaniel RubinOlivier GevaertAnn Leung

Dimitri Van de Ville, UNIGECamille Kurtz, Paris Descartes Pierre-Alexandre Poletti, HUGJohn-Paul Ward, UCF

REFERENCES (SORTING IN ALPHABETICAL ORDER)

67

[Ahonen2009] Ahonen, T.; Matas, J.; He, C. & Pietikäinen, M. Salberg, A.-B.; Hardeberg, J. & Jenssen, R. (Eds.)Rotation Invariant Image Description with Local Binary Pattern Histogram Fourier Features Image Analysis, Springer Berlin Heidelberg, 2009, 5575, 61-70

[Aerts2014] Aerts, H. J. W. L.; Velazquez, E. R.; Leijenaar, R. T. H.; Parmar, C.; Grossmann, P.; Carvalho, S.; Bussink, J.; Monshouwer, R.; Haibe-Kains, B.; Rietveld, D.; Hoebers, F.; Rietbergen, M. M.; Leemans, C. R.; Dekker, A.; Quackenbush, J.; Gillies, R. J. & Lambin, P. Decoding Tumour Phenotype by Noninvasive Imaging Using a Quantitative Radiomics Approach Nature Communications, Nature Publishing Group, a division of Macmillan Publishers Limited. All Rights Reserved, 2014, 5

[Blakemore1969] Blakemore, C. & Campbell, F. W. On the Existence of Neurones in the Human Visual System Selectively Sensitive to the Orientation and Size of Retinal ImagesThe Journal of Physiology, 1969, 203, 237-260

[Candes2000] Candès, E. J. & Donoho, D. L. Curvelets - A Surprisingly Effective Nonadaptive Representation For Objects with Edges Curves and Surface Fitting, Vanderbilt University Press, 2000, 105-120

[Chenouard2011] Chenouard, N. & Unser, M. 3D Steerable Wavelets and Monogenic Analysis for Bioimaging 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, 2011, 2132-2135

[Chenouard2012] Chenouard, N. & Unser, M. 3D Steerable Wavelets in Practice IEEE Transactions on Image Processing, 2012, 21, 4522-4533


68

[Dalal2005] Dalal, N. & Triggs, B. Histograms of Oriented Gradients for Human Detection Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), IEEE Computer Society, 2005, 1, 886-893

[Depeursinge2012a] Depeursinge, A.; Foncubierta-Rodrguez, A.; Van De Ville, D. & Müller, H. Multiscale Lung Texture Signature Learning Using The Riesz Transform Medical Image Computing and Computer-Assisted Intervention MICCAI 2012, Springer Berlin / Heidelberg, 2012, 7512, 517-524

[Depeursinge2012b] Depeursinge, A.; Van De Ville, D.; Platon, A.; Geissbuhler, A.; Poletti, P.-A. & Müller, H. Near-Affine-Invariant Texture Learning for Lung Tissue Analysis Using Isotropic Wavelet Frames IEEE Transactions on Information Technology in BioMedicine, 2012, 16, 665-675

[Depeursinge2014a] Depeursinge, A.; Foncubierta-Rodrguez, A.; Van De Ville, D. & Müller, H. Three-Dimensional Solid Texture Analysis and Retrieval in Biomedical Imaging: Review and Opportunities Medical Image Analysis, 2014, 18, 176-196

[Depeursinge2014b] Depeursinge, A.; Foncubierta-Rodrguez, A.; Van De Ville, D. & Müller, H. Rotation-Covariant Texture Learning Using Steerable Riesz Wavelets IEEE Transactions on Image Processing, 2014, 23, 898-908

[Depeursinge2015a] Depeursinge, A.; Pad, P.; Chin, A. C.; Leung, A. N.; Rubin, D. L.; Müller, H. & Unser, M. Optimized Steerable Wavelets for Texture Analysis of Lung Tissue in 3-D CT: Classification of Usual Interstitial PneumoniaIEEE 12th International Symposium on Biomedical Imaging, IEEE, 2015, 403-406

[Depeursinge2015b] Depeursinge, A.; Chin, A. C.; Leung, A. N.; Terrone, D.; Bristow, M.; Rosen, G. & Rubin, D. L. Automated Classification of Usual Interstitial Pneumonia Using Regional Volumetric Texture Analysis in High-Resolution CTInvestigative Radiology, 2015, 50, 261-267


69

[Dumas2009] Dumas, A.; Brigitte, M.; Moreau, M.; Chrétien, F.; Baslé, M. & Chappard, D.Bone Mass and Microarchitecture of Irradiated and Bone Marrow-Transplanted Mice: Influences of the Donor StrainOsteoporosis International, Springer-Verlag, 2009, 20, 435-443

[Galloway1975] Galloway, M. M.Texture Analysis Using Gray Level Run LengthsComputer Graphics and Image Processing, 1975, 4, 172-179 (Gal1975)

[Gerlinger2012] Gerlinger, M.; Rowan, A. J.; Horswell, S.; Larkin, J.; Endesfelder, D.; Gronroos, E.; Martinez, P.; Matthews, N.; Stewart, A.; Tarpey, P.; Varela, I.; Phillimore, B.; Begum, S.; McDonald, N. Q.; Butler, A.; Jones, D.; Raine, K.; Latimer, C.; Santos, C. R.; Nohadani, M.; Eklund, A. C.; Spencer-Dene, B.; Clark, G.; Pickering, L.; Stamp, G.; Gore, M.; Szallasi, Z.; Downward, J.; Futreal, P. A. & Swanton, C.Intratumor Heterogeneity and Branched Evolution Revealed by Multiregion Sequencing New England Journal of Medicine, 2012, 366, 883-892

[Gurcan2009] Gurcan, M. N.; Boucheron, L. E.; Can, A.; Madabhushi, A.; Rajpoot, N. M. & Yener, B. Histopathological Image Analysis: A Review IEEE Reviews in Biomedical Engineering, 2009, 2, 147-171

[Haidekker2011] Haidekker, M. A. Texture AnalysisAdvanced Biomedical Image Analysis, John Wiley & Sons, Inc., 2010, 236-275

[Haralick1979] Haralick, R. M.Statistical and Structural Approaches to Texture Proceedings of the IEEE, 1979, 67, 786-804


70

[Khan2013] Khan, A. M.; El-Daly, H.; Simmons, E. & Rajpoot, N. M.HyMaP: A Hybrid Magnitude-Phase Approach to Unsupervised Segmentation of Tumor Areas in Breast Cancer Histology ImagesJournal of pathology informatics, Medknow Publications, 2013, 4

[Kumar2012] Kumar, V.; Gu, Y.; Basu, S.; Berglund, A.; Eschrich, S. A.; Schabath, M. B.; Forster, K.; Aerts, H. J. W. L.; Dekker, A.; Fenstermacher, D.; Goldgof, D. B.; Hall, L. O.; Lambin, P.; Balagurunathan, Y.; Gatenby, R. A. & Gillies, R. J. Radiomics: The Process and the ChallengesMagnetic Resonance Imaging, 2012, 30, 1234-1248

[Lazebnik2005] Lazebnik, S.; Schmid, C. & Ponce, J. A Sparse Texture Representation Using Local Affine Regions IEEE Transactions on Pattern Analysis and Machine Intelligence, IEEE Computer Society, 2005, 27, 1265-1278

[Malik2001] Malik, J.; Belongie, S.; Leung, T. & Shi, J. Contour and Texture Analysis for Image Segmentation International Journal of Computer Vision, Kluwer Academic Publishers, 2001, 43, 7-27

[Mosquera2015] Mosquera-Lopez, C.; Agaian, S.; Velez-Hoyos, A. & Thompson, I. Computer-Aided Prostate Cancer Diagnosis From Digitized Histopathology: A Review on Texture-Based SystemsIEEE Reviews in Biomedical Engineering, 2015, 8, 98-113

[Ojala2002] Ojala, T.; Pietikäinen, M. & Mäenpää, T. Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24, 971-987


71

[Petrou2006] Petrou, M. & Garcia Sevilla, P.Image Processing: Dealing with TextureWiley, 2006

[Petrou2011] Petrou, M.Deserno, T. M. (Ed.) Texture in Biomedical Images Biomedical Image Processing, Springer-Verlag Berlin Heidelberg, 2011, 157-176

[Portilla2000] Portilla, J. & Simoncelli, E. P.A Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coefficients International Journal of Computer Vision, Kluwer Academic Publishers, 2000, 40, 49-70

[Romeny2011] ter Haar Romeny, B. M.Deserno, T. M. (Ed.) Multi-Scale and Multi-Orientation Medical Image Analysis Biomedical Image Processing, Springer-Verlag Berlin Heidelberg, 2011, 177-196

[Sifre2014] Sifre, L. & Mallat, S.Rigid-Motion Scattering for Texture Classification Submitted to International Journal of Computer Vision, 2014, abs/1403.1687, 1-19

[Simoncelli1995] Simoncelli, E. P. & Freeman, W. T. The Steerable Pyramid: A Flexible Architecture for Multi-Scale Derivative ComputationProceedings of International Conference on Image Processing, 1995, 3, 444-447

[Storath2014] Storath, M.; Weinmann, A. & Unser, M. Unsupervised Texture Segmentation Using Monogenic Curvelets and the Potts Model IEEE International Conference on Image Processing (ICIP), 2014, 4348-4352


72

[Unser2011] Unser, M.; Chenouard, N. & Van De Ville, D.Steerable Pyramids and Tight Wavelet Frames in IEEE Transactions on Image Processing, 2011, 20, 2705-2721

[Unser2013] Unser, M. & Chenouard, N. A Unifying Parametric Framework for 2D Steerable Wavelet Transforms SIAM Journal on Imaging Sciences, 2013, 6, 102-135

[Varma2005] Varma, M. & Zisserman, A. A Statistical Approach to Texture Classification from Single ImagesInternational Journal of Computer Vision, Kluwer Academic Publishers, 2005, 62, 61-81

[Ward2014] Ward, J. & Unser, M.Harmonic Singular Integrals and Steerable Wavelets in Applied and Computational Harmonic Analysis, 2014, 36, 183-197

[Ward2015] Ward, J.; Pad, P. & Unser, M. Optimal Isotropic Wavelets for Localized Tight Frame Representations IEEE Signal Processing Letters, 2015, 22, 1918-1921

[Xu2012] Xu, J.; Napel, S.; Greenspan, H.; Beaulieu, C. F.; Agrawal, N. & Rubin, D. Quantifying the Margin Sharpness of Lesions on Radiological Images for Content-Based Image Retrieval Medical Physics, 2012, 39, 5405-5418

L2(Rd)

L2(Rd)