Temporal Enhance Ultrasound: A Novel Paradigm to Enable ... · S .Azizi et al , “Detectionand Grading of Prostate Cancer Using Temporal Enhanced Ultrasound: Combining Deep Neural

Temporal Enhance Ultrasound: A Novel Paradigm to Enable Accurate

TRUS-guided Biopsy and Grading

Shekoofeh Azizi

University of British Columbia, Vancouver, BC, Canada

Outline

TeUS Framework

TeUS Application

In vivo Studies and Technical Developments

Theoretical Derivation of TeUS

Experiments and Simulations − Pathology mimicking simulation

− Tissue Phantoms

− Tissue mimicking phantoms

2

Temporal Enhanced Ultrasound (TeUS)

3

Temporal Enhanced Ultrasound

Cancer

Benign

Feature Learning

Classification

Deep Learning

4

TeUS Applications

5

ex vivo PCa detection [Moradi2009]

Classifying PCa grades [Azizi2016]

Breast cancer diagnosis [Uniyal2013]

In vivo Studies

6

Data Acquisition

Data was acquired at the National Institutes of Health (NIH), Maryland.

MR-Ultrasound fusion (UroNav Invivo Corp.)

Taking biopsy & histological processing

Temporal Enhanced US data

MR Target

197 TRUS-guided biopsy cores from 132 subjects.

7

Method Overview

Training Dataset

20 ROIs

Target

Deep Belief Network (DBN)

Visible Layer Hidden Layers

Feature Space

GS3 GS4

Distribution Learning (F1,F2)

Clustering Model

Trained Deep Network

Clustering

Model

Test Data ?

?

?

?

?

? ?

?

?

?

?

?

?

?

??

?

?

?

?

Feature 1

Feat

ure

2

Cluster of Gleason Pattern 3

Cluster of Gleason Pattern 4

Benign Cluster

S. Azizi et al., “Detection and Grading of Prostate Cancer Using Temporal Enhanced Ultrasound: Combining Deep Neural Networks and Tissue Mimicking Simulation”, MICCAI Special Issue 2017.

8

PCa Grading Results

9

Benign vs. clinically significant PCa: 0.8 Combination Rule:

− Intermediate suspicious level: TeUS (70% of the cores) − High and low suspicious level: MRI (30% of the cores)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Are

a U

nd

er

the

RO

C C

urv

e (

AU

C)

Clustering

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Are

a U

nd

er

the

RO

C C

urv

e (

AU

C)

Clustering

Clustering+mp-MRI

Length of tumor in MR ≥ 2cm


Cancer Likelihood Maps

10

MRI lesion length = 24 mm, GS ≤ 3+4

MRI lesion length = 36 mm, GS ≤ 3+4

Target

MRI lesion length = 17 mm, GS ≥ 4+3

Target

MRI lesion length = 27 mm, Benign Target

Target

Target

(Benign blue, G3 yellow, G4 red) 2nd Sample GS: 4+4

1st Sample GS: 3+3


B-mode TeUS Development

RF-mimicking B-mode TeUS Data

Layer 1

Layer 2

. . .

. . .

Representation Sequence . . .

x1 xT

Benign vs. Cancer

xi = (x1, …, xT)

FC

LSTM cell

FC Fully Connected Layer

Transfer Learning

Bmode TeUS Data RF TeUS Data

We utilize joint information of RF and B-mode data through transfer learning.

S. Azizi et al., “Transfer Learning from RF to B-mode Temporal Enhanced Ultrasound Features for Prostate Cancer Detection”, IPCAI 2017.

11

Classification Results: RF TeUS

12

0

20

40

60

80

100

120

140

TeUS MRI

Nu

mb

er

of

Co

rce

s

Incorrect Prediction

Correct Prediction

Train: RF TeUS data from 84 biopsy cores.

Test: RF TeUS Data from 121 biopsy cores.


Classification Results: RF vs. B-mode TeUS

13

0.79

0.81

0.83

0.85

0.87

0.89

0.91

0.93

0.95

All MR Level Moderate MR High MR

Are

a U

nd

er

the

Cu

rve

RF

B-mode




Classification Results: TeUS RF vs. B-mode

14

For moderate MR level cores (n=86)




Colormaps


15

Why does TeUS carry tissue typing information?

16

17

Cancer

Tissue response = f (Acoustic signal, Tissue microstructure, …)

Cell Nuclei (Scatterers)

Speckle

Benign

Cell Nuclei Speckle

TeUS: Physical Phenomena

Changes in tissue temperature

18

0

0.01

0.02

0.03

0.04

0.05

Trial 1 Trial 2 Trial 3 Trial 4

Temperature increase (°C) in 1 minute Round of eye 1

Round of eye 2

0

0.01

0.02

0.03

0.04

0.05

Trial 1 Trial 2 Trial 3

Temperature increase (°C) in 1 minute Turkey breast 1

Turkey breast 2

Turkey breast 3

TeUS: Changes in tissue temperature

Feature Visualization

Benign Gleason Pattern 3 Gleason Pattern 4

Layer 1: 100 hidden neurons



Trai

ne

d D

BN

Bac

k P

rop

agat

ion

Absolute Difference

Low-frequency components

Pulsation?

Visible Layer 50 spectral features

19

TeUS: Physical Phenomena

20

Cancer Benign

Tissue response = f (Acoustic signal, Tissue microstructure,…)

Cell Nuclei (Scatterers)

Speckle Cell Nuclei Speckle

S. Bayat et al., “Investigation of Physical Phenomena Underlying Temporal Enhanced Ultrasound as a New Diagnostic Imaging Technique: Theory and Simulations”, UFFC (submitted)

S. Bayat et al., “Tissue mimicking simulations for temporal enhanced ultrasound-based tissue typing“, SPIE 2017.

• Analytical representation of TeUS:

I 𝑥0, 𝑡 = 𝑃𝑆𝐹 𝑥0 ∗ 𝑠 𝑥0 + 𝑃𝑆𝐹 𝑥0 ∗ 𝜕𝑆

𝜕𝑥𝑥0 K/E sin (ωt)+n

* Lateral (mm)

Lateral (mm)

PSF(x)

I(x)

S(x)

21 S. Bayat et al., “Investigation of Physical Phenomena Underlying Temporal Enhanced Ultrasound as a New Diagnostic Imaging Technique: Theory and Simulations”, UFFC (submitted)




I 𝑥0, 𝑡 = 𝑃𝑆𝐹 𝑥0 ∗ 𝑠 𝑥0 + 𝑃𝑆𝐹 𝑥0 ∗ 𝜕𝑆


* Lateral (mm)

Lateral (mm)

PSF(x)

I(x)

S(x)

22


Conventional tissue characterization


I 𝑥0, 𝑡 = 𝑃𝑆𝐹 𝑥0 ∗ 𝑠 𝑥0 + 𝑃𝑆𝐹 𝑥0 ∗ 𝜕𝑆


* Lateral (mm)

Lateral (mm)

PSF(x)

I(x)

S(x)

23


Elastography


I 𝑥0, 𝑡 = 𝑃𝑆𝐹 𝑥0 ∗ 𝑠 𝑥0 + 𝑃𝑆𝐹 𝑥0 ∗ 𝜕𝑆


* Lateral (mm)

Lateral (mm)

PSF(x)

I(x)

S(x)

24


TeUS

Pathology Mimicking Simulations

• Data 14 H&E whole-mount slides (10 patients) scanned at 20x magnification using a ScanScope XT scanner (Aperio / Leica) obtained at U Colorado

26

Slide annotated by pathologist

Data

27

0

20

1 m

m

328 mm

GS4

GS5

Benign


Feature Extraction

Finite Element Simulations

Nuclei Location Extraction

Digital Pathology

Ultrasound Simulations (Field II)

Time


28

K. Iczkowski, et al., "Digital quantification of five high-grade PCa patterns, including the cribri-form pattern, and their association with adverse outcome", American Journal of Clinical Pathology (2011). (Colorado University )

S. Bayat, et al., “Tissue mimicking simulations for temporal enhanced US-based tissue typing”, SPIE 2017.

Cancer Normal

• Plot of P*S’ for fixed E

29

GS4

GS5

GS5

GS4

Benign

Benign

0

20

1 m

m

328 mm

GS4

GS5

Benign

Validation of Theoretical Derivation

Finite element modelling + Field II simulation was used

30

0

20

1 m

m

328 mm

GS4

GS5

Benign

In all cases, benign and cancer are separable.

Validation of Theoretical Derivation

Simulation Results

31

Simulation Results 1 Hz

Excitation

Backpropagation of feature from Neuron 1



Tissue Phantom Analysis

Samples

25

3D printed phantoms

4.5x3.5x0.5 cm phantom containing

aluminum oxide particles

4.5x3.5x0.5 cm phantom containing

viable liver cancer cells

Fabricated using Aspect Biosystems unique Lab-on-a-printerTM

Samples

34

Normal cells - Human aortic smooth muscle cells [T/G HA-VSMC] - 18 micrometer. - cell concentration of 1 million cells/mL of the phantom. - Approximately 8 million cells soon after printing. - Spherical in shape.

Cancer cells - Human hepatocellular carcinoma cell [HEPG2] - 18 micrometer. - cell concentration of 1 million cells/mL of the phantom. - Approximately 8 million cells soon after printing. - Spherical in shape.

Both cells clump in the phantom as they grow. The cancer cells form more clusters at a faster rate. The growth rate of the cancer cells is faster then the other normal cell.

Data Acquisition

35

Imaging Setting:

Depth = 4, 5 , and 6 cm Focal Point = 2 cm Frequency: 5, 6.6, and 10 MHz FPS: 25 and 51 Hz Dynamic Range: 75, 85 and 100 dB Power: 0, 2, and 4

Number of Samples: 6 - Benign, cancer and no-cell. - Two samples from each category. - For each sample, we have acquired RF data at

4 different planes.

Results

36

37

Tissue Mimicking Phantoms: Elasticity vs. Scatterer size

Experiment Design

• Different scatterer size or elasticity

− 60 μm phantom/1x gelatin vs. 32 μm phantom/1x gelatin

− 60 μm phantom/0.5x gelatin vs. 32 μm phantom/1x gelatin

− Simulating heart beats by vibrating a plastic tubing at 1 Hz

38

Vibration phantoms

US transducer

US phantom

Function generator controlled valve

Gas outlet Gas inlet

2 cm

The valve sends compressed gas at 1Hz frequency

The US transducer registers the

vibration

A circular inflatable SilasticTM tubing of 0.94 mm OD × 0.51 ID mm was extended through the phantom

The maximal tubing expansion was estimated by a thin wall approximation to be around 15.5 μm at a vibration level of 65 Psi.

39

Data Acquisition

a) b) c)

d)

40

e)

B-mode frames Scatterer size experiment b) 60 μm phantom/1x gelatin c) 32 μm phantom/1x gelatin Elasticity experiment d) 32 μm phantom/1x gelatin* e) 32 μm phantom/0.5x gelatin

a) The clamp holds the transducer in place and remain still during each TeUS acquisition.

Comparing the 1 Hz peaks across the phantoms

41

0

500

1000

1500

2000

2500

3000

0 psi 25 psi 40 psi 50 psi

FFT

Po

we

r A

mp

litu

de

s at

1 H

z (a

u.)

1x gelatin/60 µm scatterers 1x gelatin/32 µm scatterers

1x gelatin/32 µm scatterers* 0.5x gelatin/32 µm scatterers

The phantoms responded to the increase in the vibration amplitudes. At the baseline, the 1 Hz peaks were not distinguishing the phantoms. At higher vibration levels, the larger response resulted from smaller scatterer size and

less elastic phantoms.

Analytical Derivations: TeUS and Elastography complement each other!

42

Elasticity

50

0 m

icro

ns

?

Analytical Derivations: TeUS and Elastography complement each other!

43

?

Iczkowski, et al. 2011

Elasticity

Implications of Analytical Derivations: TeUS and Elastography complement each other!

44

PSF*S’

I 𝑥0, 𝑡 = 𝑃𝑆𝐹 𝑥0 ∗ 𝑠 𝑥0 + 𝑃𝑆𝐹 𝑥0 ∗ 𝜕𝑆


Elasticity

Cancer Type I

Cancer Type II

Summary and Future Works

23

In an in vivo study including 197 TRUS-guided biopsy:

− AUC of 0.8 in separation of clinically significant PCa from normal tissue type.

Tissue micro-structure, induced by mechanical excitation, can be used to distinguish benign and cancer.

TeUS captures a combination of spatial changes in scattering function and elasticity.

Future Works:

− Verify in larger population. − Further investigation of the underlying physical phenomena.

Thank You

Investigator and Collaborators: Dr. Purang Abolmaesumi, UBC Dr. Parvin Mousavi, Queen’s University Dr. Mehdi Moradi, IBM Dr. Amir Tahmasebi, Philips Research Dr. Pingkun Yan, Philips Research Dr. Bradford Wood, NIH Dr. Baris Turkey, NIH Dr. Peter Pinto, NIH Dr. Larry Goldenberg, VGH Dr. Martin Gleave, VGH Dr. Peter Black, VGH Dr. Storey Wilson, Dr. Kenneth A. Iczkowski Dr. Scott Lucia Students/Staff: Dr. Sharareh Bayat Dr. Farhad Imani Dr. Guy Nir Dr. Ajay Rajaram Si Jia Li Samira Sojoudi Nathan Van Woudenberg Siavash Khallaghi Hussam Ashab

46

@azizishekoofeh , UBC, Vancouver, December 2017

48

Documents

Temporal Enhance Ultrasound: A Novel Paradigm to Enable ... · S .Azizi et al , “Detectionand Grading of Prostate Cancer Using Temporal Enhanced Ultrasound: Combining Deep Neural