Ilmenau University of Technology Communications Research Laboratory 1 Tensor-Based Signal Processing with Applications in Biomedical Signal Analysis Technische

Ilmenau University of TechnologyCommunications Research Laboratory

1

Tensor-Based Signal Tensor-Based Signal Processing with Applications in Processing with Applications in

Biomedical Signal AnalysisBiomedical Signal Analysis

Technische Universität IlmenauFachgebiete Nachrichtentechnik & Biosignalverarbeitung


2

Outline

Motivation: Applications of multi-linear signal processing Introduction to multi-linear algebra Tensor decompositions

Multilinear extensions of the SVD• HOSVD• PARAFAC/CANDECOMP

Other decompositions PARAFAC via Joint Diagonalization 3-way PARAFAC for EEG data

Methodology and current status Open issues and questions

Discussion Status of the project proposals


3

Outline







4

Why tensors?Why tensors?

Well, why even matrices? Matrix equations are usually more compact insights, manipulations

Example: DFT

Not a different data model but a more compact representation More than two dimensions: tensors even more compact new insights

dn = ?

dn = ??


5

““Classical” Communications ResearchClassical” Communications Research

Description of the Mobile Radio Channel

resolve, characterize individual propagation paths of the mobile radio channel


6

Biomedical engineeringBiomedical engineering

For example: EEG data

diagnostics (neurology, ophthalmology) human-machine interface


7

Automotive engineeringAutomotive engineering

Wind tunnel analysis

find sources of disturbance to optimize aerodynamic behavior

Audio SourceLocalization


8

MotivationMotivation

More applications

Signal Processing (sensor arrays, blind multi-user detection, source separation, CDMA, SONAR and seismo-acoustic signal processing)

Computer vision (Face and facial expression recognition, handwritten text recognition)

Data mining (weblink analysis, personalized web search, cross-language information retrieval)

Neuroscience (Multisubject fMRI anlaysis, concurrent EEG/fMRI)

Chemical engineering (food industry, NIR spectroscopy)

Geophysics (moment tensor inversion)

Data compression (image coding, video coding)


9

Outline







10

The term “tensor”The term “tensor”

• Here: Intuitive definition: Here: Intuitive definition: ””A tensor of order p is a collection of elements referenced by p incides“A tensor of order p is a collection of elements referenced by p incides“ multi-way array multi-way array

• Mathematics: Mathematics: 1846: W. Voigt1846: W. Voigt

• Physics: Physics: 1915: M. Grossmann and A. Einstein1915: M. Grossmann and A. Einstein

very abstract definitionvery abstract definition

describe physical quantitiesdescribe physical quantities

ScalarsScalars VectorsVectors MatricesMatrices Order-3-tensorsOrder-3-tensors Order-4-tensorsOrder-4-tensors

??

……


11

NotationNotation Symbols

Matrix operations


12

Matrix unfoldingsMatrix unfoldings

n-mode matrix unfoldings vary the n-th along rows, the others along columns e.g., R = 3:

n-rank of . In general, 1-, 2-, and 3-rank can differ.

M1

M2

M3

“1-mode vectors”




13

nn-mode products-mode products

i.e., all the n-mode vectors multiplied from the left-hand-side by

n-mode product between a tensor and a matrix

11 22

outer product between two tensors: all pair-wise products between elements


14

The tensor rankThe tensor rank

Definition of the tensor rank

A tensor is rank one, iff

A tensor is rank r iff it can be decomposed into a sum of r and not less

than r rank one tensors

(Only) connection to the n-ranks:

The rank of a tensor can exceed its size (which is a good thing and a bad

thing)2 3 4 5 6 7 8

2 3 3 4 4 4 4 4

3 3 4 5 5 6 6 6

4 4 5 6 6 7 7 8

5 4 5 6 7 8 8 9

6 4 6 7 8 9 9 10

7 4 6 7 8 9 10 11

8 4 6 8 9 10 11 12

2 x

(maximum rank, cf. [Kolda08])


15

Outline







16

The Higher-Order SVD (HOSVD)The Higher-Order SVD (HOSVD)

Singular Value Decomposition Higher-Order SVD (Tucker3)

“Full HOSVD”“Full SVD”

[Tucker: 1966][de Lathauwer: 2000]


17

Computing the HOSVDComputing the HOSVD

Computing the HOSVD

The core tensor not necessarily any zero elements (only if n-rank-deficient)

all-orthogonality condition:

also three sets of singular values: n-mode singular values

4

35


18

The Higher-Order SVD (HOSVD)The Higher-Order SVD (HOSVD)

Singular Value Decomposition Higher-Order SVD (Tucker3)

“Full HOSVD”

“Economy size HOSVD”

Low-rank approximation (truncated HOSVD)

“Full SVD”

“Economy size SVD”

Low-rank approximation


19

Summary HOSVDSummary HOSVD

The HOSVD …

is an extension of the SVD to tensors.

generalizes the concept of row-space and column-space to r-spaces.

• the r-mode singular vectors are an orthonormal basis for the r-space of the tensor.

is very easy to compute (Matrix-SVD of the unfoldings).

the remaining core tensor is not diagonal, it may be full of non-zero elements (same size as original data)

• not a decomposition into rank-one components


20

Outline







21

Another way to look at the SVD

decomposition into a sum of rank one matrices also referred to as principal components (PCA)

Tensor case:

PARAFAC: MotivationPARAFAC: Motivation

+ +=

+ +=

Canonical Decomposition (CANDECOMP)Parallel Factor Analysis (PARAFAC)

[Carroll, Chang 1970][Harshman 1970]


22

PARAFAC expressionsPARAFAC expressions

Many equations to express the same model:

HOSVD:


23

HOSVD vs. PARAFACHOSVD vs. PARAFAC

Example:

HOSVD

PARAFAC “Core tensor” diagonal Not easy to find the factors Factors may be flat (underdetermined) Reveals the tensor rank Often used for analyzing data

Core tensor not necessarily diagonal, can be full. Direct, easy computation via matrix-SVDs “Factors” always tall or square Reveals the n-ranks Often used for “compressing” data2211

11 22


24

UniquenessUniqueness

When is the PARAFAC decomposition of X into A,B,C unique?

Let be the Kruskal-rank of A.

Then, given that … … the PARAFAC decomposition is unique up to scaling and permutation.

[Kruskal, 1966]

scaling and permutation can be removed if additional constraints are imposed or prior knowledge is used

scaling:

permutation:


25

Finding the parallel factorsFinding the parallel factors Since we only have the noisy tensor, we restate the goal:

“Plain vanilla” approach: ALS

Many years of research to improve convergence speed smart initializations smart updates: Enhanced line search … “PARAFAC”, “COMFAC”

works, but

• very slow convergence

• requires good initial solution

“Least Squares Fit”


26

Finding the parallel factorsFinding the parallel factors

Closed-form solutions? GRAM (generalized rank annihilation method): An exact closed-form

solution if either of M1, M2 or M3 = 2 (only two slices). Can be used as initialization for other methods.

DTLD (direct trilinear decomposition): A suboptimal approximation, mostly as initialization to PARAFAC. Based on Tucker3 (HOSVD) and GRAM. Very fast though.

Ours: Reduced the problem onto joint diagonalization of matrices (which by itself is a very well studied problem).


27

Outline







28


29

Outline







30

PARAFAC via Joint DiagonalizationPARAFAC via Joint Diagonalization

First, consider the case where

The transform matrices diagonalize the core tensor:

We can estimate the transform matrices via joint diagonalization

the fundamental link between the HOSVD and PARAFAC


31

PARAFAC via Joint DiagonalizationPARAFAC via Joint Diagonalization

One slide on the six diagonalization problems


32

One slide on the R-D extension


33

Outline







34

Processing ChainProcessing Chain

Time-Frequency-Analysis

Time

Fre

quen

cy

Chann

el

Time

Fre

quen

cy

Chann

el

Time

Cha

nnel

Wavelet-basiert Wigner-basiert

Biomedicalprocess

MeasurementTime-Frequency-

AnalysisComponent

Analysis


35

Component AnalysisComponent Analysis Given a three-way tensor (time, frequency, channel), we decompose it into a

predefined number of components for each component: time-, frequency-, and spatial characteristics

Zeit

Fre

quen

z

Raum≈ + +


36

Outline







37

Outline







38

Geplante Folgeprojekte (1)Geplante Folgeprojekte (1)

BMBF - Innovationswettbewerb Medizintechnik Modul I „Innovationswettbewerb - BASIS“ Schlüsselexperiment zum Nachweis der Machbarkeit:

„Tensor-basierte Analyse von polygraphischen Biosignalen zur Anfallsvorhersage bei Epilepsie“

Projektpartner• TU Ilmenau: FG BSV & FG NT • GJB Datentechnik GmbH, Langewiesen• Zentralklinik Bad Berka GmbH, Klinik für Neurologie

Status• Projektskizze eingereicht• Hauptantrag vorzubereiten im Sommer/Herbst 2008

Modul II „Innovationswettbewerb – Transfer“ F&E-Vorhaben „Neue Methoden der Tensor-basierten Analyse von polygraphischen Biosignalen“ Projektpartner

• TU Ilmenau: FG BSV & FG NT • GJB Datentechnik GmbH, Langewiesen• Zentralklinik Bad Berka GmbH, Klinik für Neurologie• Psychotherapeutische und Neurologische Praxen

Status• Projektskizze vorzubereiten im Herbst 2008


39


BMBF - Innovationswettbewerb Medizintechnik „Frühdiagnostik und Intervention von Essanfällen mittels Polygraphie bei Patienten mit

Bulimia Nervosa“ (BuPoly) Projektpartner

• TU Ilmenau: FG BSV & FG NT • NeuroConn GmbH, Ilmenau• Praxis Dr. Braun, Gotha


„Zeitvariable Niederfeldmagnetstimulation in der Therapie depressiver Erkrankungen und deren Wirkung auf die Herzratenvariabilität“ (DeNFMagS) Projektpartner

• TU Ilmenau: FG BSV & FG NT • Neurologische Praxis Henkel/Müller, Ilmenau



40


BMBF - Ernährungsforschung „Polygraphiebasierte methodische und experimentelle Untersuchung der

Reizreaktion auf lebensmittelbezogene visuelle Stimulationen bei Personen mit und ohne psychogene Essstörungen“ Projektpartner

• TU Ilmenau: FG BSV & FG NT • Praxis Dr. Braun, Gotha• Neurologische Praxis Henkel/Müller, Ilmenau

Status• Projektskizze eingereicht• Hauptantrag vorzubereiten im Sommer 2008


41

Geplante Folgeprojekte (4)Geplante Folgeprojekte (4)LUBOM – Thüringen „Zeitvariable Niederfeldmagnetstimulation in der Therapie depressiver

Erkrankungen und deren Wirkung auf die Herzratenvariabilität“ Projektpartner

• in der ersten Phase TU Ilmenau: FG BSV & FG NT • in der nächsten Phase zusätzlich neurologische und

psychotherapeutische Praxen Status

• Projektbeginn bei Bewilligung Anfang 2009 „Die Wirkungsweise von Eye Movement Desensitization and

Reprocessing analysiert anhand polygraphischer Untersuchungen multimodaler Biosignale“ Projektpartner

• in der ersten Phase TU Ilmenau: FG BSV & FG NT • in der nächsten Phase zusätzlich neurologische und

psychotherapeutische Praxen Status

• Projektbeginn bei Bewilligung Anfang 2009


42


LUBOM – Thüringen „EKG-Analyse zur Bestimmung der anaeroben Schwelle anhand der

Absenkung des ST-Komplexes“ Projektpartner

• TU Ilmenau: FG BSV & FG NT Status

• Projektbeginn bei Bewilligung Herbst 2008TAB – Thüringen „Erkennung von psychischen Verarbeitungsprozessen angstgestörter

Patienten zur Unterstützung der lnterventionstherapie mittels mobiler onIinefähiger Biofeedbackgeräte“ Projektpartner

• TU Ilmenau: FG BSV & FG NT • GJB Datentechnik GmbH, Langewiesen• Psychotherapie Dr. Wilms, Erfurt

Status• Projektbeginn bei Bewilligung Herbst 2008


43


European Research Council: Advanced Investigators Grants im FP 7

„Mikrosensoren zur Erfassung wichtiger Lebensfunktionen“

Projektpartner

• TU Ilmenau: FG BSV & FG NT & IMN & FG NIKR

• IDMT Fraunhofer, Ilmenau

Status

• Projektskizze in Vorbereitung

• einzureichen im Winter 2008


44


DFG Forschungsprojekt zur dynamischen tensorbasierten

Analyse von nichtlinearen zeitvariablen Prozessen Projektpartner

• TU Ilmenau: FG BSV & FG NT • Zentralklinik Bad Berka GmbH, Klinik für Neurologie• GJB Datentechnik GmbH, Langewiesen• Psychotherapeutische und Neurologische Praxen

Status• Projektskizze vorzubereiten im Herbst 2008

Documents

Ilmenau University of Technology Communications Research Laboratory 1 Tensor-Based Signal Processing with Applications in Biomedical Signal Analysis Technische