Algorithm Development with Higher Order SVD
Algorithms Meeting September 16, 2004 Algorithm Development with Higher Order SVD Adam Goodenough Tensors N-Mode Generalization of a Matrix Notation: T
In is the length of the tensor in mode (i.e. dimension) n A tensor has order N Two basic operations: unfolding (T(n)) and the mode-n product of a tensor and a matrix (Tx(n)M) I1xI2xxIN Example 5 1 2 4 3 Mode-3 Tensor 5 1 2 4 3 I2 I1 I3 Unfolding Unfolding flattens an N-mode tensor into a 2-mode tensor
The operation, T(n), unfolds the matrix along mode n Each column of the resulting tensor is composed of twhere in varies and the remaining indices are held constant (for a column) i1i2iniN Example - Unfolding Mode-1 Unfolding Example - Unfolding Mode-2 Unfolding Example - Unfolding Mode-3 Unfolding Tensor Matrix Product
The operation, Tx(n)M, denotes the product of an Nth order Tensor with a Matrix (2nd order Tensor) Example - Product 5 1 2 4 3 1 5 2 3 4 Matrix PCA Two standard ways of performing Principal Component Analysis: Find the eigenvalues/eigenvectors of the covariance matrix Take the SVD of the raw data The basis vectors are weighted by the eigen/singular values The SVD approach can be easily extended to Tensors Matrix Decomposition I1 I2 Data Matrix = J1 U J2 S V T Matrix Decomposition I1 I2 Data Matrix A Tensor! Matrix Decomposition I1 J1 U I2 J2 V Data Matrix = T S Matrix Decomposition U1 U2 I1 I2 Data Matrix = J1 U J2 S V T I1 J1 U Matrix SVD Tensor SVD HOSVD Given a tensor, T, of order N:
Each Un is obtained from the SVD of the unfolding of T along mode n Z is a full tensor (i.e. not equivalent to the diagonal S matrix) HOSVD The tensor, Z, is found by Z has dimensionality equivalent to T
i.e. Given Algorithm Applications
Three general areas: Prediction Detection Compression All algorithms work through building a training data sets by varying parameters (mode indices) Applications - Prediction
Tensor Textures (Vasilescu and Terzopoulos 2004) Applications - Prediction
Data set: DTx I x V Rasterized Texture Images: T = 240x320x3 = 23040 Illumination Orientations: I = 21 View Orientations: V = 37 Total of 777 images taken (21x37) Tangent Normal PCA of this data set would perform SVD on the matrix of all observations This matrix is equivalent to the mode-1 unfolding of the tensor, i.e. Tangent This means that the SVD of the data set is related to Utexel
It turns out that R is constructed by the Z, Uillum, Uview terms (using the Kronecker product) We end up having explicit control over the eigenvalues (compression) Applications - Prediction
i, v are computed based on nearest neighbors Applications - Prediction
MODTRAN (MakeADB) Applications - Detection
Tensor Faces (Vasilescu and Terzopoulos 2002) Applications - Detection
Vast improvement over PCA based techniques Applications - Detection
Attempts on plume data Applications - Compression
HOSVD allows for explicit control over dimensionality reduction Applications - Compression
Compression can be done so that reconstruction results are better perceptually (though usually worse RMSE) PCA HOSVD Applications - Compression
Can having explicit control over dimensionality reduction allow us to perform invariant based detection algorithms better? Goal is to use photon mapping to generate spectral images of underwater targets given varying environmental conditions and to test traditional PCA based invariant algorithms versus HOSVD based invariant algorithms