3. PRINCIPLE OF VECTOR DECOMPOSITION
3.2 Singular Value Decomposition (SVD)
The Singular Value Decomposition (SVD) is the generalisation of the diagonalization of a square matrix, for a non-square matrix. It is a matrix decomposition technique (into singular values) that allows a simpler inversion of the matrixM than calculating M−1 directly. The goal is to do a vector decomposition of the vector X~ (finding the factors).
3.2.1 Principle
The diagonalization of a square matrix is the idea that a diagonalizable square matrix M can be expressed as the product of three matrices:
M =U ·S·U−1
where U is the matrix of eigenvectors; andS is a diagonal matrix where the values are the eigenvalues associated to the eigenvectors in U, in the same order.
A non-square matrix can be decomposed into singular values. It is called Singular Value Decomposition (SVD), and is calculated by a computational technique (algo- rithm) of high precision (cf. § 4.6). The decomposition is exact only to the precision of the algorithm, but we will assume here that it is exact.
Mm×n≡Um×m·Σm×n·Wn×nT (3.7)
where:
M is the original matrix, of sizem×n;
U is an orthogonal unitary matrix, of size m×m;
W is an orthogonal unitary matrix, of sizen×n;
Σ is a non-square diagonal matrix, of sizem×n.
3.2 Singular Value Decomposition (SVD)
(~v1) · · · (~vn)
| {z }
Mm×n
=
(~u1) · · · (~um)
| {z }
Um×m
·
λ1 0 · · · 0 0 λ2 · · · 0 ... ... . .. ...
0 0 · · · λn
0 0 · · · 0 ... ... ... 0 0 · · · 0
| {z }
Σm×n
·
—(w~1)—
...
—(w~n)—
| {z }
Wn×nT
Figure 3.1: Structure of the matrices in the decomposition. M is the original matrix, made of the vectors (~vi);U and W are the matrices of the left and right singular vectors respectively. Σ is a diagonal matrix (withn < m) holding the Singular Values: all values are null, apart from the diagonalλi= (Σ)i,i.
Σ is such that only the diagonal elements are non-null, and there arenin total. These values (λi) are unique and are called singular values.
U and W are orthogonal matrices: they generate an orthogonal basis of the cor- responding N-vector space (N =n orm). They satisfy U ·UT =IN. The vectors of U are called left singular vectors, and the ones ofW areright singular vectors. These vectors are not unique for the SVD of one matrixM.
The transformation associated to the matrix M is decomposed by the SVD into three steps: a rotation (matrix W), a stretching (matrix Σ) and another rotation (matrixU) (12).
3.2.2 Advantages
The Singular Value Decomposition calculates the singular vectorsU and W as well as the values λi of Σ. The vectors of U, of dimension m, are orthonormal and form a basis of them-vector space, in which the losses are decomposed. Moreover, only then first vectors at most are used: there are onlynvaluesλi. All vectors (~uj) forj > nare associated with a zero in Σ during the matrix multiplication.
The singular values in Σ are sorted in decreasing order. The lower λi, at higher i, can be ignored for faster recomposition at the price of a small error. In this work n ≪ m, so no λi are omitted. The first vectors (ui) of U, associated to the λi of
3. PRINCIPLE OF VECTOR DECOMPOSITION
higher value, will have a strong contribution in the recomposition; the ones with higher i(i < n) will have a small contribution (λi of smaller value); the ones with i > n will have no contribution.
3.2.3 Important observations
In this section, we will give the exact expression of the coefficients of M as calculated by the SVD. The coefficients of the matrices are written in their standard form: (wij) for the matrix W, wherei is the line number andj the column number, both varying between 0 and n. Note that the vector (w~1) will then have the coefficients (w1j).
The product between the matrices Cm×n and Dn×p, the result is a matrix of coef- ficients (eij) expressed as:
∀i, j :eij ≡ Xn k=1
cikdkj (3.8)
Let’s start by the multiplication Um×m ·Σm×n. The elements will be called (sij) and are:
sij = Xm k=1
uikλkj
The elements λkj are null for allk > n and allj6=k, so the intermediate result is:
sij = Xn k=1
uikλkj =uijλj
Now, let’s consider the multiplication of (UΣ)m×n by Wn×n. Eq. 3.8 gives:
mij = Xn k=1
sikwkj = Xn k=1
uikλkwkj
mij = Xn k=1
λkuikwkj (3.9)
The most important point here is that only the first n vectors (~uk) are used. The coefficients ofM are decomposed into the equivalent of a matrix product of the firstn vectors ofU and the vectors ofW, but each term of the sum is weighted by a factorλk. As a consequence, the vectors (uk) fork > ndo not take part of the decomposition;
so they are left by the algorithm as canonical vectors: the kth coordinate is 1 and all the others are zero.
3.2 Singular Value Decomposition (SVD)
3.2.4 Pseudoinverse
Once the matrix Σm×n has been created, thepseudoinverse can be created. It is noted Σ+ and has a size ofn × m. All values are null, apart from the diagonal ones which are (Σ)i,i= λ1
i forλi 6= 0. This matrix is called pseudoinverse because it only satisfies Σ+·Σ =In; the product Σ·Σ+ is a matrix of sizem×m, with only thenfirst diagonal elements are 1; all the others are zero.
The pseudoinverse of M is then:
M+≡W ·Σ+·UT (3.10)
and satisfies:
M+·M = (W ·Σ+·UT)·(U
| {z }
Im
·Σ·WT)
=W ·Σ+·Σ·WT
=W ·WT M+·M =In
Note that it is different for M ·M+: for the same reason as Σ·Σ+, it is a matrix of sizem×m, with only the nfirst diagonal elements are 1; all the others are zero.
3.2.5 Recomposition
Once the pseudoinverse has been calculated, the factors can be calculated (vector de- composition). The result is not exact, because the vectorX~ is generally not part of the subspace generated by the vectors ofM.
M·F~ ≃X~ M+·M·F~ ≃M+·X~
F~ ≃W ·Σ+·UT
| {z }
M+
·X~ F~ ≃M+·X~ The recomposition X~′ is then calculated by:
X~′ =M ·F~ X~′ =M ·W ·Σ+·UT
| {z }
M+
·X~
3. PRINCIPLE OF VECTOR DECOMPOSITION
3.2.6 Limitations
Accuracy of the recomposition
Given the matrixM, the SVD algorithm will always calculate a matrix decomposition M =U·Σ·WT. Once the pseudoinverseM+calculated, the set of factorsF~ can always be calculated, as well as the recomposition X~′. This recomposition is the combination of the vectors (~vi) which is the closest to X, and belongs to the subspace generated by~ the vectors of the loss scenarios. However, it doesn’t mean that the recomposition X~′ will actually be close to X; it can even be null. An example of such a situation is:~
M =
1 1 0 1 0 0
X~ =
0 0 1
No set of factors can recomposeX: the projection of~ X~ on the subspace generated by the vectors ofM is the point of coordinates (0,0,0).
Without a proper evaluation of the accuracy of the vector decomposition, the factors are meaningless.
Negative factors
In the standardRm vector space, each factorfican take any value inR. However, there are physical limitations. The factors (fi) represent the contribution of the different
“standard” loss profiles (vectors (~vi)): negative values are not physical, nothing would
“remove” a loss profile. Moreover, the losses are only positive: again, negative losses make no sense. The vectors (~vi) are in (R+)m: the space generated by the positive half of the field of real numbers (noted R+) considered m times.
A negative factor in the vector decomposition indicates that the corresponding vector can not contribute to the reconstruction of the vector X. This means that~ the corresponding vector can not take part of the decomposition. It often points to a problem in the decomposition, for example that the chosen loss scenarios are not exactly adapted to the problem, and that the measured vector has other components than the selected vectors (~vi).
One way to deal with this effect would be to remove this vector from the matrix M: the matrix would then have a sizem−1. The SVD could be calculated again; the