Not to be confused with matrix factorization of a polynomial.

In the mathematical discipline of linear algebra, a matrix decomposition or matrix factorization is a factorization of a matrix into a product of matrices. There are many different matrix decompositions; each finds use among a particular class of problems.

Example

In numerical analysis, different decompositions are used to implement efficient matrix algorithms.

For instance, when solving a system of linear equations , the matrix A can be decomposed via the LU decomposition. The LU decomposition factorizes a matrix into a lower triangular matrix L and an upper triangular matrix U. The systems and require fewer additions and multiplications to solve, compared with the original system , though one might require significantly more digits in inexact arithmetic such as floating point.

Similarly, the QR decomposition expresses A as QR with Q an orthogonal matrix and R an upper triangular matrix. The system Q(Rx) = b is solved by Rx = Q^Tb = c, and the system Rx = c is solved by 'back substitution'. The number of additions and multiplications required is about twice that of using the LU solver, but no more digits are required in inexact arithmetic because the QR decomposition is numerically stable.

LU decomposition

• Applicable to: square matrix A
• Decomposition: , where L is lower triangular and U is upper triangular
• Related: the LDU decomposition is , where L is lower triangular with ones on the diagonal, U is upper triangular with ones on the diagonal, and D is a diagonal matrix.
• Related: the LUP decomposition is , where L is lower triangular, U is upper triangular, and P is a permutation matrix.
• Existence: An LUP decomposition exists for any square matrix A. When P is an identity matrix, the LUP decomposition reduces to the LU decomposition. If the LU decomposition exists, then the LDU decomposition exists.^[1]
• Comments: The LUP and LU decompositions are useful in solving an n-by-n system of linear equations . These decompositions summarize the process of Gaussian elimination in matrix form. Matrix P represents any row interchanges carried out in the process of Gaussian elimination. If Gaussian elimination produces the row echelon form without requiring any row interchanges, then P = I, so an LU decomposition exists.

LU reduction

Block LU decomposition

Rank factorization

Main article: Rank factorization

Cholesky decomposition

QR decomposition

RRQR factorization

Main article: RRQR factorization

Interpolative decomposition

Main article: Interpolative decomposition

Decompositions based on eigenvalues and related concepts

Eigendecomposition

Main article: Eigendecomposition (matrix)

• Also called spectral decomposition.
• Applicable to: square matrix A with linearly independent eigenvectors (not necessarily distinct eigenvalues).
• Decomposition: , where D is a diagonal matrix formed from the eigenvalues of A, and the columns of V are the corresponding eigenvectors of A.
• Existence: An n-by-n matrix A always has n (complex) eigenvalues, which can be ordered (in more than one way) to form an n-by-n diagonal matrix D and a corresponding matrix of nonzero columns V that satisfies the eigenvalue equation . is invertible if and only if the n eigenvectors are linearly independent (i.e., each eigenvalue has geometric multiplicity equal to its algebraic multiplicity). A sufficient (but not necessary) condition for this to happen is that all the eigenvalues are different (in this case geometric and algebraic multiplicity are equal to 1)
• Comment: One can always normalize the eigenvectors to have length one (see the definition of the eigenvalue equation)
• Comment: Every normal matrix A (i.e., matrix for which , where is a conjugate transpose) can be eigendecomposed. For a normal matrix A (and only for a normal matrix), the eigenvectors can also be made orthonormal () and the eigendecomposition reads as . In particular all unitary, Hermitian, or skew-Hermitian (in the real-valued case, all orthogonal, symmetric, or skew-symmetric, respectively) matrices are normal and therefore possess this property.
• Comment: For any real symmetric matrix A, the eigendecomposition always exists and can be written as , where both D and V are real-valued.
• Comment: The eigendecomposition is useful for understanding the solution of a system of linear ordinary differential equations or linear difference equations. For example, the difference equation starting from the initial condition is solved by , which is equivalent to , where V and D are the matrices formed from the eigenvectors and eigenvalues of A. Since D is diagonal, raising it to power , just involves raising each element on the diagonal to the power t. This is much easier to do and understand than raising A to power t, since A is usually not diagonal.

Jordan decomposition

The Jordan normal form and the Jordan–Chevalley decomposition

• Applicable to: square matrix A
• Comment: the Jordan normal form generalizes the eigendecomposition to cases where there are repeated eigenvalues and cannot be diagonalized, the Jordan–Chevalley decomposition does this without choosing a basis.

Schur decomposition

Real Schur decomposition

QZ decomposition

Main article: QZ decomposition

• Also called: generalized Schur decomposition
• Applicable to: square matrices A and B
• Comment: there are two versions of this decomposition: complex and real.
• Decomposition (complex version): and where Q and Z are unitary matrices, the * superscript represents conjugate transpose, and S and T are upper triangular matrices.
• Comment: in the complex QZ decomposition, the ratios of the diagonal elements of S to the corresponding diagonal elements of T, , are the generalized eigenvalues that solve the generalized eigenvalue problem (where is an unknown scalar and v is an unknown nonzero vector).
• Decomposition (real version): and where A, B, Q, Z, S, and T are matrices containing real numbers only. In this case Q and Z are orthogonal matrices, the T superscript represents transposition, and S and T are block upper triangular matrices. The blocks on the diagonal of S and T are of size 1×1 or 2×2.

Takagi's factorization

Singular value decomposition

Scale-invariant decompositions

Refers to variants of existing matrix decompositions, such as the SVD, that are invariant with respect to diagonal scaling.

Analogous scale-invariant decompositions can be derived from other matrix decompositions, e.g., to obtain scale-invariant eigenvalues.^[3][4]

Other decompositions

Polar decomposition

Algebraic polar decomposition

Mostow's decomposition

Main article: Mostow decomposition

Sinkhorn normal form

Main article: Sinkhorn's theorem

• Applicable to: square real matrix A with strictly positive elements.
• Decomposition: , where S is doubly stochastic and D₁ and D₂ are real diagonal matrices with strictly positive elements.

Sectoral decomposition

Williamson's normal form

Generalizations

There exist analogues of the SVD, QR, LU and Cholesky factorizations for quasimatrices and cmatrices or continuous matrices.^[13] A ‘quasimatrix’ is, like a matrix, a rectangular scheme whose elements are indexed, but one discrete index is replaced by a continuous index. Likewise, a ‘cmatrix’, is continuous in both indices. As an example of a cmatrix, one can think of the kernel of an integral operator.

These factorizations are based on early work by Fredholm (1903), Hilbert (1904) and Schmidt (1907). For an account, and a translation to English of the seminal papers, see Stewart (2011).

See also

• Matrix splitting
• Non-negative matrix factorization
• Principal component analysis

Notes

References

• Choudhury, Dipa; Horn, Roger A. (April 1987). "A Complex Orthogonal-Symmetric Analog of the Polar Decomposition". SIAM Journal on Algebraic and Discrete Methods. 8 (2): 219–225. doi:10.1137/0608019.
• Fredholm, I. (1903), "Sur une classe d'´equations fonctionnelles", Acta Mathematica (in French), 27: 365–390, doi:10.1007/bf02421317
• Hilbert, D. (1904), "Grundzüge einer allgemeinen Theorie der linearen Integralgleichungen", Nachr. Königl. Ges. Gött (in German), 1904: 49–91
• Horn, Roger A.; Merino, Dennis I. (January 1995). "Contragredient equivalence: A canonical form and some applications". Linear Algebra and Its Applications. 214: 43–92. doi:10.1016/0024-3795(93)00056-6.
• Meyer, C. D. (2000), Matrix Analysis and Applied Linear Algebra, SIAM, ISBN 978-0-89871-454-8
• Schmidt, E. (1907), "Zur Theorie der linearen und nichtlinearen Integralgleichungen. I Teil. Entwicklung willkürlichen Funktionen nach System vorgeschriebener", Mathematische Annalen (in German), 63 (4): 433–476, doi:10.1007/bf01449770
• Simon, C.; Blume, L. (1994). Mathematics for Economists. Norton. ISBN 978-0-393-95733-4.
• Stewart, G. W. (2011), Fredholm, Hilbert, Schmidt: three fundamental papers on integral equations (PDF), retrieved 2015-01-06
• Townsend, A.; Trefethen, L. N. (2015), "Continuous analogues of matrix factorizations", Proc. R. Soc. A, 471 (2173): 20140585, Bibcode:2014RSPSA.47140585T, doi:10.1098/rspa.2014.0585, PMC 4277194, PMID 25568618

External links

• Online Matrix Calculator
• Wolfram Alpha Matrix Decomposition Computation » LU and QR Decomposition
• Springer Encyclopaedia of Mathematics » Matrix factorization
• GraphLab GraphLab collaborative filtering library, large scale parallel implementation of matrix decomposition methods (in C++) for multicore.