Singular value decomposition

In linear algebra, the singular value decomposition (SVD) is a factorization of a real or complex matrix into a rotation, followed by a rescaling followed by another rotation. It generalizes the eigendecomposition of a square normal matrix with an orthonormal eigenbasis to any ⁠${\displaystyle m\times n}$⁠ matrix. It is related to the polar decomposition.

Specifically, the singular value decomposition of an ${\displaystyle m\times n}$ complex matrix ⁠${\displaystyle \mathbf {M} }$⁠ is a factorization of the form ${\displaystyle \mathbf {M} =\mathbf {U\Sigma V^{*}} ,}$ where ⁠${\displaystyle \mathbf {U} }$⁠ is an ⁠${\displaystyle m\times m}$⁠ complex unitary matrix, ${\displaystyle \mathbf {\Sigma } }$ is an ${\displaystyle m\times n}$ rectangular diagonal matrix with non-negative real numbers on the diagonal, ⁠${\displaystyle \mathbf {V} }$⁠ is an ${\displaystyle n\times n}$ complex unitary matrix, and ${\displaystyle \mathbf {V} ^{*}}$ is the conjugate transpose of ⁠${\displaystyle \mathbf {V} }$⁠. Such decomposition always exists for any complex matrix. If ⁠${\displaystyle \mathbf {M} }$⁠ is real, then ⁠${\displaystyle \mathbf {U} }$⁠ and ⁠${\displaystyle \mathbf {V} }$⁠ can be guaranteed to be real orthogonal matrices; in such contexts, the SVD is often denoted ${\displaystyle \mathbf {U} \mathbf {\Sigma } \mathbf {V} ^{\mathrm {T} }.}$

The diagonal entries ${\displaystyle \sigma _{i}=\Sigma _{ii}}$ of ${\displaystyle \mathbf {\Sigma } }$ are uniquely determined by ⁠${\displaystyle \mathbf {M} }$⁠ and are known as the singular values of ⁠${\displaystyle \mathbf {M} }$⁠. The number of non-zero singular values is equal to the rank of ⁠${\displaystyle \mathbf {M} }$⁠. The columns of ⁠${\displaystyle \mathbf {U} }$⁠ and the columns of ⁠${\displaystyle \mathbf {V} }$⁠ are called left-singular vectors and right-singular vectors of ⁠${\displaystyle \mathbf {M} }$⁠, respectively. They form two sets of orthonormal bases ⁠${\displaystyle \mathbf {u} _{1},\ldots ,\mathbf {u} _{m}}$⁠ and ⁠${\displaystyle \mathbf {v} _{1},\ldots ,\mathbf {v} _{n},}$⁠ and if they are sorted so that the singular values ${\displaystyle \sigma _{i}}$ with value zero are all in the highest-numbered columns (or rows), the singular value decomposition can be written as

$${\displaystyle \mathbf {M} =\sum _{i=1}^{r}\sigma _{i}\mathbf {u} _{i}\mathbf {v} _{i}^{*},}$$

where ${\displaystyle r\leq \min\{m,n\}}$ is the rank of ⁠${\displaystyle \mathbf {M} .}$⁠

The SVD is not unique. However, it is always possible to choose the decomposition such that the singular values ${\displaystyle \Sigma _{ii}}$ are in descending order. In this case, ${\displaystyle \mathbf {\Sigma } }$ (but not ⁠${\displaystyle \mathbf {U} }$⁠ and ⁠${\displaystyle \mathbf {V} }$⁠) is uniquely determined by ⁠${\displaystyle \mathbf {M} .}$⁠

The term sometimes refers to the compact SVD, a similar decomposition ⁠${\displaystyle \mathbf {M} =\mathbf {U\Sigma V} ^{*}}$⁠ in which ⁠${\displaystyle \mathbf {\Sigma } }$⁠ is square diagonal of size ⁠${\displaystyle r\times r,}$⁠ where ⁠${\displaystyle r\leq \min\{m,n\}}$⁠ is the rank of ⁠${\displaystyle \mathbf {M} ,}$⁠ and has only the non-zero singular values. In this variant, ⁠${\displaystyle \mathbf {U} }$⁠ is an ⁠${\displaystyle m\times r}$⁠ semi-unitary matrix and ${\displaystyle \mathbf {V} }$ is an ⁠${\displaystyle n\times r}$⁠ semi-unitary matrix, such that ${\displaystyle \mathbf {U} ^{*}\mathbf {U} =\mathbf {V} ^{*}\mathbf {V} =\mathbf {I} _{r}.}$

Mathematical applications of the SVD include computing the pseudoinverse, matrix approximation, and determining the rank, range, and null space of a matrix. The SVD is also extremely useful in many areas of science, engineering, and statistics, such as signal processing, least squares fitting of data, and process control.