Misplaced Pages

Latest revision as of 17:11, 26 December 2024

In mathematics, and in particular linear algebra, the Moore–Penrose inverse ⁠${\displaystyle A^{+}}$⁠ of a matrix ⁠${\displaystyle A}$⁠, often called the pseudoinverse, is the most widely known generalization of the inverse matrix. It was independently described by E. H. Moore in 1920, Arne Bjerhammar in 1951, and Roger Penrose in 1955. Earlier, Erik Ivar Fredholm had introduced the concept of a pseudoinverse of integral operators in 1903. The terms pseudoinverse and generalized inverse are sometimes used as synonyms for the Moore–Penrose inverse of a matrix, but sometimes applied to other elements of algebraic structures which share some but not all properties expected for an inverse element.

A common use of the pseudoinverse is to compute a "best fit" (least squares) approximate solution to a system of linear equations that lacks an exact solution (see below under § Applications). Another use is to find the minimum (Euclidean) norm solution to a system of linear equations with multiple solutions. The pseudoinverse facilitates the statement and proof of results in linear algebra.

The pseudoinverse is defined for all rectangular matrices whose entries are real or complex numbers. Given a rectangular matrix with real or complex entries, its pseudoinverse is unique. It can be computed using the singular value decomposition. In the special case where ⁠${\displaystyle A}$⁠ is a normal matrix (for example, a Hermitian matrix), the pseudoinverse ⁠${\displaystyle A^{+}}$⁠ annihilates the kernel of ⁠${\displaystyle A}$⁠ and acts as a traditional inverse of ⁠${\displaystyle A}$⁠ on the subspace orthogonal to the kernel.

Notation

In the following discussion, the following conventions are adopted.

• ⁠${\displaystyle \mathbb {K} }$⁠ will denote one of the fields of real or complex numbers, denoted ⁠${\displaystyle \mathbb {R} }$⁠, ⁠${\displaystyle \mathbb {C} }$⁠, respectively. The vector space of ⁠${\displaystyle m\times n}$⁠ matrices over ⁠${\displaystyle \mathbb {K} }$⁠ is denoted by ⁠${\displaystyle \mathbb {K} ^{m\times n}}$⁠.
• For ⁠${\displaystyle A\in \mathbb {K} ^{m\times n}}$⁠, the transpose is denoted ⁠${\displaystyle A^{\operatorname {T} }}$⁠ and the Hermitian transpose (also called conjugate transpose) is denoted ⁠${\displaystyle A^{*}}$⁠. If ${\displaystyle \mathbb {K} =\mathbb {R} }$, then ${\displaystyle A^{*}=A^{\operatorname {T} }}$.
• For ⁠${\displaystyle A\in \mathbb {K} ^{m\times n}}$⁠, ⁠${\displaystyle \operatorname {ran} (A)}$⁠ (standing for "range") denotes the column space (image) of ⁠${\displaystyle A}$⁠ (the space spanned by the column vectors of ⁠${\displaystyle A}$⁠) and ⁠${\displaystyle \ker(A)}$⁠ denotes the kernel (null space) of ⁠${\displaystyle A}$⁠.
• For any positive integer ⁠${\displaystyle n}$⁠, the ⁠${\displaystyle n\times n}$⁠ identity matrix is denoted ⁠${\displaystyle I_{n}\in \mathbb {K} ^{n\times n}}$⁠.

Definition

For ${\displaystyle A\in \mathbb {K} ^{m\times n}}$, a pseudoinverse of A is defined as a matrix ⁠${\displaystyle A^{+}\in \mathbb {K} ^{n\times m}}$⁠ satisfying all of the following four criteria, known as the Moore–Penrose conditions:

1. ⁠${\displaystyle AA^{+}}$⁠ need not be the general identity matrix, but it maps all column vectors of A to themselves: $${\displaystyle AA^{+}A=\;A.}$$
2. ⁠${\displaystyle A^{+}}$⁠ acts like a weak inverse: $${\displaystyle A^{+}AA^{+}=\;A^{+}.}$$
3. ⁠${\displaystyle AA^{+}}$⁠ is Hermitian: $${\displaystyle \left(AA^{+}\right)^{*}=\;AA^{+}.}$$
4. ⁠${\displaystyle A^{+}A}$⁠ is also Hermitian: $${\displaystyle \left(A^{+}A\right)^{*}=\;A^{+}A.}$$

Note that ${\displaystyle A^{+}A}$ and ${\displaystyle AA^{+}}$ are idempotent operators, as follows from ${\displaystyle (AA^{+})^{2}=AA^{+}}$ and ${\displaystyle (A^{+}A)^{2}=A^{+}A}$. More specifically, ${\displaystyle A^{+}A}$ projects onto the image of ${\displaystyle A^{T}}$ (equivalently, the span of the rows of ${\displaystyle A}$), and ${\displaystyle AA^{+}}$ projects onto the image of ${\displaystyle A}$ (equivalently, the span of the columns of ${\displaystyle A}$). In fact, the above four conditions are fully equivalent to ${\displaystyle A^{+}A}$ and ${\displaystyle AA^{+}}$ being such orthogonal projections: ${\displaystyle AA^{+}}$ projecting onto the image of ${\displaystyle A}$ implies ${\displaystyle (AA^{+})A=A}$, and ${\displaystyle A^{+}A}$ projecting onto the image of ${\displaystyle A^{T}}$ implies ${\displaystyle (A^{+}A)A^{T}=A^{T}}$.

The pseudoinverse ${\displaystyle A^{+}}$ exists for any matrix ${\displaystyle A\in \mathbb {K} ^{m\times n}}$. If furthermore ${\displaystyle A}$ is full rank, that is, its rank is ⁠${\displaystyle \min\{m,n\}}$⁠, then ⁠${\displaystyle A^{+}}$⁠ can be given a particularly simple algebraic expression. In particular:

• When ⁠${\displaystyle A}$⁠ has linearly independent columns (equivalently, ⁠${\displaystyle A}$⁠ is injective, and thus ⁠${\displaystyle A^{*}A}$⁠ is invertible), ⁠${\displaystyle A^{+}}$⁠ can be computed as$${\displaystyle A^{+}=\left(A^{*}A\right)^{-1}A^{*}.}$$This particular pseudoinverse is a left inverse, that is, ${\displaystyle A^{+}A=I}$.
• If, on the other hand, ${\displaystyle A}$ has linearly independent rows (equivalently, ${\displaystyle A}$ is surjective, and thus ⁠${\displaystyle AA^{*}}$⁠ is invertible), ⁠${\displaystyle A^{+}}$⁠ can be computed as$${\displaystyle A^{+}=A^{*}\left(AA^{*}\right)^{-1}.}$$This is a right inverse, as ${\displaystyle AA^{+}=I}$.

In the more general case, the pseudoinverse can be expressed leveraging the singular value decomposition. Any matrix can be decomposed as ${\displaystyle A=UDV^{*}}$ for some isometries ${\displaystyle U,V}$ and diagonal nonnegative real matrix ${\displaystyle D}$. The pseudoinverse can then be written as ${\displaystyle A^{+}=VD^{+}U^{*}}$, where ${\displaystyle D^{+}}$ is the pseudoinverse of ${\displaystyle D}$ and can be obtained by transposing the matrix and replacing the nonzero values with their multiplicative inverses. That this matrix satisfies the above requirement is directly verified observing that ${\displaystyle AA^{+}=UU^{*}}$ and ${\displaystyle A^{+}A=VV^{*}}$, which are the projections onto image and support of ${\displaystyle A}$, respectively.

Properties

Existence and uniqueness

As discussed above, for any matrix ⁠${\displaystyle A}$⁠ there is one and only one pseudoinverse ⁠${\displaystyle A^{+}}$⁠.

A matrix satisfying only the first of the conditions given above, namely ${\textstyle AA^{+}A=A}$, is known as a generalized inverse. If the matrix also satisfies the second condition, namely ${\textstyle A^{+}AA^{+}=A^{+}}$, it is called a generalized reflexive inverse. Generalized inverses always exist but are not in general unique. Uniqueness is a consequence of the last two conditions.

Basic properties

Proofs for the properties below can be found at b:Topics in Abstract Algebra/Linear algebra.

• If ⁠${\displaystyle A}$⁠ has real entries, then so does ⁠${\displaystyle A^{+}}$⁠.
• If ⁠${\displaystyle A}$⁠ is invertible, its pseudoinverse is its inverse. That is, ${\displaystyle A^{+}=A^{-1}}$.
• The pseudoinverse of the pseudoinverse is the original matrix: ${\displaystyle \left(A^{+}\right)^{+}=A}$.
• Pseudoinversion commutes with transposition, complex conjugation, and taking the conjugate transpose: $${\displaystyle \left(A^{\operatorname {T} }\right)^{+}=\left(A^{+}\right)^{\operatorname {T} },\quad \left({\overline {A}}\right)^{+}={\overline {A^{+}}},\quad \left(A^{*}\right)^{+}=\left(A^{+}\right)^{*}.}$$
• The pseudoinverse of a scalar multiple of ⁠${\displaystyle A}$⁠ is the reciprocal multiple of ⁠${\displaystyle A^{+}}$⁠:$${\displaystyle \left(\alpha A\right)^{+}=\alpha ^{-1}A^{+}}$$ for ⁠${\displaystyle \alpha \neq 0}$⁠; otherwise, ${\displaystyle \left(0A\right)^{+}=0A^{+}=0A^{\operatorname {T} }}$, or ${\displaystyle 0^{+}=0^{\operatorname {T} }}$.
• The kernel and image of the pseudoinverse coincide with those of the conjugate transpose: ${\displaystyle \ker \left(A^{+}\right)=\ker \left(A^{*}\right)}$ and ${\displaystyle \operatorname {ran} \left(A^{+}\right)=\operatorname {ran} \left(A^{*}\right)}$.

Identities

The following identity formula can be used to cancel or expand certain subexpressions involving pseudoinverses: $${\displaystyle A={}A{}A^{*}{}A^{+*}{}={}A^{+*}{}A^{*}{}A.}$$ Equivalently, substituting ${\displaystyle A^{+}}$ for ${\displaystyle A}$ gives $${\displaystyle A^{+}={}A^{+}{}A^{+*}{}A^{*}{}={}A^{*}{}A^{+*}{}A^{+},}$$ while substituting ${\displaystyle A^{*}}$ for ${\displaystyle A}$ gives $${\displaystyle A^{*}={}A^{*}{}A{}A^{+}{}={}A^{+}{}A{}A^{*}.}$$

Reduction to Hermitian case

The computation of the pseudoinverse is reducible to its construction in the Hermitian case. This is possible through the equivalences: $${\displaystyle A^{+}=\left(A^{*}A\right)^{+}A^{*},}$$ $${\displaystyle A^{+}=A^{*}\left(AA^{*}\right)^{+},}$$

as ⁠${\displaystyle A^{*}A}$⁠ and ⁠${\displaystyle AA^{*}}$⁠ are Hermitian.

Pseudoinverse of products

The equality ⁠${\displaystyle (AB)^{+}=B^{+}A^{+}}$⁠ does not hold in general. Rather, suppose ⁠${\displaystyle A\in \mathbb {K} ^{m\times n},\ B\in \mathbb {K} ^{n\times p}}$⁠. Then the following are equivalent:

1. ${\textstyle (AB)^{+}=B^{+}A^{+}}$
2. ${\displaystyle A^{+}ABB^{*}A^{*}=BB^{*}A^{*}}$ and ${\displaystyle BB^{+}A^{*}AB=A^{*}AB}$
3. ${\textstyle \left(A^{+}ABB^{*}\right)^{*}=A^{+}ABB^{*}}$ and ${\displaystyle \left(A^{*}ABB^{+}\right)^{*}=A^{*}ABB^{+}}$
4. ${\textstyle A^{+}ABB^{*}A^{*}ABB^{+}=BB^{*}A^{*}A}$
5. ${\textstyle A^{+}AB=B(AB)^{+}AB}$ and ${\displaystyle BB^{+}A^{*}=A^{*}AB(AB)^{+}}$.

The following are sufficient conditions for ⁠${\displaystyle (AB)^{+}=B^{+}A^{+}}$⁠:

1. ⁠${\displaystyle A}$⁠ has orthonormal columns (then ${\displaystyle A^{*}A=A^{+}A=I_{n}}$), or
2. ⁠${\displaystyle B}$⁠ has orthonormal rows (then ${\displaystyle BB^{*}=BB^{+}=I_{n}}$), or
3. ⁠${\displaystyle A}$⁠ has linearly independent columns (then ${\displaystyle A^{+}A=I}$ ) and ⁠${\displaystyle B}$⁠ has linearly independent rows (then ${\displaystyle BB^{+}=I}$),   or
4. ${\displaystyle B=A^{*}}$, or
5. ${\displaystyle B=A^{+}}$.

The following is a necessary condition for ⁠${\displaystyle (AB)^{+}=B^{+}A^{+}}$⁠:

1. ${\displaystyle (A^{+}A)(BB^{+})=(BB^{+})(A^{+}A)}$

The fourth sufficient condition yields the equalities $${\displaystyle {\begin{aligned}\left(AA^{*}\right)^{+}&=A^{+*}A^{+},\\\left(A^{*}A\right)^{+}&=A^{+}A^{+*}.\end{aligned}}}$$

Here is a counterexample where ⁠${\displaystyle (AB)^{+}\neq B^{+}A^{+}}$⁠:

$${\displaystyle {\Biggl (}{\begin{pmatrix}1&1\\0&0\end{pmatrix}}{\begin{pmatrix}0&0\\1&1\end{pmatrix}}{\Biggr )}^{+}={\begin{pmatrix}1&1\\0&0\end{pmatrix}}^{+}={\begin{pmatrix}{\tfrac {1}{2}}&0\\{\tfrac {1}{2}}&0\end{pmatrix}}\quad \neq \quad {\begin{pmatrix}{\tfrac {1}{4}}&0\\{\tfrac {1}{4}}&0\end{pmatrix}}={\begin{pmatrix}0&{\tfrac {1}{2}}\\0&{\tfrac {1}{2}}\end{pmatrix}}{\begin{pmatrix}{\tfrac {1}{2}}&0\\{\tfrac {1}{2}}&0\end{pmatrix}}={\begin{pmatrix}0&0\\1&1\end{pmatrix}}^{+}{\begin{pmatrix}1&1\\0&0\end{pmatrix}}^{+}}$$

Projectors

${\displaystyle P=AA^{+}}$ and ${\displaystyle Q=A^{+}A}$ are orthogonal projection operators, that is, they are Hermitian (${\displaystyle P=P^{*}}$, ${\displaystyle Q=Q^{*}}$) and idempotent (${\displaystyle P^{2}=P}$ and ${\displaystyle Q^{2}=Q}$). The following hold:

• ${\displaystyle PA=AQ=A}$ and ${\displaystyle A^{+}P=QA^{+}=A^{+}}$
• ⁠${\displaystyle P}$⁠ is the orthogonal projector onto the range of ⁠${\displaystyle A}$⁠ (which equals the orthogonal complement of the kernel of ⁠${\displaystyle A^{*}}$⁠).
• ⁠${\displaystyle Q}$⁠ is the orthogonal projector onto the range of ⁠${\displaystyle A^{*}}$⁠ (which equals the orthogonal complement of the kernel of ⁠${\displaystyle A}$⁠).
• ${\displaystyle I-Q=I-A^{+}A}$ is the orthogonal projector onto the kernel of ⁠${\displaystyle A}$⁠.
• ${\displaystyle I-P=I-AA^{+}}$ is the orthogonal projector onto the kernel of ⁠${\displaystyle A^{*}}$⁠.

The last two properties imply the following identities:

• ${\displaystyle A\,\ \left(I-A^{+}A\right)=\left(I-AA^{+}\right)A\ \ =0}$
• ${\displaystyle A^{*}\left(I-AA^{+}\right)=\left(I-A^{+}A\right)A^{*}=0}$

Another property is the following: if ⁠${\displaystyle A\in \mathbb {K} ^{n\times n}}$⁠ is Hermitian and idempotent (true if and only if it represents an orthogonal projection), then, for any matrix ⁠${\displaystyle B\in \mathbb {K} ^{m\times n}}$⁠ the following equation holds: $${\displaystyle A(BA)^{+}=(BA)^{+}}$$

This can be proven by defining matrices ${\displaystyle C=BA}$, ${\displaystyle D=A(BA)^{+}}$, and checking that ⁠${\displaystyle D}$⁠ is indeed a pseudoinverse for ⁠${\displaystyle C}$⁠ by verifying that the defining properties of the pseudoinverse hold, when ⁠${\displaystyle A}$⁠ is Hermitian and idempotent.

From the last property it follows that, if ⁠${\displaystyle A\in \mathbb {K} ^{n\times n}}$⁠ is Hermitian and idempotent, for any matrix ⁠${\displaystyle B\in \mathbb {K} ^{n\times m}}$⁠ $${\displaystyle (AB)^{+}A=(AB)^{+}}$$

Finally, if ⁠${\displaystyle A}$⁠ is an orthogonal projection matrix, then its pseudoinverse trivially coincides with the matrix itself, that is, ${\displaystyle A^{+}=A}$.

Geometric construction

If we view the matrix as a linear map ⁠${\displaystyle A:\mathbb {K} ^{n}\to \mathbb {K} ^{m}}$⁠ over the field ⁠${\displaystyle \mathbb {K} }$⁠ then ⁠${\displaystyle A^{+}:\mathbb {K} ^{m}\to \mathbb {K} ^{n}}$⁠ can be decomposed as follows. We write ⁠${\displaystyle \oplus }$⁠ for the direct sum, ⁠${\displaystyle \perp }$⁠ for the orthogonal complement, ⁠${\displaystyle \ker }$⁠ for the kernel of a map, and ⁠${\displaystyle \operatorname {ran} }$⁠ for the image of a map. Notice that ${\displaystyle \mathbb {K} ^{n}=\left(\ker A\right)^{\perp }\oplus \ker A}$ and ${\displaystyle \mathbb {K} ^{m}=\operatorname {ran} A\oplus \left(\operatorname {ran} A\right)^{\perp }}$. The restriction ${\displaystyle A:\left(\ker A\right)^{\perp }\to \operatorname {ran} A}$ is then an isomorphism. This implies that ⁠${\displaystyle A^{+}}$⁠ on ⁠${\displaystyle \operatorname {ran} A}$⁠ is the inverse of this isomorphism, and is zero on ${\displaystyle \left(\operatorname {ran} A\right)^{\perp }.}$

In other words: To find ⁠${\displaystyle A^{+}b}$⁠ for given ⁠${\displaystyle b}$⁠ in ⁠${\displaystyle \mathbb {K} ^{m}}$⁠, first project ⁠${\displaystyle b}$⁠ orthogonally onto the range of ⁠${\displaystyle A}$⁠, finding a point ⁠${\displaystyle p(b)}$⁠ in the range. Then form ⁠${\displaystyle A^{-1}(\{p(b)\})}$⁠, that is, find those vectors in ⁠${\displaystyle \mathbb {K} ^{n}}$⁠ that ⁠${\displaystyle A}$⁠ sends to ⁠${\displaystyle p(b)}$⁠. This will be an affine subspace of ⁠${\displaystyle \mathbb {K} ^{n}}$⁠ parallel to the kernel of ⁠${\displaystyle A}$⁠. The element of this subspace that has the smallest length (that is, is closest to the origin) is the answer ⁠${\displaystyle A^{+}b}$⁠ we are looking for. It can be found by taking an arbitrary member of ⁠${\displaystyle A^{-1}(\{p(b)\})}$⁠ and projecting it orthogonally onto the orthogonal complement of the kernel of ⁠${\displaystyle A}$⁠.

This description is closely related to the minimum-norm solution to a linear system.

Limit relations

The pseudoinverse are limits: $${\displaystyle A^{+}=\lim _{\delta \searrow 0}\left(A^{*}A+\delta I\right)^{-1}A^{*}=\lim _{\delta \searrow 0}A^{*}\left(AA^{*}+\delta I\right)^{-1}}$$ (see Tikhonov regularization). These limits exist even if ⁠${\displaystyle \left(AA^{*}\right)^{-1}}$⁠ or ⁠${\displaystyle \left(A^{*}A\right)^{-1}}$⁠ do not exist.

Continuity

In contrast to ordinary matrix inversion, the process of taking pseudoinverses is not continuous: if the sequence ⁠${\displaystyle \left(A_{n}\right)}$⁠ converges to the matrix ⁠${\displaystyle A}$⁠ (in the maximum norm or Frobenius norm, say), then ⁠${\displaystyle (A_{n})^{+}}$⁠ need not converge to ⁠${\displaystyle A^{+}}$⁠. However, if all the matrices ⁠${\displaystyle A_{n}}$⁠ have the same rank as ⁠${\displaystyle A}$⁠, ⁠${\displaystyle (A_{n})^{+}}$⁠ will converge to ⁠${\displaystyle A^{+}}$⁠.

Derivative

Let ${\displaystyle x\mapsto A(x)}$ be a real-valued differentiable matrix function with constant rank at a point ⁠${\displaystyle x_{0}}$⁠. The derivative of ${\displaystyle x\mapsto A^{+}(x)}$ at ${\displaystyle x_{0}}$ may be calculated in terms of the derivative of ${\displaystyle A}$ at ${\displaystyle x_{0}}$: $${\displaystyle \left.{\frac {\mathrm {d} }{\mathrm {d} x}}\right|_{x=x_{0}\!\!\!\!\!\!\!}A^{+}=-A^{+}\left({\frac {\mathrm {d} A}{\mathrm {d} x}}\right)A^{+}~+~A^{+}A^{+\top }\left({\frac {\mathrm {d} A^{\top }}{\mathrm {d} x}}\right)\left(I-AA^{+}\right)~+~\left(I-A^{+}A\right)\left({\frac {\mathrm {d} A^{\top }}{\mathrm {d} x}}\right)A^{+\top }A^{+},}$$ where the functions ${\displaystyle A}$, ${\displaystyle A^{+}}$ and derivatives on the right side are evaluated at ${\displaystyle x_{0}}$ (that is, ${\displaystyle A:=A(x_{0})}$, ${\displaystyle A^{+}:=A^{+}(x_{0})}$, etc.). For a complex matrix, the transpose is replaced with the conjugate transpose. For a real-valued symmetric matrix, the Magnus-Neudecker derivative is established.

Examples

Since for invertible matrices the pseudoinverse equals the usual inverse, only examples of non-invertible matrices are considered below.

• For ${\displaystyle A={\begin{pmatrix}0&0\\0&0\end{pmatrix}},}$ the pseudoinverse is ${\displaystyle A^{+}={\begin{pmatrix}0&0\\0&0\end{pmatrix}}.}$ The uniqueness of this pseudoinverse can be seen from the requirement ${\displaystyle A^{+}=A^{+}AA^{+}}$, since multiplication by a zero matrix would always produce a zero matrix.
• For ${\displaystyle A={\begin{pmatrix}1&0\\1&0\end{pmatrix}},}$ the pseudoinverse is ${\displaystyle A^{+}={\begin{pmatrix}{\frac {1}{2}}&{\frac {1}{2}}\\0&0\end{pmatrix}}}$.

Indeed, ${\displaystyle A\,A^{+}={\begin{pmatrix}{\frac {1}{2}}&{\frac {1}{2}}\\{\frac {1}{2}}&{\frac {1}{2}}\end{pmatrix}}}$, and thus ${\displaystyle A\,A^{+}A={\begin{pmatrix}1&0\\1&0\end{pmatrix}}=A}$. Similarly, ${\displaystyle A^{+}A={\begin{pmatrix}1&0\\0&0\end{pmatrix}}}$, and thus ${\displaystyle A^{+}A\,A^{+}={\begin{pmatrix}{\frac {1}{2}}&{\frac {1}{2}}\\0&0\end{pmatrix}}=A^{+}}$.

Note that ⁠${\displaystyle A}$⁠ is neither injective nor surjective, and thus the pseudoinverse cannot be computed via ${\displaystyle A^{+}=\left(A^{*}A\right)^{-1}A^{*}}$ nor ${\displaystyle A^{+}=A^{*}\left(AA^{*}\right)^{-1}}$, as ${\displaystyle A^{*}A}$ and ${\displaystyle AA^{*}}$ are both singular, and furthermore ${\displaystyle A^{+}}$ is neither a left nor a right inverse.

Nonetheless, the pseudoinverse can be computed via SVD observing that ${\displaystyle A={\sqrt {2}}\left({\frac {\mathbf {e} _{1}+\mathbf {e} _{2}}{\sqrt {2}}}\right)\mathbf {e} _{1}^{*}}$, and thus ${\displaystyle A^{+}={\frac {1}{\sqrt {2}}}\,\mathbf {e} _{1}\left({\frac {\mathbf {e} _{1}+\mathbf {e} _{2}}{\sqrt {2}}}\right)^{*}}$.

• For ${\displaystyle A={\begin{pmatrix}1&0\\-1&0\end{pmatrix}},}$ ${\displaystyle A^{+}={\begin{pmatrix}{\frac {1}{2}}&-{\frac {1}{2}}\\0&0\end{pmatrix}}.}$
• For ${\displaystyle A={\begin{pmatrix}1&0\\2&0\end{pmatrix}},}$ ${\displaystyle A^{+}={\begin{pmatrix}{\frac {1}{5}}&{\frac {2}{5}}\\0&0\end{pmatrix}}}$. The denominators are here ${\displaystyle 5=1^{2}+2^{2}}$.
• For ${\displaystyle A={\begin{pmatrix}1&1\\1&1\end{pmatrix}},}$ ${\displaystyle A^{+}={\begin{pmatrix}{\frac {1}{4}}&{\frac {1}{4}}\\{\frac {1}{4}}&{\frac {1}{4}}\end{pmatrix}}.}$
• For ${\displaystyle A={\begin{pmatrix}1&0\\0&1\\0&1\end{pmatrix}},}$ the pseudoinverse is ${\displaystyle A^{+}={\begin{pmatrix}1&0&0\\0&{\frac {1}{2}}&{\frac {1}{2}}\end{pmatrix}}}$.

For this matrix, the left inverse exists and thus equals ${\displaystyle A^{+}}$, indeed, ${\displaystyle A^{+}A={\begin{pmatrix}1&0\\0&1\end{pmatrix}}.}$

Special cases

Scalars

It is also possible to define a pseudoinverse for scalars and vectors. This amounts to treating these as matrices. The pseudoinverse of a scalar ⁠${\displaystyle x}$⁠ is zero if ⁠${\displaystyle x}$⁠ is zero and the reciprocal of ⁠${\displaystyle x}$⁠ otherwise: $${\displaystyle x^{+}={\begin{cases}0,&{\mbox{if }}x=0;\\x^{-1},&{\mbox{otherwise}}.\end{cases}}}$$

Vectors

The pseudoinverse of the null (all zero) vector is the transposed null vector. The pseudoinverse of a non-null vector is the conjugate transposed vector divided by its squared magnitude:

$${\displaystyle {\vec {x}}^{+}={\begin{cases}{\vec {0}}^{\operatorname {T} },&{\text{if }}{\vec {x}}={\vec {0}};\\{\vec {x}}^{*}/({\vec {x}}^{*}{\vec {x}}),&{\text{otherwise}}.\end{cases}}}$$

Diagonal matrices

The pseudoinverse of a squared diagonal matrix is obtained by taking the reciprocal of the nonzero diagonal elements. Formally, if ${\displaystyle D}$ is a squared diagonal matrix with ${\displaystyle D={\tilde {D}}\oplus \mathbf {0} _{k\times k}}$ and ${\displaystyle {\tilde {D}}>0}$, then ${\displaystyle D^{+}={\tilde {D}}^{-1}\oplus \mathbf {0} _{k\times k}}$. More generally, if ${\displaystyle A}$ is any ${\displaystyle m\times n}$ rectangular matrix whose only nonzero elements are on the diagonal, meaning ${\displaystyle A_{ij}=\delta _{ij}a_{i}}$, ${\displaystyle a_{i}\in \mathbb {K} }$, then ${\displaystyle A^{+}}$ is a ${\displaystyle n\times m}$ rectangular matrix whose diagonal elements are the reciprocal of the original ones, that is, ${\displaystyle A_{ii}\neq 0\implies A_{ii}^{+}=1/A_{ii}}$.

Linearly independent columns

If the rank of ⁠${\displaystyle A}$⁠ is identical to the number of columns, ⁠${\displaystyle n}$⁠, (for ⁠${\displaystyle n\leq m}$⁠,) there are ⁠${\displaystyle n}$⁠ linearly independent columns, and ⁠${\displaystyle A^{*}A}$⁠ is invertible. In this case, an explicit formula is: $${\displaystyle A^{+}=\left(A^{*}A\right)^{-1}A^{*}.}$$

It follows that ⁠${\displaystyle A^{+}}$⁠ is then a left inverse of ⁠${\displaystyle A}$⁠:   ${\displaystyle A^{+}A=I_{n}}$.

Linearly independent rows

If the rank of ⁠${\displaystyle A}$⁠ is identical to the number of rows, ⁠${\displaystyle m}$⁠, (for ⁠${\displaystyle m\leq n}$⁠,) there are ⁠${\displaystyle m}$⁠ linearly independent rows, and ⁠${\displaystyle AA^{*}}$⁠ is invertible. In this case, an explicit formula is: $${\displaystyle A^{+}=A^{*}\left(AA^{*}\right)^{-1}.}$$

It follows that ⁠${\displaystyle A^{+}}$⁠ is a right inverse of ⁠${\displaystyle A}$⁠:   ${\displaystyle AA^{+}=I_{m}}$.

Orthonormal columns or rows

This is a special case of either full column rank or full row rank (treated above). If ⁠${\displaystyle A}$⁠ has orthonormal columns (${\displaystyle A^{*}A=I_{n}}$) or orthonormal rows (${\displaystyle AA^{*}=I_{m}}$), then: $${\displaystyle A^{+}=A^{*}.}$$

Normal matrices

If ⁠${\displaystyle A}$⁠ is normal, that is, it commutes with its conjugate transpose, then its pseudoinverse can be computed by diagonalizing it, mapping all nonzero eigenvalues to their inverses, and mapping zero eigenvalues to zero. A corollary is that ⁠${\displaystyle A}$⁠ commuting with its transpose implies that it commutes with its pseudoinverse.

EP matrices

A (square) matrix ⁠${\displaystyle A}$⁠ is said to be an EP matrix if it commutes with its pseudoinverse. In such cases (and only in such cases), it is possible to obtain the pseudoinverse as a polynomial in ⁠${\displaystyle A}$⁠. A polynomial ${\displaystyle p(t)}$ such that ${\displaystyle A^{+}=p(A)}$ can be easily obtained from the characteristic polynomial of ⁠${\displaystyle A}$⁠ or, more generally, from any annihilating polynomial of ⁠${\displaystyle A}$⁠.

Orthogonal projection matrices

This is a special case of a normal matrix with eigenvalues 0 and 1. If ⁠${\displaystyle A}$⁠ is an orthogonal projection matrix, that is, ${\displaystyle A=A^{*}}$ and ${\displaystyle A^{2}=A}$, then the pseudoinverse trivially coincides with the matrix itself: $${\displaystyle A^{+}=A.}$$

Circulant matrices

For a circulant matrix ⁠${\displaystyle C}$⁠, the singular value decomposition is given by the Fourier transform, that is, the singular values are the Fourier coefficients. Let ⁠${\displaystyle {\mathcal {F}}}$⁠ be the Discrete Fourier Transform (DFT) matrix; then $${\displaystyle {\begin{aligned}C&={\mathcal {F}}\cdot \Sigma \cdot {\mathcal {F}}^{*},\\C^{+}&={\mathcal {F}}\cdot \Sigma ^{+}\cdot {\mathcal {F}}^{*}.\end{aligned}}}$$

Construction

Rank decomposition

Let ⁠${\displaystyle r\leq \min(m,n)}$⁠ denote the rank of ⁠${\displaystyle A\in \mathbb {K} ^{m\times n}}$⁠. Then ⁠${\displaystyle A}$⁠ can be (rank) decomposed as ${\displaystyle A=BC}$ where ⁠${\displaystyle B\in \mathbb {K} ^{m\times r}}$⁠ and ⁠${\displaystyle C\in \mathbb {K} ^{r\times n}}$⁠ are of rank ⁠${\displaystyle r}$⁠. Then ${\displaystyle A^{+}=C^{+}B^{+}=C^{*}\left(CC^{*}\right)^{-1}\left(B^{*}B\right)^{-1}B^{*}}$.

The QR method

For ${\displaystyle \mathbb {K} \in \{\mathbb {R} ,\mathbb {C} \}}$ computing the product ⁠${\displaystyle AA^{*}}$⁠ or ⁠${\displaystyle A^{*}A}$⁠ and their inverses explicitly is often a source of numerical rounding errors and computational cost in practice. An alternative approach using the QR decomposition of ⁠${\displaystyle A}$⁠ may be used instead.

Consider the case when ⁠${\displaystyle A}$⁠ is of full column rank, so that ${\displaystyle A^{+}=\left(A^{*}A\right)^{-1}A^{*}}$. Then the Cholesky decomposition ${\displaystyle A^{*}A=R^{*}R}$, where ⁠${\displaystyle R}$⁠ is an upper triangular matrix, may be used. Multiplication by the inverse is then done easily by solving a system with multiple right-hand sides, $${\displaystyle A^{+}=\left(A^{*}A\right)^{-1}A^{*}\quad \Leftrightarrow \quad \left(A^{*}A\right)A^{+}=A^{*}\quad \Leftrightarrow \quad R^{*}RA^{+}=A^{*}}$$