Schur product theorem: Difference between revisions

Browse history interactively Next edit →Content deleted Content addedVisual WikitextInline

Revision as of 11:58, 23 November 2013 editLilily (talk \| contribs)260 edits create page with references and proofs		Revision as of 11:36, 24 November 2013 edit undoLilily (talk \| contribs)260 edits →Proof using Gaussian integration: corrected and generalised proofNext edit →
Line 16:		Line 16:
	=== Proof using Gaussian integration ===		=== Proof using Gaussian integration ===

			==== Case of M=N ====
⚫	Let <math>X</math> be an <math>n</math> dimensional centered ] with ] <math>\langle X_i X_j \rangle = C_{ij}</math>.

		⚫	Let <math>X</math> be an <math>n</math> dimensional centered ] with ] <math>\langle X_i X_j \rangle = M_{ij}</math>.
	Then the covariance matrix of <math>X_i^2</math> and <math>X_j^2</math> is		Then the covariance matrix of <math>X_i^2</math> and <math>X_j^2</math> is
	: <math>\operatorname{Cov}(X_i^2, X_j^2) = \langle X_i^2 X_j^2 \rangle - \langle X_i^2 \rangle \langle X_j^2 \rangle</math>		: <math>\operatorname{Cov}(X_i^2, X_j^2) = \langle X_i^2 X_j^2 \rangle - \langle X_i^2 \rangle \langle X_j^2 \rangle</math>
	Using ] to develop <math>\langle X_i^2 X_j^2 \rangle = 2 \langle X_i X_j \rangle^2 + \langle X_i^2 \rangle \langle X_j^2 \rangle</math> we have		Using ] to develop <math>\langle X_i^2 X_j^2 \rangle = 2 \langle X_i X_j \rangle^2 + \langle X_i^2 \rangle \langle X_j^2 \rangle</math> we have
	: <math>\operatorname{Cov}(X_i^2, X_j^2) = 2 \langle X_i X_j \rangle^2 = 2 C_{ij}^2</math>		: <math>\operatorname{Cov}(X_i^2, X_j^2) = 2 \langle X_i X_j \rangle^2 = 2 M_{ij}^2</math>
	Since a covariance matrix is positive definite, this proves that the matrix with elements <math>C_{ij}^2</math> is a positive definite matrix.		Since a covariance matrix is positive definite, this proves that the matrix with elements <math>M_{ij}^2</math> is a positive definite matrix.

			==== General case ====

			Let <math>X</math> and <math>Y</math> be <math>n</math> dimensional centered ]s with ]s <math>\langle X_i X_j \rangle = M_{ij}</math>, <math>\langle Y_i Y_j \rangle = N_{ij}</math> and independt from each other so that we have
			: <math>\langle X_i Y_j \rangle = 0</math> for any <math>i, j</math>
			Then the covariance matrix of <math>X_i Y_i</math> and <math>X_j Y_j</math> is
			: <math>\operatorname{Cov}(X_i Y_i, X_j Y_j) = \langle X_i Y_i X_j Y_j \rangle - \langle X_i Y_i \rangle \langle X_j Y_j \rangle</math>
			Using ] to develop
			: <math>\langle X_i Y_i X_j Y_j \rangle = \langle X_i X_j \rangle \langle Y_i Y_j \rangle + \langle X_i Y_i \rangle \langle X_i Y_j \rangle + \langle X_i Y_j \rangle \langle X_j Y_i \rangle</math>
			and also using the independence of <math>X</math> and <math>Y</math>, we have
			: <math>\operatorname{Cov}(X_i Y_i, X_j Y_j) = \langle X_i X_j \rangle \langle Y_i Y_j \rangle = M_{ij} N_{ij}</math>
			Since a covariance matrix is positive definite, this proves that the matrix with elements <math>M_{ij} N_{ij}</math> is a positive definite matrix.

	== Refercenes ==		== Refercenes ==

Revision as of 11:36, 24 November 2013

In mathematics, particularly linear algebra, the Schur product theorem, named after Issai Schur (Schur 1911, p. 14, Theorem VII) (note that Schur signed as J. Schur in Journal für die reine und angewandte Mathematik) states that the Hadamard product of two positive definite matrices is also a positive definite matrix.

Proof

Proof using the trace formula

It is easy to show that for matrices $M$ and $N$ , the Hadamard product $M\circ N$ considered as a bilinear form acts on vectors $a,b$ as

a^{T}(M\circ N)b=\operatorname {Tr} (M\operatorname {diag} (a)N\operatorname {diag} (b))

where $\operatorname {Tr}$ is the matrix trace and $\operatorname {diag} (a)$ is the diagonal matrix having as diagonal entries the elements of $a$ .

Since $M$ and $N$ are positive definite, we can consider their square-roots $M^{1/2}$ and $N^{1/2}$ and write

\operatorname {Tr} (M\operatorname {diag} (a)N\operatorname {diag} (b))=\operatorname {Tr} (M^{1/2}M^{1/2}\operatorname {diag} (a)N^{1/2}N^{1/2}\operatorname {diag} (b))=\operatorname {Tr} (M^{1/2}\operatorname {diag} (a)N^{1/2}N^{1/2}\operatorname {diag} (b)M^{1/2})

Then, for $a=b$ , this is written as $\operatorname {Tr} (A^{T}A)$ for $A=N^{1/2}\operatorname {diag} (a)M^{1/2}$ and thus is positive. This shows that $(M\circ N)$ is a positive definite matrix.

Proof using Gaussian integration

Case of M=N

Let $X$ be an $n$ dimensional centered Gaussian random variable with covariance $\langle X_{i}X_{j}\rangle =M_{ij}$ . Then the covariance matrix of $X_{i}^{2}$ and $X_{j}^{2}$ is

\operatorname {Cov} (X_{i}^{2},X_{j}^{2})=\langle X_{i}^{2}X_{j}^{2}\rangle -\langle X_{i}^{2}\rangle \langle X_{j}^{2}\rangle

Using Wick's theorem to develop $\langle X_{i}^{2}X_{j}^{2}\rangle =2\langle X_{i}X_{j}\rangle ^{2}+\langle X_{i}^{2}\rangle \langle X_{j}^{2}\rangle$ we have

\operatorname {Cov} (X_{i}^{2},X_{j}^{2})=2\langle X_{i}X_{j}\rangle ^{2}=2M_{ij}^{2}

Since a covariance matrix is positive definite, this proves that the matrix with elements $M_{ij}^{2}$ is a positive definite matrix.

General case

Let $X$ and $Y$ be $n$ dimensional centered Gaussian random variables with covariances $\langle X_{i}X_{j}\rangle =M_{ij}$ , $\langle Y_{i}Y_{j}\rangle =N_{ij}$ and independt from each other so that we have

\langle X_{i}Y_{j}\rangle =0

for any

i,j

Then the covariance matrix of $X_{i}Y_{i}$ and $X_{j}Y_{j}$ is

\operatorname {Cov} (X_{i}Y_{i},X_{j}Y_{j})=\langle X_{i}Y_{i}X_{j}Y_{j}\rangle -\langle X_{i}Y_{i}\rangle \langle X_{j}Y_{j}\rangle

Using Wick's theorem to develop

\langle X_{i}Y_{i}X_{j}Y_{j}\rangle =\langle X_{i}X_{j}\rangle \langle Y_{i}Y_{j}\rangle +\langle X_{i}Y_{i}\rangle \langle X_{i}Y_{j}\rangle +\langle X_{i}Y_{j}\rangle \langle X_{j}Y_{i}\rangle

and also using the independence of $X$ and $Y$ , we have

\operatorname {Cov} (X_{i}Y_{i},X_{j}Y_{j})=\langle X_{i}X_{j}\rangle \langle Y_{i}Y_{j}\rangle =M_{ij}N_{ij}

Since a covariance matrix is positive definite, this proves that the matrix with elements $M_{ij}N_{ij}$ is a positive definite matrix.

Refercenes

Attention: This template ({{cite doi}}) is deprecated. To cite the publication identified by doi:10.1515/crll.1911.140.1, please use {{cite journal}} (if it was published in a bona fide academic journal, otherwise {{cite report}} with |doi=10.1515/crll.1911.140.1 instead.
Attention: This template ({{cite doi}}) is deprecated. To cite the publication identified by doi:10.1007/b105056, please use {{cite journal}} (if it was published in a bona fide academic journal, otherwise {{cite report}} with |doi=10.1007/b105056 instead., page 9, Ch. 0.6 Publication under J. Schur
Attention: This template ({{cite doi}}) is deprecated. To cite the publication identified by doi:10.1112/blms/15.2.97, please use {{cite journal}} (if it was published in a bona fide academic journal, otherwise {{cite report}} with |doi=10.1112/blms/15.2.97 instead.