Misplaced Pages

Blackwell-Girshick equation

Article snapshot taken from Wikipedia with creative commons attribution-sharealike license. Give it a read and then ask your questions in the chat. We can research this topic together.
Variance of random sum

The Blackwell-Girshick equation is an equation in probability theory that allows for the calculation of the variance of random sums of random variables. It is the equivalent of Wald's lemma for the expectation of composite distributions.

It is named after David Blackwell and Meyer Abraham Girshick.

Statement

Let N {\displaystyle N} be a random variable with values in Z ≥ 0 {\displaystyle \mathbb {Z} _{\geq 0}} , let X 1 , X 2 , X 3 , … {\displaystyle X_{1},X_{2},X_{3},\dots } be independent and identically distributed random variables, which are also independent of N {\displaystyle N} , and assume that the second moment exists for all X i {\displaystyle X_{i}} and N {\displaystyle N} . Then, the random variable defined by

Y := ∑ i = 1 N X i {\displaystyle Y:=\sum _{i=1}^{N}X_{i}}

has the variance

Var ⁡ ( Y ) = Var ⁡ ( N ) E ⁡ ( X 1 ) 2 + E ⁡ ( N ) Var ⁡ ( X 1 ) {\displaystyle \operatorname {Var} (Y)=\operatorname {Var} (N)\operatorname {E} (X_{1})^{2}+\operatorname {E} (N)\operatorname {Var} (X_{1})} .

The Blackwell-Girshick equation can be derived using conditional variance and variance decomposition. If the X i {\displaystyle X_{i}} are natural number-valued random variables, the derivation can be done elementarily using the chain rule and the probability-generating function.

Proof

For each n ≥ 0 {\displaystyle n\geq 0} , let χ n {\displaystyle \chi _{n}} be the random variable which is 1 if N {\displaystyle N} equals n {\displaystyle n} and 0 otherwise, and let Y n := X 1 + ⋯ + X n {\displaystyle Y_{n}:=X_{1}+\cdots +X_{n}} . Then

E ⁡ ( Y 2 ) = ∑ n = 0 ∞ E ⁡ ( χ n Y n 2 ) = ∑ n = 0 ∞ P ⁡ ( N = n ) E ⁡ ( Y n 2 ) = ∑ n = 0 ∞ P ⁡ ( N = n ) ( Var ⁡ ( Y n ) + E ⁡ ( Y n ) 2 ) = ∑ n = 0 ∞ P ⁡ ( N = n ) ( n Var ⁡ ( X 1 ) + n 2 E ⁡ ( X 1 ) 2 ) = E ⁡ ( N ) Var ⁡ ( X 1 ) + E ⁡ ( N 2 ) E ⁡ ( X 1 ) 2 . {\displaystyle {\begin{aligned}\operatorname {E} (Y^{2})&=\sum _{n=0}^{\infty }\operatorname {E} (\chi _{n}Y_{n}^{2})\\&=\sum _{n=0}^{\infty }\operatorname {P} (N=n)\operatorname {E} (Y_{n}^{2})\\&=\sum _{n=0}^{\infty }\operatorname {P} (N=n)(\operatorname {Var} (Y_{n})+\operatorname {E} (Y_{n})^{2})\\&=\sum _{n=0}^{\infty }\operatorname {P} (N=n)(n\operatorname {Var} (X_{1})+n^{2}\operatorname {E} (X_{1})^{2})\\&=\operatorname {E} (N)\operatorname {Var} (X_{1})+\operatorname {E} (N^{2})\operatorname {E} (X_{1})^{2}.\end{aligned}}}

By Wald's equation, under the given hypotheses, E ⁡ ( Y ) = E ⁡ ( N ) E ⁡ ( X 1 ) {\displaystyle \operatorname {E} (Y)=\operatorname {E} (N)\operatorname {E} (X_{1})} . Therefore,

Var ⁡ ( Y ) = E ⁡ ( Y 2 ) − E ⁡ ( Y ) 2 = E ⁡ ( N ) Var ⁡ ( X 1 ) + E ⁡ ( N 2 ) E ⁡ ( X 1 ) 2 − E ⁡ ( N ) 2 E ⁡ ( X 1 ) 2 = E ⁡ ( N ) Var ⁡ ( X 1 ) + Var ⁡ ( N ) E ⁡ ( X 1 ) 2 , {\displaystyle {\begin{aligned}\operatorname {Var} (Y)&=\operatorname {E} (Y^{2})-\operatorname {E} (Y)^{2}\\&=\operatorname {E} (N)\operatorname {Var} (X_{1})+\operatorname {E} (N^{2})\operatorname {E} (X_{1})^{2}-\operatorname {E} (N)^{2}\operatorname {E} (X_{1})^{2}\\&=\operatorname {E} (N)\operatorname {Var} (X_{1})+\operatorname {Var} (N)\operatorname {E} (X_{1})^{2},\end{aligned}}}

as desired.

Example

Let N {\displaystyle N} have a Poisson distribution with expectation λ {\displaystyle \lambda } , and let X 1 , X 2 , … {\displaystyle X_{1},X_{2},\dots } follow a Bernoulli distribution with parameter p {\displaystyle p} . In this case, Y {\displaystyle Y} is also Poisson distributed with expectation λ p {\displaystyle \lambda p} , so its variance must be λ p {\displaystyle \lambda p} . We can check this with the Blackwell-Girshick equation: N {\displaystyle N} has variance λ {\displaystyle \lambda } while each X i {\displaystyle X_{i}} has mean p {\displaystyle p} and variance p ( 1 − p ) {\displaystyle p(1-p)} , so we must have

Var ⁡ ( Y ) = λ p 2 + λ p ( 1 − p ) = λ p {\displaystyle \operatorname {Var} (Y)=\lambda p^{2}+\lambda p(1-p)=\lambda p} .

Application and related concepts

The Blackwell-Girshick equation is used in actuarial mathematics to calculate the variance of composite distributions, such as the compound Poisson distribution. Wald's equation provides similar statements about the expectation of composite distributions.

Literature

  • For an example of an application: Mühlenthaler, M.; Raß, A.; Schmitt, M.; Wanka, R. (2021). "Exact Markov chain-based runtime analysis of a discrete particle swarm optimization algorithm on sorting and OneMax". Natural Computing: 1–27.

References

  1. Blackwell, D. A.; Girshick, M. A. (1979). Theory of games and statistical decisions. Courier Corporation.
  2. Achim Klenke (2013), Wahrscheinlichkeitstheorie (3rd ed.), Berlin Heidelberg: Springer-Verlag, p. 109, doi:10.1007/978-3-642-36018-3, ISBN 978-3-642-36017-6, S2CID 242882110
  3. Probability Theory : A Comprehensive Course, Achim Klenke, London, Heidelberg, New York, Dordrecht: Springer, 2nd ed., 2014, ISBN 978-1-4471-5360-3.
Category: