A well known fact about joint normally distributed random variables, is that they are independent if and only if their covariance is zero. In one direction, this statement is trivial. *Any* independent pair of random variables has zero covariance (assuming that they are integrable, so that the covariance has a well-defined value). The strength of the statement is in the other direction. Knowing the value of the covariance does not tell us a lot about the joint distribution so, in the case that they are joint normal, the fact that we can determine independence from this is a rather strong statement.

Theorem 1A joint normal pair of random variables are independent if and only if their covariance is zero.

*Proof:* Suppose that *X,Y* are joint normal, such that and , and that their covariance is *c*. Then, the characteristic function of can be computed as

for all . It is standard that the joint characteristic function of a pair of random variables is equal to the product of their characteristic functions if and only if they are independent which, in this case, corresponds to the covariance *c* being zero. ⬜

To demonstrate necessity of the joint normality condition, consider the example from the previous post.

Example 1A pair of standard normal random variablesX,Ywhich have zero covariance, butis not normal.

As their sum is not normal, *X* and *Y* cannot be independent. This example was constructed by setting for some fixed , which is standard normal whenever *X* is. As explained in the previous post, the intermediate value theorem ensures that there is a unique value for *K* making the covariance equal to zero.

Theorem 1 generalizes in a straightforward manner to more than two random variables, even for infinitely many.

Theorem 2Letbe a joint normal collection of random variables. Then, theare independent if and only if their covariances are zero.

This result can be established by a straightforward extension of the proof given or theorem 1 above. However, it is also a special case of a more general independence result to be given further below, so I leave the proof until then. Using the equivalence of pairwise independence and zero covariances, theorem 2 can also be expressed without direct reference to covariances.

Theorem 3Letbe a joint normal collection of random variables. Then, theare independent if and only if they are pairwise independent.

Theorem 1 can also be extended in a different direction. Rather than replacing the pair of random variables by an arbitrary collection, all of which have zero covariance, they can instead be replaced by a pair of collections of variables.

Theorem 4Letandbe collections of random variables such thatis joint normal. Then,andare independent if and only ifXandYhave zero covariance for alland.

Just to clarify, this statement does not mean that random variables in are independent from each other but, rather, that the collection is independent of . As above, the equivalence of zero covariances and pairwise independence stated by theorem 1 allows to write this result without reference to covariances.

Theorem 5Letandbe collections of random variables such thatis joint normal. Then,andare independent if and only ifXandYare independent for alland.

Although it is not difficult to generalize the proof of theorem 1 in order to directly prove theorems 4 and 5, they are also just special cases of a more general result which I prove further below. For now, the following example application is very useful in describing a Brownian motion *X* over the unit interval in terms of its endpoints and an independent Brownian bridge .

Example 2 (Brownian bridge)Letbe a standard Brownian motion. Then, the processoveris independent ofover.

This example is a direct application of theorem 4 to the collections and . The covariances of the Brownian motion are given by . Hence, for any , we compute

as required.

The necessity of the joint normal condition in theorems 2, 3, 4, and 5 is demonstrated by the following example.

Example 3A tripleof standard normals which arepairwiseindependent (and, hence, pairwise joint normal), but are not all independent (hence, not all joint normal).

This shows that we cannot infer independence from pairwise independence for arbitrary collections of normal random variables, so that the joint normality of in theorems 2 and 3 is necessary. Furthermore, consider the collections and , both of which are joint normal. Then, *X* and *Y* are independent for all and even though is not independent of , demonstrating that joint normality of the union is necessary for theorems 4 and 5.

Example 3 can be constructed as follows. Let be independent random variables, where the are standard normal and have the Rademacher distribution, . Then, set , which also has the Rademacher distribution. As the are pairwise independent, then the random variables are also pairwise independent and, by symmetry of the standard normal distribution, are also standard normal. However, the product is equal to 1 and, hence,

almost surely.

Theorems 1, 2, 3, 4 and 5 are all special cases of the following `master’ theorem.

Theorem 6Letbe a collection of collections of random variables, whose union is joint normal.

Then, they are independent collections if and only iffor allandforinI.

For clarity, I state precisely what the independence property means in this result. For each , let be the sigma-algebra generated by , which is just the smallest sigma-algebra on the underlying probability space with respect to which every is measurable. Then, independence of the collections is equivalent to independence of these sigma-algebras, so that

(1) |

for all finite pairwise distinct sequences and .

We could prove theorem 6 directly by using the characteristic function for multivariate normals along similar lines to 1, although it can get a little messy. I will take an alternative approach and, instead, use theorem 1 proven above to imply the much more general result. This does not require making use of any further properties of joint normals beyond the basic statement that linear combinations of them remain joint normal. To make the leap from theorem 1 to 6, I use the following statement which applies to arbitrary collections of (real-valued) random variables. For any such collection , then denotes the collection of finite linear combinations from .

Lemma 7Letbe a collection of collections of random variables. The following are equivalent,

are independent over.for any finite sequenceand, then each pairandare independent.

*Proof:* The second statement immediately follows from the first, by the definitions, so I just concentrate on the proof that the second statement implies the first. Recall from the definition of independence, we need to show that (1) holds for any finite pairwise distinct sequence . We use induction on *n*, so suppose that the result holds for , and set and . We just need to show that

(2) |

since, by the induction hypothesis, (1) follows immediately from this. We show that (2) holds for all *A* and *B* in the sigma-algebras generated by and respectively. Letting and be the union of the sigma-algebras generated by each finite subset of, respectively, and , these are pi-systems and, by the pi-system lemma, it is sufficient to prove the result for and . Hence, we suppose that *A* and *B* are, respectively, in the sigma-algebras of finite sub-collections

The characteristic function of the random vector is computed by

for any and , where and . Here, independence of and was used, and is guaranteed by the second statement of the lemma. Since the distribution of a random vector is uniquely determined by its characteristic function, this shows that *Y* and *Z* are independent. Hence, *A* and *B* are independent, giving (1) as required. ⬜

Finally, I use this to give a proof of the main independence result.

*Proof of theorem 6:* Choose a finite sequence and . Setting , by lemma 7 we just need to show that every and are independent. By assumption, we know that every and have zero covariance. By linearity of the covariance, this immediately extends to all and which, by theorem 1, are therefore independent, as required. ⬜