According to Kolmogorov’s axioms, to define a probability space we start with a set Ω and an event space consisting of a sigma-algebra F  on Ω. A probability measure ℙ on this gives the probability space (Ω, F , ℙ), on which we can define random variables as measurable maps from Ω to the reals or other measurable space.

However, it is common practice to suppress explicit mention of the underlying sample space Ω. The values of a random variable X: Ω → ℝ are simply written as X, rather than X(ω) for ω ∈ Ω. It is intuitively thought of as a real number which happens to be random, rather than a function. For one thing, we usually do not really care what the sample space is and, instead, only care about events and their probabilities, and about random variables and their expectations. This philosophy has some benefits. Frequently, when performing constructions, it can be useful to introduce new supplementary random variables to work with. It may be necessary to enlarge the sample space and add new events to the sigma-algebra to accommodate these. If the underlying space is not set in stone then this is straightforward to do, and we can continue to work with these new variables as if they were always there from the start.

Definition 1 An extension π of a probability space (Ω, F , ℙ) to a new space (Ω′, F ′, ℙ′),

is a probability preserving measurable map π: Ω′ → Ω. That is, ℙ′(π^-1E) = ℙ(E) for events E ∈ F .

By construction, events E ∈ F  pull back to events π^-1E ∈ F ′ with the same probabilities. Random variables X defined on (Ω, F , ℙ) lift to variables π^∗X with the same distribution defined on (Ω′, F ′, ℙ′), given by π^∗X(ω) ≡ X(π(ω)). I will use the notation X^∗ in place of π^∗X for brevity although, in applications, it is common to reuse the same symbol X and simply note that we are now working with respect to an enlarged the probability space if necessary.

The extension can be thought of in two steps. First, the enlargement of the sample space, π: Ω′ → Ω on which we induce the sigma algebra π^∗F  consisting of events π^-1E for E ∈ F , and the measure ℙ′(π^-1E) = ℙ(E). This is essentially a no-op, since events and random variables on the initial space are in one-to-one correspondence with those on the enlarged space (at least, up to zero probability events). Next, we enlarge the sigma-algebra to F ′ ⊇ π^∗F  and extend the measure ℙ′ to this. It is this second step which introduces new events and random variables.

Since we may want to extend a probability space more than a single time, I look at how these combine. Consider an extension π of the original probability space, and then a further extension ρ of this.

These can be combined into a single extension ϕ = π○ρ of the original space,

Lemma 2 The composition ϕ = π○ρ is itself an extension of the probability space.

Proof: As compositions of measurable maps are measurable, it is sufficient to check that ϕ preserves probabilities. This is straightforward,

for all E ∈ F . ⬜

So far, so simple. The main purpose of this post, however, is to look at the situation with two separate extensions of the same underlying space. Both of these will add in some additional source of randomness, and we would like to combine them into a single extension.

Separate probability spaces can be combined by the product measure, which is the measure on the product space for which the projections onto the original spaces preserves probability, and for which the sigma-algebras generated by these projections are independent. Recall that a pair of sigma-algebras F  and G  defined on a probability space are independent if, for any sets A ∈ F  and B ∈ G  then ℙ(A ∩ B) = ℙ(A)ℙ(B).

Combining extensions of probability spaces will, instead, make use of relative independence.

Definition 3 Let (Ω, F , ℙ) be a probability space. Two sub-sigma-algebras G , H  ⊆ F  are relatively independent over a third sigma-algebra K  ⊆ G  ∩ H  if

(1)

for all A ∈ G  and B ∈ H .

It can be shown that the following properties are each equivalent to this definition;

• 𝔼[XY|K ] = 𝔼[X|K ]𝔼[Y|K ] for all bounded G -measurable random variables X and H -measurable Y.
• 𝔼[X|G ] = 𝔼[X|K ] for all bounded H -measurable X.
• 𝔼[X|H ] = 𝔼[X|K ] for all bounded G -measurable X.

Once a probability measure is specified separately on G  and H  then its extension to the sigma-algebra generated by G  ∪ H , if it exists, is uniquely determined by relative independence. This is a consequence of the pi-system lemma, since (1) defines it on the events {A ∩ B: A ∈ G , B ∈ H }, which is a pi-system generating the same sigma-algebra.

Now consider two separate extensions π₁ and π₂ of the same underlying probability space,

As maps between sets, these can both be embedded into a single extension known as the pullback or fiber product. This is the set Ω′= Ω₁ ×_Ω Ω₂ defined by

Defining projection maps ρ_i: Ω′ → Ω_i by

results in a commutative square with ϕ ≡ π₁○ρ₁ = π₂○ρ₂,

In fact, Ω′ is exactly the cartesian product Ω₁ × Ω₂ restricted to the subset on which π₁○ρ₁ and π₂○ρ₂ agree.

This constructs an extension ϕ of the sample space containing π₁ and π₂ as sub-extensions. However, it still needs to be made into a probability space. Use the smallest sigma-algebra F ′ on Ω′ making ρ₁, ρ₂ into measurable maps, which is generated by ρ₁^∗F ¹ ∪ ρ₂^∗F ². The probability measure ℙ′ on (Ω′, F ′) is uniquely determined on each of the sub-sigma-algebras by the requirement that ρ_i preserve probabilities,

for i = 1, 2 and A ∈ F ⁱ. These necessarily agree on ϕ^∗F  ⊆ ρ₁^∗F ¹ ∩ ρ₂^∗F ²,

for A ∈ F . The natural way to extend ℙ′ to all of F ′ is to use relative independence over ϕ^∗F .

Definition 4 The relative product of the extensions π₁ and π₂ is the extension

with ϕ, Ω′, F ′ constructed as above, and ℙ′ is the unique probability measure for which the projections ρ₁, ρ₂ preserve probabilities, and for which ρ₁^∗F ¹ and ρ₂^∗F ² are relatively independent over ϕ^∗F .

Continue reading “Probability Space Extensions and Relative Products” →