2.1 Probability Spaces and Random Elements

2.1.1 Recap: Measure theory

We start from some basic concepts in the classical measure theory and use them to describe the probability space.

Definition 2.1 ( \(\sigma\)-algebra) Let \(\cal F\) be a collection of subsets of a sample space \(\Omega\). \(\cal F\) is called a \(\sigma\)-algebra or \(\sigma\)-field on \(\Omega\) if

  1. The empty set \(\phi \in \cal F\).
  2. If \(A \in \cal F\), then \(A^c \in \cal F\).
  3. If \(A_i \in \cal F\) for \(i=1,2,\cdots\), then \(\cup_{i=1}^{\infty} A_i \in \cal F\).

We denote the smallest sigma algebra containing a collection \(C\) as \(\sigma(C)\). In particular, we denote \(\cal B= \sigma(C)\) the \(\sigma\)-field (called Borel field) where \(C\) denotes all finite open interval on \(\mathbb{R}\). By Linderberg’s covering lemma, we can see \(\cal B\) is also the smallest \(\sigma\)-field containing the collection of all open sets on \(\mathbb{R}\). Furthermore, we denote \(\cal B^k\) as the Borel field in \(\mathbb{R}^k\). Define the pair \((\Omega, \cal F)\) a measurable space if \(\cal F\) is a \(\sigma\)-field on \(\Omega\).

Definition 2.2 (Measure) A set function \(\nu(\cdot)\) defined on a \(\sigma\)-field \(\cal F\) is a measure if

  1. \(0 \leq \nu(A) \leq \infty\) for any \(A \in \cal F\).
  2. \(\nu(\phi)=0\)
  3. If \(A_i \in \cal F\), \(i=1,2,\cdots\) and \(A_i\)’s are disjoint for any \(i\neq j\), then \[\nu(\cup_i A_i)=\sum_i \nu(A_i)\]

The triple \((\Omega, \cal F, \nu)\) is called a measure space and it is called a probability space if \(\nu(\Omega)=1\).

Proposition 2.1 Let \((\Omega,\cal F,\nu)\) be a measure space.

  1. If \(A \subset B \in \cal F\), then \(\nu(A)\leq \nu (B)\).
  2. For any sequence \(A_1,A_2,\cdots \in \cal F\), \[\nu(\cup_i A_i)\leq \sum_i \nu(A_i).\]
  3. If \(A_1 \subset A_2 \subset \cdots \in \cal F\) (or \(A_1 \supset A_2 \supset \cdots\) and \(\nu(A_k)<\infty\) for some \(k\in \mathbb{N}\)), then \[\nu(\lim_{k \to \infty}A_k)=\lim_{k\to \infty}\nu(A_k).\]

Below we see some uniqueness theorem about the measure, or the result of the well known Dinkin’s \(\pi\)-\(\lambda\) theorem.

Definition 2.3 A collection \(C\) of some subsets of \(\Omega\) is a \(\pi\)-system if it is closed under intersection.

Definition 2.4 Let \(C\) be a collection of some subsets of \(\Omega\). A measure \(\mu\) is \(\sigma\)-finite on \(C\) if there exist a sequence of sets \(\{A_k\}\) in \(C\) such that \(\Omega=\cup_k A_k\) and \(\mu(A_K)<\infty\) for all \(k\).

Lebesgue measure is an example of \(\sigma\)-finite measure on \(\cal B\). Since probability measure is always finite, so it is \(\sigma\)-finite.

Theorem 2.1 (Theorem 10.3 in Billingsley, 1986) Suppose that \(\mu_1\) and \(\mu_2\) are measures on \(\sigma(\cal P)\), where \(\cal P\) is a \(\pi\)-system. Suppose they are both \(\sigma\)-finite and agree on \(\cal P\), then they also agree on \(\sigma(\cal P)\).

Example 2.1 Here are some examples using the uniqueness theorem:

  1. If \(\mu\) is a measure defined on \(\cal B\) and \(\mu([a,b])=b-a\), then by checking that \(\cal P:=\{[a,b]\,|-\infty<a<b<\infty\}\) is a \(\pi\)- system and \(\cal B=\sigma(\cal P)\) we can show that \(\mu\) agrees with the Lebesgue measure on \(\cal B\).

  2. (Billingsley, pp.225-236, 1986) Given measurable spaces \((\Omega_i, \cal F_i,\nu_i)_{i=1}^k\) with \(\sigma\)- finite measures, there exists a unique \(\sigma\)- finite measure on product \(\sigma\)- field \(\sigma(\prod_\limits{i=1}^k \cal F_i)\), which is known as product measure, defined by \[\nu_1\times\cdots\times\nu_k\,(A_1\times\cdots\times A_k):=\prod_\limits{i=1}^k \nu_i(A_i)\] for all \(A_i \in \cal F_i,\,i=1,\cdots,k\).

2.1.2 Measurable Functions

Definition 2.5 (Measurable functions) Let \(f\) be a function from \((\Omega,\cal F)\) to \((\Lambda, \cal G)\) (both are measurable spaces). Then \(f\) is called a measurable function if \(f^{-1}(\cal G) \subset \cal F\), where \(f^{-1}(\cal G):=\{f^{-1}(A)\,|\, A\in \cal G\}.\)

In the other words, preimage of any set in \(\cal G\) of a measurable fucntion also lies in \(\cal F\). If \(\Lambda=\mathbb{R}\) and \(\cal G=\cal B\), \(f\) is called a Borel function. Furthermore, a Borel function defined on a probability space is called a random Variable (a random vector with respect to (\(\mathbb{R}^k,\cal B^k\))).

It is worthy noting that we may not require \(\cal F\) to be the \(\sigma\)- field on which \(f\) is measurable. In particular, it is easy to prove that \(f^{-1}(\cal G)\) is also a \(\sigma\)- field. This “smaller \(\sigma\)- field is called \(\sigma\)- field induced by \(f\) and we denote it by \(\sigma(f)\).

We omit the technical proof of the measurability of a function under fundamental operation such as sup, inf, limsup, liminf. Indicator functions, continuous functions, composition of measurable functions are also measurable functions. Readers can refer to the textbook for the proof.

Proposition 2.2 (Approximation property) Suppose that f is measurable from \((\Omega,\cal F)\) to \((\bar{\mathbb{R}}, \bar{\cal B})\), where \(\bar{\mathbb{R}}\) is the extended real line with respect to \(\bar{\cal B}\). First assume \(f \geq 0\), then there exists a positive and monotone sequence of simple functions \(\{f_n\}\) such that \(f_n \to f\). For general measurable function \(f\), consider \(f=f^{+}-f^{-}\) for general function \(f\) and similar result goes.

Below we show a lemma which let us connect two measurable function.

Lemma 2.1 (Theorem A.42 in Schervish, 1995) Let \(Y\) be measurable from \((\Omega,\cal F)\) to \((\Lambda_Y,\cal G_Y)\) and \(Z\) be measurable from \((\Omega,\cal F)\) to \((\Lambda_Z,\cal G_Z)\). Define \(T=\{Y(\omega),\omega \in \Omega\} \subset \Lambda_Y\). Then \(Z\) is measurable from \((\Omega,\sigma(Y))\) to \((\Lambda_Z,\cal G_Z)\) if and only if \(Z=h \circ Y\) for some \(h\) that is measurable from \((T,T\cap \cal G_Y)\) to \((\Lambda_Z,\cal G_Z)\).

Remark: Proof of the “if” side is relatively obvious. (WLOG, we can assume \(\Lambda_Y\) is the range of \(Y\).)

Proof (only if). At first, we show that for \(\omega_1,\ \omega_2 \in \Omega\), \(Y(\omega_1)=Y(\omega_2)\) implies \(Z(\omega_1)=Z(\omega_2)\). Suppose that \(Y(\omega_1)=Y(\omega_1)=a\). Since \(Z\) is measurable with respect to \(\sigma(Y)\), there exist \(A \in \cal G_Y\) such that \(Y^{-1}(A) =Z^{-1}(\{Z(\omega_1)\})\). Clearly \(\omega_1,\ \omega_2 \in Z^{-1}(\{Z(\omega_1)\})\), thus \(Z(\omega_1)=Z(\omega_2)\). By this first step, the function \(h\) with \(h(Y(\omega))=Z(\omega)\) is well defined with domain being the range of \(Y\).

Secondly, we prove such \(h\) is measurable with respect to \(T \,\cap\, \cal G_Y\). Given \(B \in \cal G_Z\), let \(A\) be an event in \(\cal G_Y\) such that \(Y^{-1}(A)=Z^{-1}(B)\) which exists since \(Z\) is measurable with respect to \(\sigma(Y)\), then \(h\) is measurable if \(h^{-1}(B)=A\,\cap\,T\), where \(T\) is the range of \(Y\).

(\(h^{-1}(B)\subset A\,\cap\,T\)). Given \(a \in h^{-1}(B)\), then \(h(a) \in B\) and \(a=Y(\omega)\) for some \(\omega\in \Omega\) (\(a \in T\)) by the domain of \(h\), and thus \(Z(\omega)=h(Y(\omega)) \in B\), \(\omega \in Z^{-1}(B)=Y^{-1}(A)\) implies \(a \in A\).

(\(A\,\cap\,T\subset h^{-1}(B)\)). Given \(a \in A \cap T\), \(a=Y(\omega) \in A\) for some \(\omega \in \Omega\), then \(\omega \in Y^{-1}(A)=Z^{-1}(B)\) and by definition of \(h\), \(h(a)=Z(\omega)\in B\), thus \(a \in h^{-1}(B)\).