Markov Processes

In these notes, the approach taken to stochastic calculus revolves around stochastic integration and the theory of semimartingales. An alternative starting point would be to consider Markov processes. Although I do not take the second approach, all of the special processes considered in the current section are Markov, so it seems like a good idea to introduce the basic definitions and properties now. In fact, all of the special processes considered (Brownian motion, Poisson processes, Lévy processes, Bessel processes) satisfy the much stronger property of being Feller processes, which I will define in the next post.

Intuitively speaking, a process X is Markov if, given its whole past up until some time s, the future behaviour depends only its state at time s. To make this precise, let us suppose that X takes values in a measurable space ${(E,\mathcal{E})}$ and, to denote the past, let ${\mathcal{F}_t}$ be the sigma-algebra generated by ${\{X_s\colon s\le t\}}$ . The Markov property then says that, for any times ${s\le t}$ and bounded measurable function ${f\colon E\rightarrow{\mathbb R}}$ , the expected value of ${f(X_t)}$ conditional on ${\mathcal{F}_s}$ is a function of ${X_s}$ . Equivalently,

$\displaystyle {\mathbb E}\left[f(X_t)\mid\mathcal{F}_s\right]={\mathbb E}\left[f(X_t)\mid X_s\right]$

(1)

(almost surely). More generally, this idea makes sense with respect to any filtered probability space ${\mathbb{F}=(\Omega,\mathcal{F},\{\mathcal{F}_t\}_{t\ge 0},{\mathbb P})}$ . A process X is Markov with respect to ${\mathbb{F}}$ if it is adapted and (1) holds for times ${s\le t}$ . Continue reading “Markov Processes” →

Poisson Processes

A Poisson process is a continuous-time stochastic process which counts the arrival of randomly occurring events. Commonly cited examples which can be modeled by a Poisson process include radioactive decay of atoms and telephone calls arriving at an exchange, in which the number of events occurring in each consecutive time interval are assumed to be independent. Being piecewise constant, Poisson processes have very simple pathwise properties. However, they are very important to the study of stochastic calculus and, together with Brownian motion, forms one of the building blocks for the much more general class of Lévy processes. I will describe some of their properties in this post.

A random variable N has the Poisson distribution with parameter ${\lambda}$ , denoted by ${N\sim{\rm Po}(\lambda)}$ , if it takes values in the set of nonnegative integers and

$\displaystyle {\mathbb P}(N=n)=\frac{\lambda^n}{n!}e^{-\lambda}$

(1)

for each ${n\in{\mathbb Z}_+}$ . The mean and variance of N are both equal to ${\lambda}$ , and the moment generating function can be calculated,

$\displaystyle {\mathbb E}\left[e^{aN}\right] = \exp\left(\lambda(e^a-1)\right),$

which is valid for all ${a\in{\mathbb C}}$ . From this, it can be seen that the sum of independent Poisson random variables with parameters ${\lambda}$ and ${\mu}$ is again Poisson with parameter ${\lambda+\mu}$ . The Poisson distribution occurs as a limit of binomial distributions. The binomial distribution with success probability p and m trials, denoted by ${{\rm Bin}(m,p)}$ , is the sum of m independent ${\{0,1\}}$ -valued random variables each with probability p of being 1. Explicitly, if ${N\sim{\rm Bin}(m,p)}$ then

$\displaystyle {\mathbb P}(N=n)=\frac{m!}{n!(m-n)!}p^n(1-p)^{m-n}.$

In the limit as ${m\rightarrow\infty}$ and ${p\rightarrow 0}$ such that ${mp\rightarrow\lambda}$ , it can be verified that this tends to the Poisson distribution (1) with parameter ${\lambda}$ .

Poisson processes are then defined as processes with independent increments and Poisson distributed marginals, as follows.

Definition 1 A Poisson process X of rate ${\lambda\ge0}$ is a cadlag process with ${X_0=0}$ and ${X_t-X_s\sim{\rm Po}(\lambda(t-s))}$ independently of ${\{X_u\colon u\le s\}}$ for all ${s\le t}$ .

An immediate consequence of this definition is that, if X and Y are independent Poisson processes of rates ${\lambda}$ and ${\mu}$ respectively, then their sum ${X+Y}$ is also Poisson with rate ${\lambda+\mu}$ . Continue reading “Poisson Processes” →

Continuous Processes with Independent Increments

A stochastic process X is said to have independent increments if ${X_t-X_s}$ is independent of ${\{X_u\}_{u\le s}}$ for all ${s\le t}$ . For example, standard Brownian motion is a continuous process with independent increments. Brownian motion also has stationary increments, meaning that the distribution of ${X_{t+s}-X_t}$ does not depend on t. In fact, as I will show in this post, up to a scaling factor and linear drift term, Brownian motion is the only such process. That is, any continuous real-valued process X with stationary independent increments can be written as

$\displaystyle X_t = X_0 + b t + \sigma B_t$

(1)

for a Brownian motion B and constants ${b,\sigma}$ . This is not so surprising in light of the central limit theorem. The increment of a process across an interval [s,t] can be viewed as the sum of its increments over a large number of small time intervals partitioning [s,t]. If these terms are independent with relatively small variance, then the central limit theorem does suggest that their sum should be normally distributed. Together with the previous posts on Lévy’s characterization and stochastic time changes, this provides yet more justification for the ubiquitous position of Brownian motion in the theory of continuous-time processes. Consider, for example, stochastic differential equations such as the Langevin equation. The natural requirements for the stochastic driving term in such equations is that they be continuous with stationary independent increments and, therefore, can be written in terms of Brownian motion.

The definition of standard Brownian motion extends naturally to multidimensional processes and general covariance matrices. A standard d-dimensional Brownian motion ${B=(B^1,\ldots,B^d)}$ is a continuous process with stationary independent increments such that ${B_t}$ has the ${N(0,tI)}$ distribution for all ${t\ge 0}$ . That is, ${B_t}$ is joint normal with zero mean and covariance matrix tI. From this definition, ${B_t-B_s}$ has the ${N(0,(t-s)I)}$ distribution independently of ${\{B_u\colon u\le s\}}$ for all ${s\le t}$ . This definition can be further generalized. Given any ${b\in{\mathbb R}^d}$ and positive semidefinite ${\Sigma\in{\mathbb R}^{d^2}}$ , we can consider a d-dimensional process X with continuous paths and stationary independent increments such that ${X_t}$ has the ${N(tb,t\Sigma)}$ distribution for all ${t\ge 0}$ . Here, ${b}$ is the drift of the process and ${\Sigma}$ is the `instantaneous covariance matrix’. Such processes are sometimes referred to as ${(b,\Sigma)}$ -Brownian motions, and all continuous d-dimensional processes starting from zero and with stationary independent increments are of this form.

Theorem 1 Let X be a continuous ${{\mathbb R}^d}$ -valued process with stationary independent increments.

Then, there exist unique ${b\in{\mathbb R}^d}$ and ${\Sigma\in{\mathbb R}^{d^2}}$ such that ${X_t-X_0}$ is a ${(b,\Sigma)}$ -Brownian motion.

Continue reading “Continuous Processes with Independent Increments” →

The Martingale Representation Theorem

The martingale representation theorem states that any martingale adapted with respect to a Brownian motion can be expressed as a stochastic integral with respect to the same Brownian motion.

Theorem 1 Let B be a standard Brownian motion defined on a probability space ${(\Omega,\mathcal{F},{\mathbb P})}$ and ${\{\mathcal{F}_t\}_{t\ge 0}}$ be its natural filtration.

Then, every ${\{\mathcal{F}_t\}}$ –local martingale M can be written as

$\displaystyle M = M_0+\int\xi\,dB$

for a predictable, B-integrable, process ${\xi}$ .

As stochastic integration preserves the local martingale property for continuous processes, this result characterizes the space of all local martingales starting from 0 defined with respect to the filtration generated by a Brownian motion as being precisely the set of stochastic integrals with respect to that Brownian motion. Equivalently, Brownian motion has the predictable representation property. This result is often used in mathematical finance as the statement that the Black-Scholes model is complete. That is, any contingent claim can be exactly replicated by trading in the underlying stock. This does involve some rather large and somewhat unrealistic assumptions on the behaviour of financial markets and ability to trade continuously without incurring additional costs. However, in this post, I will be concerned only with the mathematical statement and proof of the representation theorem.

In more generality, the martingale representation theorem can be stated for a d-dimensional Brownian motion as follows.

Theorem 2 Let ${B=(B^1,\ldots,B^d)}$ be a d-dimensional Brownian motion defined on the filtered probability space ${(\Omega,\mathcal{F},\{\mathcal{F}_t\}_{t\ge 0},{\mathbb P})}$ , and suppose that ${\{\mathcal{F}_t\}}$ is the natural filtration generated by B and ${\mathcal{F}_0}$ .

$\displaystyle \mathcal{F}_t=\sigma\left(\{B_s\colon s\le t\}\cup\mathcal{F}_0\right)$

Then, every ${\{\mathcal{F}_t\}}$ -local martingale M can be expressed as

$\displaystyle M=M_0+\sum_{i=1}^d\int\xi^i\,dB^i$ (1)

for predictable processes ${\xi^i}$ satisfying ${\int_0^t(\xi^i_s)^2\,ds<\infty}$ , almost surely, for each ${t\ge0}$ .

Continue reading “The Martingale Representation Theorem” →

SDEs Under Changes of Time and Measure

The previous two posts described the behaviour of standard Brownian motion under stochastic changes of time and equivalent changes of measure. I now demonstrate some applications of these ideas to the study of stochastic differential equations (SDEs). Surprisingly strong results can be obtained and, in many cases, it is possible to prove existence and uniqueness of solutions to SDEs without imposing any continuity constraints on the coefficients. This is in contrast to most standard existence and uniqueness results for both ordinary and stochastic differential equations, where conditions such as Lipschitz continuity is required. For example, consider the following SDE for measurable coefficients ${a,b\colon{\mathbb R}\rightarrow{\mathbb R}}$ and a Brownian motion B

$\displaystyle dX_t=a(X_t)\,dB_t+b(X_t)\,dt.$

(1)

If a is nonzero, ${a^{-2}}$ is locally integrable and b/a is bounded then we can show that this has weak solutions satisfying uniqueness in law for any specified initial distribution of X. The idea is to start with X being a standard Brownian motion and apply a change of time to obtain a solution to (1) in the case where the drift term b is zero. Then, a Girsanov transformation can be used to change to a measure under which X satisfies the SDE for nonzero drift b. As these steps are invertible, every solution can be obtained from a Brownian motion in this way, which uniquely determines the distribution of X.

A standard example demonstrating the concept of weak solutions and uniqueness in law is provided by Tanaka’s SDE

$\displaystyle dX_t={\rm sgn}(X_t)\,dB_t$

(2)

Continue reading “SDEs Under Changes of Time and Measure” →

Girsanov Transformations

Girsanov transformations describe how Brownian motion and, more generally, local martingales behave under changes of the underlying probability measure. Let us start with a much simpler identity applying to normal random variables. Suppose that X and ${Y=(Y^1,\ldots,Y^n)}$ are jointly normal random variables defined on a probability space ${(\Omega,\mathcal{F},{\mathbb P})}$ . Then ${U\equiv\exp(X-\frac{1}{2}{\rm Var}(X)-{\mathbb E}[X])}$ is a positive random variable with expectation 1, and a new measure ${{\mathbb Q}=U\cdot{\mathbb P}}$ can be defined by ${{\mathbb Q}(A)={\mathbb E}[1_AU]}$ for all sets ${A\in\mathcal{F}}$ . Writing ${{\mathbb E}_{\mathbb Q}}$ for expectation under the new measure, then ${{\mathbb E}_{\mathbb Q}[Z]={\mathbb E}[UZ]}$ for all bounded random variables Z. The expectation of a bounded measurable function ${f\colon{\mathbb R}^n\rightarrow{\mathbb R}}$ of Y under the new measure is

$\displaystyle {\mathbb E}_{\mathbb Q}\left[f(Y)\right]={\mathbb E}\left[f\left(Y+{\rm Cov}(X,Y)\right)\right],$

(1)

where ${{\rm Cov}(X,Y)}$ is the covariance. This is a vector whose i’th component is the covariance ${{\rm Cov}(X,Y^i)}$ . So, Y has the same distribution under ${{\mathbb Q}}$ as ${Y+{\rm Cov}(X,Y)}$ has under ${{\mathbb P}}$ . That is, when changing to the new measure, Y remains jointly normal with the same covariance matrix, but its mean increases by ${{\rm Cov}(X,Y)}$ . Equation (1) follows from a straightforward calculation of the characteristic function of Y with respect to both ${{\mathbb P}}$ and ${{\mathbb Q}}$ .

Now consider a standard Brownian motion B and fix a time ${T>0}$ and a constant ${\mu}$ . Then, for all times ${t\ge 0}$ , the covariance of ${B_t}$ and ${B_T}$ is ${{\rm Cov}(B_t,B_T)=t\wedge T}$ . Applying (1) to the measure ${{\mathbb Q}=\exp(\mu B_T-\mu^2T/2)\cdot{\mathbb P}}$ shows that

$\displaystyle B_t=\tilde B_t + \mu (t\wedge T)$

where ${\tilde B}$ is a standard Brownian motion under ${{\mathbb Q}}$ . Under the new measure, B has gained a constant drift of ${\mu}$ over the interval ${[0,T]}$ . Such transformations are widely applied in finance. For example, in the Black-Scholes model of option pricing it is common to work under a risk-neutral measure, which transforms the drift of a financial asset to be the risk-free rate of return. Girsanov transformations extend this idea to much more general changes of measure, and to arbitrary local martingales. However, as shown below, the strongest results are obtained for Brownian motion which, under a change of measure, just gains a stochastic drift term. Continue reading “Girsanov Transformations” →

Time-Changed Brownian Motion

From the definition of standard Brownian motion B, given any positive constant c, ${B_{ct}-B_{cs}}$ will be normal with mean zero and variance c(t–s) for times ${t>s\ge 0}$ . So, scaling the time axis of Brownian motion B to get the new process ${B_{ct}}$ just results in another Brownian motion scaled by the factor ${\sqrt{c}}$ .

This idea is easily generalized. Consider a measurable function ${\xi\colon{\mathbb R}_+\rightarrow{\mathbb R}_+}$ and Brownian motion B on the filtered probability space ${(\Omega,\mathcal{F},\{\mathcal{F}_t\}_{t\ge 0},{\mathbb P})}$ . So, ${\xi}$ is a deterministic process, not depending on the underlying probability space ${\Omega}$ . If ${\theta(t)\equiv\int_0^t\xi^2_s\,ds}$ is finite for each ${t>0}$ then the stochastic integral ${X=\int\xi\,dB}$ exists. Furthermore, X will be a Gaussian process with independent increments. For piecewise constant integrands, this results from the fact that linear combinations of joint normal variables are themselves normal. The case for arbitrary deterministic integrands follows by taking limits. Also, the Ito isometry says that ${X_t-X_s}$ has variance

$\displaystyle \setlength\arraycolsep{2pt} \begin{array}{rl} \displaystyle{\mathbb E}\left[\left(\int_s^t\xi\,dB\right)^2\right]&\displaystyle={\mathbb E}\left[\int_s^t\xi^2_u\,du\right]\smallskip\\ &\displaystyle=\theta(t)-\theta(s)\smallskip\\ &\displaystyle={\mathbb E}\left[(B_{\theta(t)}-B_{\theta(s)})^2\right]. \end{array}$

So, ${\int\xi\,dB=\int\sqrt{\theta^\prime(t)}\,dB_t}$ has the same distribution as the time-changed Brownian motion ${B_{\theta(t)}}$ .

With the help of Lévy’s characterization, these ideas can be extended to more general, non-deterministic, integrands and to stochastic time-changes. In fact, doing this leads to the startling result that all continuous local martingales are just time-changed Brownian motion. Continue reading “Time-Changed Brownian Motion” →

Lévy’s Characterization of Brownian Motion

Standard Brownian motion, ${\{B_t\}_{t\ge 0}}$ , is defined to be a real-valued process satisfying the following properties.

${B_0=0}$ .
${B_t-B_s}$ is normally distributed with mean 0 and variance t–s independently of ${\{B_u\colon u\le s\}}$ , for any ${t>s\ge 0}$ .
B has continuous sample paths.

As always, it only really matters is that these properties hold almost surely. Now, to apply the techniques of stochastic calculus, it is assumed that there is an underlying filtered probability space ${(\Omega,\mathcal{F},\{\mathcal{F}_t\}_{t\ge 0},{\mathbb P})}$ , which necessitates a further definition; a process B is a Brownian motion on a filtered probability space ${(\Omega,\mathcal{F},\{\mathcal{F}_t\}_{t\ge 0},{\mathbb P})}$ if in addition to the above properties it is also adapted, so that ${B_t}$ is ${\mathcal{F}_t}$ -measurable, and ${B_t-B_s}$ is independent of ${\mathcal{F}_s}$ for each ${t>s\ge 0}$ . Note that the above condition that ${B_t-B_s}$ is independent of ${\{B_u\colon u\le s\}}$ is not explicitly required, as it also follows from the independence from ${\mathcal{F}_s}$ . According to these definitions, a process is a Brownian motion if and only if it is a Brownian motion with respect to its natural filtration.

The property that ${B_t-B_s}$ has zero mean independently of ${\mathcal{F}_s}$ means that Brownian motion is a martingale. Furthermore, we previously calculated its quadratic variation as ${[B]_t=t}$ . An incredibly useful result is that the converse statement holds. That is, Brownian motion is the only local martingale with this quadratic variation. This is known as Lévy’s characterization, and shows that Brownian motion is a particularly general stochastic process, justifying its ubiquitous influence on the study of continuous-time stochastic processes.

Theorem 1 (Lévy’s Characterization of Brownian Motion) Let X be a local martingale with ${X_0=0}$ . Then, the following are equivalent.

X is standard Brownian motion on the underlying filtered probability space.

X is continuous and ${X^2_t-t}$ is a local martingale.

X has quadratic variation ${[X]_t=t}$ .

Continue reading “Lévy’s Characterization of Brownian Motion” →

Special Processes

The point in these stochastic calculus notes has been reached where the theory of stochastic integration is sufficiently well developed to apply in a wide range of situations.

Results such as Ito’s lemma, properties of quadratic variations and existence and uniqueness of solutions to stochastic differential equations followed quite directly from the definition of stochastic integration. Then, once it was shown that integration with respect to martingales is well-defined, results such as preservation of the local martingale property and Ito’s isometry also followed without too much effort.

Over the next few posts, I will take a break from further development of the general theory. Instead, I look at certain special processes, applying the calculus developed so far and gaining a few examples to motivate further development of the theory.

This will include properties of Brownian motion, such as Lévy’s characterization, Girsanov transforms, stochastic time changes and martingale representation. Other important processes which I take a brief look at include Bessel processes, the Poisson process and the Cauchy process. We will also derive a general description of processes with independent increments, including the Lévy-Khintchine formula characterizing Lévy processes.

	Anonymous on Spitzer’s Formula
	Anonymous on Spitzer’s Formula
	Anonymous on Brownian Bridges
	Anonymous on Spitzer’s Formula
	Anonymous on Spitzer’s Formula
	Anonymous on Spitzer’s Formula
	Anonymous on Spitzer’s Formula
	Anonymous on Spitzer’s Formula
	Anonymous on Spitzer’s Formula
	Anonymous on Spitzer’s Formula