Books / Probability and Statistical Inference

Random Variables of the Continuous Type

Let $𝑋$ denote the outcome when a point is selected at random from an interval $[𝑎,𝑏]$, $-\infty <𝑎<𝑏<\infty$. If the experiment is performed in a fair manner, it is reasonable to assume that the probability that the point is selected from the interval $[𝑎,𝑥]$, $𝑎\leq 𝑥\leq 𝑏$, is $\frac{𝑥-𝑎}{𝑏-𝑎}$. That is, the probability is proportional to the length of the interval. So, the CDF (cumulative distribution function), with the same meaning borrowed from the case for discrete random variables, of $𝑋$ is

Note that $𝐹(𝑥)$ can be written as

where

and $𝑓$ assumes the value 0 outside the interval $[𝑎,𝑏]$. On $(𝑎,𝑏)$ we have ${𝐹}^{\prime }(𝑥)=𝑓(𝑥)$, and $𝑓$ is called the probability density function (PDF) of $𝑋$. The random variable $𝑋$ like this is said to have a uniform distribution if its probability density function is constant on an interval.

For a continuous random variable $𝑋$, it is not natural to assign probability to the event $\{𝑋=𝑥\}$ for an isolated point $𝑥\in 𝑆$ in the same way as in the discrete case. As an analogy, consider a rod of total mass $1$: one does not speak of the mass at a single point; instead one works with density (mass per unit length). Probability for continuous outcomes is analogous, which motivates the name probability density function.

We say a probability density function (PDF) of a random variable $𝑋$ of the continuous type, with space $𝑆$ that is an interval or union of intervals, is an integrable function $𝑓(𝑥)$ on $𝑆$ satisfying the following conditions:

1. $𝑓(𝑥)\geq 0$ for all $𝑥\in 𝑆$.
2. ${\int }_{𝑆}𝑓(𝑥)\, \mathrm{d}𝑥=1$.

3. If $(𝑎,𝑏)\subseteq 𝑆$, then the probability of the event $\{𝑎<𝑋<𝑏\}$ is

   $$ℙ(𝑎<𝑋<𝑏)={\int }_{𝑎}^{𝑏}𝑓(𝑥)\, \mathrm{d}𝑥.$$

The cumulative distribution function (CDF) or distribution function of $𝑋$ defined in terms of the PDF is then given by

The expected value of $𝑋$, or the mean of $𝑋$, is defined as

the variance of $𝑋$ is

and the moment-generating function of $𝑋$ is

This is not the first place the moment-generating function appears: we already introduced $𝑀$ for discrete random variables and related it to moments there. That earlier discussion will be completed when we typeset the discrete chapter; here we only spell out the integral form for the continuous case and recall the same hypotheses.

The algebraic identity ${\sigma }^{2}=\mathbb{E}[{𝑋}^{2}]-{\mu }^{2}$ holds whenever the second moment is finite: if $\mathbb{E}[{𝑋}^{2}]<\infty$, then $\mathbb{E}[𝑋]$ and $\operatorname{Var}(𝑋)$ are finite and the formula follows.

Suppose $𝑀(𝑡)=\mathbb{E}[{𝑒}^{𝑡𝑋}]$ is finite for all $𝑡$ in some open interval $(-h,h)$ with $h>0$. Then $𝑀$ is infinitely differentiable on $(-h,h)$, and for every nonnegative integer $𝑟$,

Casella and Berger Casella and Berger (2002) give a standard statement and proof of this identity. In particular, $\mu ={𝑀}^{\prime }(0)$ and ${\sigma }^{2}={𝑀}^{\prime\prime }(0)-{[{𝑀}^{\prime }(0)]}^{2}$. Under these hypotheses, the conclusions agree with those already recorded for discrete random variables.

The CDF also answers percentile (or quantile) questions. Fix $𝑝\in (0,1)$. A value ${\pi }_{𝑝}$ is called a 100p-th percentile of the distribution of $𝑋$ if

Equivalently, ${\pi }_{𝑝}$ is a p-quantile of $𝑋$. The median is the 50th percentile, i.e. any $𝑚$ with $𝐹(𝑚)=1/2$; when the solution is unique, $𝑚={\pi }_{1/2}$. When $𝐹$ is continuous and strictly increasing on an interval, there is a unique ${\pi }_{𝑝}$ for each $𝑝$, and the map $𝑝\mapsto {\pi }_{𝑝}$ coincides with the inverse function ${𝐹}^{-1}$. More generally, when $𝐹$ has flat pieces, one uses the generalized inverse ${𝐹}^{-1}(𝑝)=\inf \{𝑥:𝐹(𝑥)\geq 𝑝\}$ to define the quantile in a unique way; we will not need that refinement here.