Books Probability and Statistical Inference

Approximations for Discrete Distributions

The central limit theorem, Theorem 5.10, applies not only to continuous random variables but also to discrete ones.

As an example, consider $𝑌∼\operatorname{Bin}(𝑛,𝑝)$ . Recall that this distribution arises as the sum of $𝑛$ independent Bernoulli trials: if ${𝑋}_{𝑖}∼\operatorname{Ber}(𝑝)$ are $i.i.d.$ and $𝑌={\sum }_{𝑖=1}^{𝑛}{𝑋}_{𝑖}$ , then $𝑌∼\operatorname{Bin}(𝑛,𝑝)$ . For large $𝑛$ , the central limit theorem suggests approximating $𝑌$ by $𝑊∼𝒩(𝑛𝑝,𝑛𝑝(1-𝑝))$ . Because $𝑌$ is discrete and $𝑊$ is continuous, we need one more convention before replacing binomial probabilities by normal ones.

For each $𝑘$ in the support of $𝑌$ , namely $\{0,1,\ldots ,𝑛\}$ , draw a rectangle of height ${𝑓}_{𝑌}(𝑘)$ and width $1$ centered at $𝑘$ . Equivalently, define a histogram density $h$ by

h(𝑦)={𝑓}_{𝑌}(𝑘)\quad \forall 𝑘-1/2\leq 𝑦<𝑘+1/2.

This probability histogram represents the binomial masses as a continuous density on the real line. The point probability $ℙ(𝑌=𝑘)$ is the area of the rectangle centered at $𝑘$ , which we approximate by

ℙ(𝑌=𝑘)\approx ℙ(𝑘-1/2\leq 𝑊<𝑘+1/2).

More generally,

ℙ(𝑘\leq 𝑌\leq 𝑚)\approx ℙ(𝑘-1/2\leq 𝑊<𝑚+1/2).

The half-unit extension on each side is called the continuity correction.

Figure 5.11 Probability histogram for $𝑌∼\operatorname{Bin}(10,1/2)$ with normal approximation $𝒩(5,5/2)$ : each unit-width rectangle has area $ℙ(𝑌=𝑘)$ ; the shaded band shows the continuity correction $ℙ(𝑘-1/2\leq 𝑊<𝑘+1/2)$ at $𝑘=5$ .

Figure 5.11 overlays the histogram with the approximating normal density. The dashed curve follows the binomial masses closely, and replacing the area of one rectangle by the area under the normal curve over the same interval is the idea behind the continuity correction.