Books / Probability and Statistical Inference

Bivariate Distributions of the Discrete Type

Example 4.1

Roll a pair of fair dice. Let $𝑋$ be the integer of the smaller number and $𝑌$ be the integer of the larger number. The outcome is $(𝑥,𝑦)$ where $𝑥,𝑦\in \{1,2,3,4,5,6\}$ and $𝑥\leq 𝑦$. When $𝑥<𝑦$, there are $2$ possibilities: either the result of the first die is $𝑥$ and the result of the other one is $𝑦$, or the other way around. Thus, we have

$$𝑓(𝑥,𝑦)=\frac{2}{36},\quad 𝑥<𝑦.$$

When $𝑥=𝑦$, there is only $1$ possibility: the result of the two dice is $𝑥$ (or $𝑦$). We have

$$𝑓(𝑥,𝑥)=\frac{1}{36},\quad 𝑥=𝑦.$$

Therefore, the joint probability mass function is given by

$$𝑓(𝑥,𝑦)=\begin{cases}\frac{2}{36},\quad & 𝑥<𝑦 \\ \frac{1}{36},\quad & 𝑥=𝑦\end{cases}$$

One can check that summing $𝑓$ over $𝑥$ and $𝑦$ gives $1$.

Figure 4.1 shows the joint PMF for this example in a form typical of discrete bivariate mass functions: support points, joint mass beside each marker, and marginal masses at the axis ticks.

Note the numbers in the margins of Figure 4.1. Each bottom margin entry sums the joint masses in that $𝑥$-column over $𝑦$; each left margin entry sums those in that $𝑦$-row over $𝑥$. Intuitively, these are the PMF values of $𝑋$ and $𝑌$ alone, respectively.

Let ${𝑋}_{1}$ and ${𝑋}_{2}$ be random variables of the discrete type with the joint PMF $𝑓({𝑥}_{1},{𝑥}_{2})$ on space $𝑆$. Let $𝑢({𝑋}_{1},{𝑋}_{2})$ be a function of ${𝑋}_{1}$ and ${𝑋}_{2}$. Then

if it exists, is called the mathematical expectation of $𝑢({𝑋}_{1},{𝑋}_{2})$.

Specially:

1. If $𝑢({𝑋}_{1},{𝑋}_{2})={𝑋}_{𝑘}$, then

1. If $𝑢({𝑋}_{1},{𝑋}_{2})={({𝑋}_{𝑘}-{\mu }_{𝑘})}^{2}$, then

We now extend the concept of binomial distribution to the trinomial distribution. Different from the Bernoulli experiment that only has two possible outcomes, the trinomial experiment has three, say ${𝐶}_{1}$, ${𝐶}_{2}$, and ${𝐶}_{3}$. Think of this as drawing a ball from an urn with $3$ different colors, say red, blue, and green. Let ${𝑝}_{𝑋}$, ${𝑝}_{𝑌}$, and ${𝑝}_{𝑍}$ be the probabilities of getting ${𝐶}_{1}$, ${𝐶}_{2}$, and ${𝐶}_{3}$, respectively. Of course, we have ${𝑝}_{𝑋}+{𝑝}_{𝑌}+{𝑝}_{𝑍}=1$. Now, the experiment is repeated $𝑛$ times and we are interested in the number of times we get each color. Let $𝑋=\#{𝐶}_{1}$, $𝑌=\#{𝐶}_{2}$, and $𝑍=\#{𝐶}_{3}=𝑛-𝑋-𝑌$. We want to find the probability mass function $𝑓(𝑥,𝑦)$ of $𝑋$ and $𝑌$, i.e., $ℙ(𝑋=𝑥,𝑌=𝑦)$.

Fix nonnegative integers $𝑥$ and $𝑦$ with $𝑥+𝑦\leq 𝑛$, and suppose that among the $𝑛$ trials there are $𝑥$ outcomes ${𝐶}_{1}$, $𝑦$ outcomes ${𝐶}_{2}$, and $𝑛-𝑥-𝑦$ outcomes ${𝐶}_{3}$. To evaluate $ℙ(𝑋=𝑥,𝑌=𝑦)$ we must count how many length-$𝑛$ sequences contain exactly those counts; call this number $𝑁$.

Think of the $𝑛$ trials as distinguishable for the moment (label the trials $1,2,\ldots ,𝑛$). Then there are $𝑛!$ ways to order the $𝑛$ labels. Among the $𝑛!$ orders of labels, fix one outcome pattern with counts $(𝑥,𝑦,𝑛-𝑥-𝑦)$ (which positions show ${𝐶}_{1}$, ${𝐶}_{2}$, or ${𝐶}_{3}$). Exactly $𝑥!𝑦!(𝑛-𝑥-𝑦)!$ label orders induce that same pattern: we may permute labels among the $𝑥$ trials labeled ${𝐶}_{1}$ in $𝑥!$ ways, among the $𝑦$ trials labeled ${𝐶}_{2}$ in $𝑦!$ ways, and among the $𝑛-𝑥-𝑦$ trials labeled ${𝐶}_{3}$ in $(𝑛-𝑥-𝑦)!$ ways, independently. So each such pattern accounts for $𝑥!𝑦!(𝑛-𝑥-𝑦)!$ of the $𝑛!$ orders, and there are $𝑁$ distinct patterns. Hence

so

This quotient is the multinomial coefficient with parameters $(𝑛;𝑥,𝑦,𝑛-𝑥-𝑦)$; it is often written in stacked form as

Therefore the PMF of $𝑋$ and $𝑌$ can be written as

Question: What is the marginal distribution of $𝑋$? There is no need to sum the joint PMF over $𝑦$: the $𝑛$ trials are independent, and each trial yields ${𝐶}_{1}$ with probability ${𝑝}_{𝑋}$. Thus $𝑋$ is the number of ${𝐶}_{1}$s in $𝑛$ independent Bernoulli trials with success probability ${𝑝}_{𝑋}$, so $𝑋∼\operatorname{Bin}(𝑛,{𝑝}_{𝑋})$ and similarly $𝑌∼\operatorname{Bin}(𝑛,{𝑝}_{𝑌})$. But it is worth an exercise to calculate the marginal distribution of $𝑋$ explicitly.

Exercises