Books Probability and Statistical Inference

Bivariate Distributions of the Continuous Type

The joint probability density function of two continuous type random variables 𝑋𝑋 and 𝑌𝑌 is an integrable function 𝑓(𝑥,𝑦)𝑓(𝑥,𝑦) with the following properties:

  1. 𝑓(𝑥,𝑦)≥0𝑓(𝑥,𝑦)\geq 0 where 𝑓(𝑥,𝑦)=0𝑓(𝑥,𝑦)=0 when (𝑥,𝑦)(𝑥,𝑦) is not in the support 𝑆𝑆 of 𝑋𝑋 and 𝑌𝑌.
  2. ∫−∞∞∫−∞∞𝑓(𝑥,𝑦) d𝑥 d𝑦=1{\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }𝑓(𝑥,𝑦)\, \mathrm{d}𝑥\, \mathrm{d}𝑦=1.
  3. P[(𝑋,𝑌)∈𝐴]=∬𝐴𝑓(𝑥,𝑦) d𝑥 d𝑦ℙ[(𝑋,𝑌)\in 𝐴]={\iint }_{𝐴}𝑓(𝑥,𝑦)\, \mathrm{d}𝑥\, \mathrm{d}𝑦 where {(𝑋,𝑌)∈𝐴}\{(𝑋,𝑌)\in 𝐴\} is an event defined in the plane.

The mathematical expectations are as the same as the discrete case with integrals in place of sums.

The respective marginal probability density function of 𝑋𝑋 and 𝑌𝑌 are

𝑓𝑋(𝑥)=∫−∞∞𝑓(𝑥,𝑦) d𝑦,𝑥∈𝑆𝑋{𝑓}_{𝑋}(𝑥)={\int }_{-\infty }^{\infty }𝑓(𝑥,𝑦)\, \mathrm{d}𝑦,\quad 𝑥\in {𝑆}_{𝑋}

and

𝑓𝑌(𝑦)=∫−∞∞𝑓(𝑥,𝑦) d𝑥,𝑦∈𝑆𝑌{𝑓}_{𝑌}(𝑦)={\int }_{-\infty }^{\infty }𝑓(𝑥,𝑦)\, \mathrm{d}𝑥,\quad 𝑦\in {𝑆}_{𝑌}

if they exist.

Exercise 4.6

Let 𝑋𝑋 and 𝑌𝑌 have the joint PDF

𝑓(𝑥,𝑦)=32𝑥2(1−∣𝑦∣),−1<𝑥<1,−1<𝑦<1.𝑓(𝑥,𝑦)=\frac{3}{2}{𝑥}^{2}(1-|𝑦|),\quad -1<𝑥<1,-1<𝑦<1.
  1. Calculate the probability P({(𝑋,𝑌)∈𝐴})ℙ(\{(𝑋,𝑌)\in 𝐴\}) where 𝐴={(𝑥,𝑦):0<𝑥<1,0<𝑦<𝑥}𝐴=\{(𝑥,𝑦):0<𝑥<1,0<𝑦<𝑥\}.
  2. Find the marginal PDFs of 𝑋𝑋 and 𝑌𝑌.
  3. Find μ𝑋{\mu }_{𝑋}, σ𝑋2{\sigma }_{𝑋}^{2}, μ𝑌{\mu }_{𝑌} and σ𝑌2{\sigma }_{𝑌}^{2}.
  4. Find the correlation coefficient ρ\rho .
Solution

(a) We have

P({(𝑋,𝑌)∈𝐴})=∬𝐴𝑓(𝑥,𝑦) d𝑥 d𝑦=∫01∫0𝑥32𝑥2(1−∣𝑦∣) d𝑦 d𝑥=∫0132𝑥2∫0𝑥1−𝑦 d𝑦 d𝑥=∫0132𝑥2[𝑦−𝑦22]0𝑥 d𝑥=32∫01𝑥3−𝑥42 d𝑥=940.\begin{aligned}ℙ(\{(𝑋,𝑌)\in 𝐴\})={\iint }_{𝐴}𝑓(𝑥,𝑦)\, \mathrm{d}𝑥\, \mathrm{d}𝑦 & ={\int }_{0}^{1}{\int }_{0}^{𝑥}\frac{3}{2}{𝑥}^{2}(1-|𝑦|)\, \mathrm{d}𝑦\, \mathrm{d}𝑥 \\ & ={\int }_{0}^{1}\frac{3}{2}{𝑥}^{2}{\int }_{0}^{𝑥}1-𝑦\, \mathrm{d}𝑦\, \mathrm{d}𝑥 \\ & ={\int }_{0}^{1}\frac{3}{2}{𝑥}^{2}{\left[𝑦-\frac{{𝑦}^{2}}{2}\right]}_{0}^{𝑥}\, \mathrm{d}𝑥 \\ & =\frac{3}{2}{\int }_{0}^{1}{𝑥}^{3}-\frac{{𝑥}^{4}}{2}\, \mathrm{d}𝑥 \\ & =\frac{9}{40}.\end{aligned}

(b) The marginal PDF of 𝑋𝑋 is obtained by integrating out 𝑦𝑦:

𝑓𝑋(𝑥)=∫−1132𝑥2(1−∣𝑦∣) d𝑦=32𝑥2∫−111−∣𝑦∣ d𝑦=32𝑥2⋅2∫011−∣𝑦∣ d𝑦=32𝑥2⋅2[𝑦−𝑦22]01=32𝑥2.\begin{aligned}{𝑓}_{𝑋}(𝑥) & ={\int }_{-1}^{1}\frac{3}{2}{𝑥}^{2}(1-|𝑦|)\, \mathrm{d}𝑦 \\ & =\frac{3}{2}{𝑥}^{2}{\int }_{-1}^{1}1-|𝑦|\, \mathrm{d}𝑦 \\ & =\frac{3}{2}{𝑥}^{2}\cdot 2{\int }_{0}^{1}1-|𝑦|\, \mathrm{d}𝑦 \\ & =\frac{3}{2}{𝑥}^{2}\cdot 2{\left[𝑦-\frac{{𝑦}^{2}}{2}\right]}_{0}^{1} \\ & =\frac{3}{2}{𝑥}^{2}.\end{aligned}

Now, we integrate out 𝑥𝑥 to get the marginal PDF of 𝑌𝑌:

𝑓𝑌(𝑦)=∫−1132𝑥2(1−∣𝑦∣) d𝑥=32(1−∣𝑦∣)∫−11𝑥2 d𝑥=32(1−∣𝑦∣)⋅[𝑥33]−11=1−∣𝑦∣.\begin{aligned}{𝑓}_{𝑌}(𝑦) & ={\int }_{-1}^{1}\frac{3}{2}{𝑥}^{2}(1-|𝑦|)\, \mathrm{d}𝑥 \\ & =\frac{3}{2}(1-|𝑦|){\int }_{-1}^{1}{𝑥}^{2}\, \mathrm{d}𝑥 \\ & =\frac{3}{2}(1-|𝑦|)\cdot {\left[\frac{{𝑥}^{3}}{3}\right]}_{-1}^{1} \\ & =1-|𝑦|.\end{aligned}

(c) We have

μ𝑋=∫−11𝑥𝑓𝑋(𝑥) d𝑥.{\mu }_{𝑋}={\int }_{-1}^{1}𝑥{𝑓}_{𝑋}(𝑥)\, \mathrm{d}𝑥.

Note that 𝑓𝑋(𝑥){𝑓}_{𝑋}(𝑥) is an even function, so 𝑥𝑓𝑋(𝑥)𝑥{𝑓}_{𝑋}(𝑥) is odd; hence integrating it over a symmetric interval gives 0. Thus, μ𝑋=0{\mu }_{𝑋}=0. Similarly, since

μ𝑌=∫−11𝑦𝑓𝑌(𝑦) d𝑦.{\mu }_{𝑌}={\int }_{-1}^{1}𝑦{𝑓}_{𝑌}(𝑦)\, \mathrm{d}𝑦.

and 𝑓𝑌(𝑦){𝑓}_{𝑌}(𝑦) is also an even function, we have μ𝑌=0{\mu }_{𝑌}=0.

We now compute σ𝑋2{\sigma }_{𝑋}^{2} and σ𝑌2{\sigma }_{𝑌}^{2}. We have

σ𝑋2=E[𝑋2]−μ𝑋2=E[𝑋2]=∫−11𝑥2𝑓𝑋(𝑥) d𝑥=∫−11𝑥2⋅32𝑥2 d𝑥=[3𝑥510]−11=35.\begin{aligned}{\sigma }_{𝑋}^{2}=\mathbb{E}[{𝑋}^{2}]-{\mu }_{𝑋}^{2}=\mathbb{E}[{𝑋}^{2}] & ={\int }_{-1}^{1}{𝑥}^{2}{𝑓}_{𝑋}(𝑥)\, \mathrm{d}𝑥 \\ & ={\int }_{-1}^{1}{𝑥}^{2}\cdot \frac{3}{2}{𝑥}^{2}\, \mathrm{d}𝑥 \\ & ={\left[\frac{3{𝑥}^{5}}{10}\right]}_{-1}^{1} \\ & =\frac{3}{5}.\end{aligned}

To compute σ𝑌2{\sigma }_{𝑌}^{2}, we have

σ𝑌2=E[𝑌2]−μ𝑌2=E[𝑌2]=∫−11𝑦2𝑓𝑌(𝑦) d𝑦=∫−11𝑦2⋅(1−∣𝑦∣) d𝑦=2∫01𝑦2⋅(1−𝑦) d𝑦=[2(𝑦33−𝑦44)]01=16.\begin{aligned}{\sigma }_{𝑌}^{2}=\mathbb{E}[{𝑌}^{2}]-{\mu }_{𝑌}^{2}=\mathbb{E}[{𝑌}^{2}] & ={\int }_{-1}^{1}{𝑦}^{2}{𝑓}_{𝑌}(𝑦)\, \mathrm{d}𝑦 \\ & ={\int }_{-1}^{1}{𝑦}^{2}\cdot (1-|𝑦|)\, \mathrm{d}𝑦 \\ & =2{\int }_{0}^{1}{𝑦}^{2}\cdot (1-𝑦)\, \mathrm{d}𝑦 \\ & ={\left[2\left(\frac{{𝑦}^{3}}{3}-\frac{{𝑦}^{4}}{4}\right)\right]}_{0}^{1} \\ & =\frac{1}{6}.\end{aligned}

(d) We could compute

Cov⁡(𝑋,𝑌)=E[𝑋𝑌]−μ𝑋μ𝑌=E[𝑋𝑌]=∫−11∫−11𝑥𝑦𝑓(𝑥,𝑦) d𝑥 d𝑦.\operatorname{Cov}(𝑋,𝑌)=\mathbb{E}[𝑋𝑌]-{\mu }_{𝑋}{\mu }_{𝑌}=\mathbb{E}[𝑋𝑌]={\int }_{-1}^{1}{\int }_{-1}^{1}𝑥𝑦𝑓(𝑥,𝑦)\, \mathrm{d}𝑥\, \mathrm{d}𝑦.

But we note that, by one glance, 𝑋𝑋 and 𝑌𝑌 are independent since 𝑓(𝑥,𝑦)=𝑓𝑋(𝑥)𝑓𝑌(𝑦)𝑓(𝑥,𝑦)={𝑓}_{𝑋}(𝑥){𝑓}_{𝑌}(𝑦). (In fact, we have not defined the joint PDF of continuous type random variables yet. We will do so right after this exercise.) Therefore, Cov⁡(𝑋,𝑌)=0\operatorname{Cov}(𝑋,𝑌)=0 and hence ρ=0\rho =0.

Continuous type random variables 𝑋𝑋 and 𝑌𝑌 are independent if and only if their join PDF can be factored as the product of the marginal PDFs, i.e.,

𝑓(𝑥,𝑦)=𝑓𝑋(𝑥)𝑓𝑌(𝑦),𝑥∈𝑆𝑋,𝑦∈𝑆𝑌.𝑓(𝑥,𝑦)={𝑓}_{𝑋}(𝑥){𝑓}_{𝑌}(𝑦),\quad 𝑥\in {𝑆}_{𝑋},𝑦\in {𝑆}_{𝑌}.

The definitions of conditional probability density functions and conditional mathematical expectations of continuous types also carry over naturally from the discrete case.

Exercise 4.7

Let 𝑋𝑋 and 𝑌𝑌 have the joint PDF

𝑓(𝑥,𝑦)=2,0≤𝑥≤𝑦≤1.𝑓(𝑥,𝑦)=2,\quad 0\leq 𝑥\leq 𝑦\leq 1.

That is, (𝑋,𝑌)(𝑋,𝑌) is uniformly distributed over the support 𝑆𝑆, which is plotted in Figure 4.3.

  1. Find the marginal PDFs of 𝑋𝑋 and 𝑌𝑌.
  2. Find the conditional PDF of 𝑌𝑌 given 𝑋=𝑥𝑋=𝑥.
  3. Find the conditional mean μ𝑌∣𝑥{\mu }_{𝑌|𝑥}.
  4. Find the conditional variance σ𝑌∣𝑥2{\sigma }_{𝑌|𝑥}^{2}.
Figure 4.3 The support 𝑆={(𝑥,𝑦):0≤𝑥≤𝑦≤1}𝑆=\{(𝑥,𝑦):0\leq 𝑥\leq 𝑦\leq 1\} of the joint distribution.
Solution

(a) We first find 𝑓𝑋(𝑥){𝑓}_{𝑋}(𝑥). Let 𝑋=𝑥𝑋=𝑥 be fixed. Imagine a vertical line 𝑥=𝑥𝑥=𝑥 passing through the support 𝑆𝑆 in Figure 4.3 along the direction of the 𝑦𝑦-axis. Note that it enters 𝑆𝑆 at (𝑥,𝑥)(𝑥,𝑥) and leaves at (𝑥,1)(𝑥,1). Integrating 𝑓(𝑥,𝑦)𝑓(𝑥,𝑦) over 𝑦𝑦 from 𝑥𝑥 to 11 gives the marginal PDF of 𝑋𝑋:

𝑓𝑋(𝑥)=∫𝑥12 d𝑦=2(1−𝑥).{𝑓}_{𝑋}(𝑥)={\int }_{𝑥}^{1}2\, \mathrm{d}𝑦=2(1-𝑥).

Similarly, we can find 𝑓𝑌(𝑦){𝑓}_{𝑌}(𝑦) by integrating 𝑓(𝑥,𝑦)𝑓(𝑥,𝑦) over 𝑥𝑥 from 00 to 𝑦𝑦.

𝑓𝑌(𝑦)=∫0𝑦2 d𝑥=2𝑦.{𝑓}_{𝑌}(𝑦)={\int }_{0}^{𝑦}2\, \mathrm{d}𝑥=2𝑦.

(b) The conditional PDF of 𝑌𝑌 given 𝑋=𝑥𝑋=𝑥 (0≤𝑥<10\leq 𝑥<1) is given by

h(𝑦∣𝑥)=𝑓(𝑥,𝑦)𝑓𝑋(𝑥)=11−𝑥,𝑦∈[𝑥,1].h(𝑦|𝑥)=\frac{𝑓(𝑥,𝑦)}{{𝑓}_{𝑋}(𝑥)}=\frac{1}{1-𝑥},\quad 𝑦\in [𝑥,1].

That is, given 𝑋=𝑥𝑋=𝑥, 𝑌𝑌 is uniformly distributed over the interval [𝑥,1][𝑥,1], which agrees with our intuition. Recall that if a random variable 𝑊∼𝒰(𝑎,𝑏)𝑊∼𝒰(𝑎,𝑏), then its mean and variance are μ𝑊=(𝑎+𝑏)/2{\mu }_{𝑊}=(𝑎+𝑏)/2 and σ𝑊2=(𝑏−𝑎)2/12{\sigma }_{𝑊}^{2}={(𝑏-𝑎)}^{2}/12, respectively. Therefore, before we compute μ𝑌∣𝑥{\mu }_{𝑌|𝑥} and σ𝑌∣𝑥2{\sigma }_{𝑌|𝑥}^{2} using the definition of conditional expectations, we know that μ𝑌∣𝑥=(𝑥+1)/2{\mu }_{𝑌|𝑥}=(𝑥+1)/2 and σ𝑌∣𝑥2=(1−𝑥)2/12{\sigma }_{𝑌|𝑥}^{2}={(1-𝑥)}^{2}/12. In the following, we will verify this by computing integrals.

(c) μ𝑌∣𝑥{\mu }_{𝑌|𝑥} is given by

μ𝑌∣𝑥=∫𝑥1𝑦h(𝑦∣𝑥) d𝑦=∫𝑥1𝑦1−𝑥 d𝑦=1+𝑥2.{\mu }_{𝑌|𝑥}={\int }_{𝑥}^{1}𝑦h(𝑦|𝑥)\, \mathrm{d}𝑦={\int }_{𝑥}^{1}\frac{𝑦}{1-𝑥}\, \mathrm{d}𝑦=\frac{1+𝑥}{2}.

(d) We need to compute E[𝑌2∣𝑋=𝑥]\mathbb{E}[{𝑌}^{2}|𝑋=𝑥]. We have

E[𝑌2∣𝑋=𝑥]=∫𝑥1𝑦2h(𝑦∣𝑥) d𝑦=∫𝑥1𝑦21−𝑥 d𝑦=1+𝑥+𝑥23.\mathbb{E}[{𝑌}^{2}|𝑋=𝑥]={\int }_{𝑥}^{1}{𝑦}^{2}h(𝑦|𝑥)\, \mathrm{d}𝑦={\int }_{𝑥}^{1}\frac{{𝑦}^{2}}{1-𝑥}\, \mathrm{d}𝑦=\frac{1+𝑥+{𝑥}^{2}}{3}.

Therefore,

σ𝑌∣𝑥2=E[𝑌2∣𝑋=𝑥]−μ𝑌∣𝑥2=1+𝑥+𝑥23−(1+𝑥2)2=(1−𝑥)212.{\sigma }_{𝑌|𝑥}^{2}=\mathbb{E}[{𝑌}^{2}|𝑋=𝑥]-{\mu }_{𝑌|𝑥}^{2}=\frac{1+𝑥+{𝑥}^{2}}{3}-{\left(\frac{1+𝑥}{2}\right)}^{2}=\frac{{(1-𝑥)}^{2}}{12}.