Exponential Distribution and Continuous Random Variables

flowchart TD
    A[02 Exponential And Continuous Random Variables] --> B[Key Concepts]
    A --> C[Core Principles]
    A --> D[Practical Applications]
    B --> E[Fundamental definitions]
    C --> F[Design patterns]
    D --> G[Real-world usage]

This topic explores fundamental concepts that shape our understanding of the world.

The exponential distribution models the time between events in a Poisson process, while the theory of continuous random variables extends probability to quantities that can take any value in an Interval.

Definition. A probability density function (PDF) $f(x)$ of a continuous random variable $X$ Is a non-negative function satisfying:

Probabilities are found by integration:

Definition. The cumulative distribution function (CDF) is

Properties:

• $F(-\infty) = 0$, $F(\infty) = 1$
• $F$ is non-decreasing
• $f(x) = F"(x)$ where $F$ is differentiable
• $P(a < X \leq b) = F(b) - F(a)$

Definition. The expected value of a continuous random variable $X$ is

For a function $g(X)$:

Definition.

The linear properties carry over from the discrete case:

Definition. The median $m$ satisfies $F(m) = 0.5$ I.e., $\int_{-\infty}^{m}f(x)\,dx = 0.5$.

Definition. The mode is the value of $x$ at which $f(x)$ is maximised.

Definition. The lower quartile $Q_1$ satisfies $F(Q_1) = 0.25$ and the upper quartile $Q_3$ satisfies $F(Q_3) = 0.75$.

The interquartile range is $\mathrm{IQR} = Q_3 - Q_1$.

Definition. A continuous random variable $X$ follows an exponential distribution with rate Parameter $\lambda$ (where $\lambda > 0$), written $X \sim \mathrm{Exp}(\lambda)$ If

And $f(x) = 0$ for $x < 0$.

Using integration by parts with $u = x$, $dv = \lambda e^{-\lambda x}\,dx$:

$du = dx$, $v = -e^{-\lambda x}$.

Note: $\lim_{x\to\infty}xe^{-\lambda x} = 0$ by L’Hôpital’s rule (exponential decay dominates).

First compute $E(X^2)$:

Integration by parts twice with $u = x^2$, $dv = \lambda e^{-\lambda x}\,dx$:

$du = 2x\,dx$, $v = -e^{-\lambda x}$.

Theorem. The exponential distribution is the only continuous memoryless distribution:

This uses $P(X > x) = 1 - F(x) = e^{-\lambda x}$.

A Poisson process with rate $\lambda$ satisfies:

• The number of events in an interval of length $t$ follows $\mathrm{Po}(\lambda t)$
• The time between consecutive events follows $\mathrm{Exp}(\lambda)$
• The inter-arrival times are independent and identically distributed

Proof sketch that inter-arrival times are exponential. Let $T$ be the time until the first Event. $P(T > t) = P(\mathrm{no events in }[0,t]) = P(N(t) = 0) = \dfrac{e^{-\lambda t}(\lambda t)^0}{0!} = e^{-\lambda t}$.

So $P(T \leq t) = 1 - e^{-\lambda t}$ Which is the CDF of $\mathrm{Exp}(\lambda)$. $\blacksquare$

For the exponential distribution:

The median is $x_{0.5} = -\dfrac{\ln(0.5)}{\lambda} = \dfrac{\ln 2}{\lambda}$.

Example. $X \sim \mathrm{Exp}(0.5)$. Find $P(X > 3)$, $P(1 < X < 4)$ And the median.

$P(X > 3) = e^{-0.5 \times 3} = e^{-1.5} \approx 0.2231$.

$P(1 < X < 4) = F(4) - F(1) = (1-e^{-2}) - (1-e^{-0.5}) = e^{-0.5} - e^{-2} \approx 0.6065 - 0.1353 = 0.4712$.

Median $= \dfrac{\ln 2}{0.5} = 2\ln 2 \approx 1.386$.

Example. $X$ has PDF $f(x) = \dfrac{3}{8}x^2$ for $0 \leq x \leq 2$ and $f(x) = 0$ otherwise.

Verify: $\int_0^2 \dfrac{3}{8}x^2\,dx = \dfrac{3}{8}\cdot\dfrac{8}{3} = 1$. $\checkmark$

$E(X) = \int_0^2 x\cdot\dfrac{3}{8}x^2\,dx = \dfrac{3}{8}\int_0^2 x^3\,dx = \dfrac{3}{8}\cdot\dfrac{16}{4} = \dfrac{3}{2}$.

$E(X^2) = \dfrac{3}{8}\int_0^2 x^4\,dx = \dfrac{3}{8}\cdot\dfrac{32}{5} = \dfrac{12}{5}$.

$\mathrm{Var}(X) = \dfrac{12}{5} - \left(\dfrac{3}{2}\right)^2 = \dfrac{12}{5} - \dfrac{9}{4} = \dfrac{3}{20}$.

Median: $\int_0^m \dfrac{3}{8}x^2\,dx = 0.5 \implies \dfrac{m^3}{8} = 0.5 \implies m^3 = 4 \implies m = \sqrt[3]{4} \approx 1.587$.

Example. $f(x) = 2x$ for $0 \leq x \leq 1$. Find $F(x)$.

For $0 \leq x \leq 1$: $F(x) = \int_0^x 2t\,dt = x^2$.

For $x < 0$: $F(x) = 0$. For $x > 1$: $F(x) = 1$.

Example. The lifetime of a component is modelled by $X \sim \mathrm{Exp}(\lambda)$. A sample of 10 components gives a mean lifetime of 420 hours. Test at the 5% level whether $\lambda = 0.005$ against $H_1: \lambda \neq 0.005$.

Under $H_0$: $E(X) = 1/\lambda = 200$ hours. Since $n$ is large, use the approximate normal Distribution of $\bar{X}$:

$\bar{X} \sim N\!\left(\frac{1}{\lambda}, \frac{1}{n\lambda^2}\right) = N(200, 4000)$ Approximately.

$z = \dfrac{420 - 200}{\sqrt{4000}} = \dfrac{220}{63.25} = 3.48$.

$|z| = 3.48 > 1.96$ So reject $H_0$.

Theorem. If $X \sim \mathrm{Exp}(\lambda)$ Then for all $s, t > 0$:

By definition of conditional probability:

Since $X > s + t$ implies $X > s$.

Using the survival function $S(x) = P(X > x) = e^{-\lambda x}$:

Theorem. If a continuous random variable $X$ on $(0, \infty)$ satisfies $P(X > s+t \mid X > s) = P(X > t)$ for all $s, t > 0$ Then $X \sim \mathrm{Exp}(\lambda)$ for some $\lambda > 0$.

Let $G(t) = P(X > t)$. The memoryless condition gives:

This is Cauchy’s functional equation. Since $G$ is non-increasing and $0 \leq G \leq 1$ The only Solutions are:

For some $\lambda \geq 0$. Since $G$ is non-trivial (not identically 1), $\lambda > 0$. Therefore:

Which is the CDF of $\mathrm{Exp}(\lambda)$. $\blacksquare$

The memoryless property means:

• If a light bulb has been on for 100 hours, the probability it lasts another 50 hours is the same as a new bulb lasting 50 hours.
• If you have waited 20 minutes for a bus, your expected additional wait time is the same as if you had just arrived.
• This property makes exponential models appropriate for random failure mechanisms (electronic components) but inappropriate for wear-out mechanisms (mechanical parts).

Theorem. In a Poisson process with rate $\lambda$ The time between consecutive events follows $\mathrm{Exp}(\lambda)$.

Let $T$ be the time from an arbitrary starting point until the first event.

Since the number of events in $[0,t]$ follows $\mathrm{Po}(\lambda t)$:

Therefore $P(T \leq t) = 1 - e^{-\lambda t}$ Which is the CDF of $\mathrm{Exp}(\lambda)$. $\blacksquare$

If $T_1, T_2, \ldots, T_n$ are $n$ independent inter-arrival times, each $\sim \mathrm{Exp}(\lambda)$ Then the total time until the $n$-th event is:

This connects the exponential to the gamma distribution.

Example. Calls arrive at a call centre as a Poisson process at rate 5 per hour.

(a) Find the probability that the time between two consecutive calls exceeds 20 minutes.

$T \sim \mathrm{Exp}(5)$ (rate per hour). $P(T > 1/3) = e^{-5/3} \approx 0.1889$.

(b) Find the probability that at least 3 calls arrive in the next 30 minutes.

$N(0.5) \sim \mathrm{Po}(2.5)$. $P(N \geq 3) = 1 - P(N \leq 2) = 1 - e^{-2.5}(1 + 2.5 + 2.5^2/2) = 1 - 0.0821 \times 9.125 \approx 1 - 0.749 = 0.251$.

(c) Find the median inter-arrival time.

Median $= \dfrac{\ln 2}{\lambda} = \dfrac{\ln 2}{5} \approx 0.139\,\mathrm{hours} \approx 8.3\,\mathrm{minutes}$.

Definition. $X \sim U(a, b)$ if:

And $f(x) = 0$ otherwise.

A mixture distribution arises when a random variable is selected from one of several Sub-populations. If $X$ comes from distribution 1 with probability $p$ and from distribution 2 with Probability $1-p$:

The variance formula includes an extra term from the difference in means, this is the law of total Variance.

Example. A machine produces components. With probability 0.7 it is correctly calibrated, Producing components with lifetime $\sim \mathrm{Exp}(0.01)$. With probability 0.3 it is faulty, Producing components with lifetime $\sim \mathrm{Exp}(0.002)$. Find the overall PDF, the expected Lifetime, and $P(X > 100)$.

$E(X) = 0.7 \times 100 + 0.3 \times 500 = 70 + 150 = 220\,\mathrm{hours}$.

$P(X > 100) = 0.7\,e^{-0.01 \times 100} + 0.3\,e^{-0.002 \times 100} = 0.7e^{-1} + 0.3e^{-0.2} \approx 0.7(0.3679) + 0.3(0.8187)$

$\approx 0.2575 + 0.2456 = 0.5031$.

The PDF $f(x)$ gives the density of probability at $x$. It is not a probability itself, $f(x)$ can be greater than 1. The CDF $F(x)$ gives the accumulated probability up to $x$ And always Satisfies $0 \leq F(x) \leq 1$.

Common error: writing $P(X = a) = f(a)$ for a continuous RV. This is wrong, $P(X = a) = 0$ always. Probabilities are areas under the PDF, not values of the PDF.

For a discrete RV, $P(X = a) > 0$ for specific values, and probabilities sum to 1.

For a continuous RV, $P(X = a) = 0$ for any single value, and probabilities integrate to 1.

Never use summation for a continuous RV or integration for a discrete RV (unless using the CDF Formalism).

$X \sim \mathrm{Exp}(\lambda)$ has mean $1/\lambda$. A common error is to confuse $\lambda$ (the Rate) with the mean. If the mean lifetime is 200 hours, then $\lambda = 1/200 = 0.005$ Not $\lambda = 200$.

Check: a larger $\lambda$ means shorter lifetimes on average (events happen more frequently).

If events occur at rate $\lambda = 5$ per hour, then the inter-arrival time is $\mathrm{Exp}(5)$ Measured in hours. If you want the probability of waiting more than 20 minutes, convert to Hours: $t = 1/3$ hours. Using $t = 20$ directly would give $P(T > 20) = e^{-100}$ Which is Essentially zero and wrong.

Problem. The lifetime $T$ of a component follows an exponential distribution with mean 200 Hours. Given that the component has survived 150 hours, find the probability it survives a further 100 hours.

Solution. By the memoryless property of the exponential distribution:

$\lambda = \dfrac{1}{200} = 0.005$.

$P(T > 100) = e^{-0.005 \times 100} = e^{-0.5} = \boxed{0.607}$ (3 s.f.).

Problem. $X \sim \mathrm{U}(0, 10)$. Find $P(X > 6 \mid X > 3)$.

Solution. $P(X > 6 \mid X > 3) = \dfrac{P(X > 6)}{P(X > 3)} = \dfrac{0.4}{0.7} = \dfrac{4}{7} \approx \boxed{0.571}$.

Alternatively: conditional on $X > 3$ The distribution is $\mathrm{U}(3, 10)$ So $P(X > 6 \mid X > 3) = \dfrac{10-6}{10-3} = \dfrac{4}{7}$.

Problem. $X$ and $Y$ are independent with $X \sim \mathrm{Exp}(\lambda_1)$ and $Y \sim \mathrm{Exp}(\lambda_2)$. Find $P(X < Y)$.

Solution. Using the joint density and integration:

For example, if $\lambda_1 = \lambda_2$: $P(X < Y) = \dfrac{1}{2}$ (by symmetry).

Problem. A continuous random variable $X$ has PDF $f(x) = kx(4-x)$ for $0 \leq x \leq 4$. Find $k$ The CDF, and $P(1 < X < 3)$.

Solution. $\displaystyle\int_0^4 kx(4-x)\,dx = k\!\left[2x^2 - \frac{x^3}{3}\right]_0^4 = k\!\left(32 - \frac{64}{3}\right) = \frac{32k}{3} = 1 \implies k = \frac{3}{32}$.

CDF: $F(x) = \dfrac{3}{32}\!\left(2x^2 - \dfrac{x^3}{3}\right) = \dfrac{3x^2}{16} - \dfrac{x^3}{32}$ for $0 \leq x \leq 4$.

$P(1 < X < 3) = F(3) - F(1) = \left(\dfrac{27}{16} - \dfrac{27}{32}\right) - \left(\dfrac{3}{16} - \dfrac{1}{32}\right) = \dfrac{27}{32} - \dfrac{5}{32} = \boxed{\dfrac{11}{16}}$.

Problem. $X_1, X_2, \ldots, X_{50}$ are independent, each following $\mathrm{Exp}(0.1)$. Approximate $P(\overline{X} > 12)$.

Solution. $E(X_i) = 10$, $\mathrm{Var}(X_i) = 100$. $E(\overline{X}) = 10$ $\mathrm{Var}(\overline{X}) = \dfrac{100}{50} = 2$.

By the CLT, $\overline{X} \approx N(10, 2)$ approximately.

Problem. $X$ has PDF $f_X(x) = 2x$ for $0 < x < 1$. Find the PDF of $Y = X^2$.

Solution. For $0 < y < 1$: $F_Y(y) = P(Y \leq y) = P(X^2 \leq y) = P(X \leq \sqrt{y}) = (\sqrt{y})^2 = y$.

So $Y \sim \mathrm{U}(0,1)$.