14 Exponential Distribution

14.1 Probability Distribution Function

A random variable is said to have an Exponential Distribution with parameter \(\beta\) if its probability distribution function is

\[f(x)=\left\{ \begin{array}{ll} \frac{1}{\beta}e^{\frac{-x}{B}}, & 0<x,\ \ 0<\beta\\ 0 & otherwise \end{array}\right. \]

14.2 Cumulative Distribution Function

\[\begin{aligned} F(x) &= \int\limits_{0}^{x}\frac{1}{\beta}\exp\Big\{{\frac{-t}{\beta}}\Big\}dt \\ &= -\exp\Big\{{\frac{-t}{\beta}}\Big\}\Big|_0^x \\ &= -\exp\Big\{{\frac{-x}{\beta}}\Big\}-\Big(-\exp\Big\{{\frac{-0}{\beta}}\Big\}\Big) \\ &= \exp\Big\{{\frac{0}{\beta}}\Big\}-\exp\Big\{{\frac{-x}{\beta}}\Big\} \\ &= 1-\exp\Big\{{\frac{-x}{\beta}}\Big\} \end{aligned}\]

And so the cumulative distribution function is given by \[F(x)=\left\{ \begin{array}{ll} 1-e^{\frac{-x}{\beta}}, & 0<x,\ 0<\beta\\ 0 & otherwise \end{array} \right. \]

The figures on the top and bottom display the Exponential probability and cumulative distirubtion functions, respectively, for $\beta=1,3$.

(#fig:Exponential_Distribution)The figures on the top and bottom display the Exponential probability and cumulative distirubtion functions, respectively, for \(\beta=1,3\).

14.3 Expected Values

\[\begin{aligned} E(X) &= \int\limits_{0}^{\infty}xf(x)dx \\ &= \int\limits_{0}^{\infty}x\frac{1}{\beta}e^{\frac{-x}{\beta}}dx \\ &= \frac{1}{\beta}\int\limits_{0}^{\infty}xe^{\frac{-x}{\beta}}dx \\ &= \frac{1}{\beta}\int\limits_{0}^{\infty}x^{2-1}e^{\frac{-x}{\beta}}dx\\ ^{[1]} &= \frac{1}{\beta}(\beta^2\Gamma(2)) \\ &=\frac{\beta^2\cdot 1!}{\beta} \\ &=\beta \end{aligned}\]

  1. \(\int\limits_{0}^{\infty}x^{\alpha-1}e^{-\frac{x}{\beta}}dx = \beta^\alpha\Gamma(\alpha)\)

\[\begin{aligned} E(X^2) &= \int\limits_{0}^{\infty}x^2f(x)dx \\ &= \int\limits_{0}^{\infty}x^2\frac{1}{\beta}e^{\frac{-x}{\beta}}dx \\ &= \frac{1}{\beta}\int\limits_{0}^{\infty}x^2e^{\frac{-x}{\beta}}dx \\ &= \frac{1}{\beta}\int\limits_{0}^{\infty}x^{3-1}e^{\frac{-x}{\beta}}dx \\ ^{[1]} &= \frac{1}{\beta}(\beta^3\Gamma(3)) \\ &= \frac{\beta^3\cdot 2!}{\beta} \\ &= 2\beta^2 \end{aligned}\]

  1. \(\int\limits_{0}^{\infty}x^{\alpha-1}e^{-\frac{x}{\beta}}dx = \beta^\alpha\Gamma(\alpha)\)

\[\begin{aligned} \mu &= E(X) \\ &= \beta \\ \\ \\ \sigma^2 &= E(X^2)-E(X)^2 \\ &= 2\beta^2-\beta^2 \\ &= \beta^2 \end{aligned}\]

14.4 Moment Generating Function

\[\begin{aligned} M_X(t) &= E(e^{tX}) \\ &= \int\limits_{0}^{\infty}e^{tx}\frac{1}{\beta}e^{\frac{-x}{\beta}}dx \\ &= \frac{1}{\beta}\int\limits_{0}^{\infty}e^{tx}e^{\frac{-x}{\beta}}dx \\ &= \frac{1}{\beta}\int\limits_{0}^{\infty}e^{tx-\frac{x}{\beta}}dx \\ &= \frac{1}{\beta}\int\limits_{0}^{\infty}e^{\frac{\beta tx}{\beta}-\frac{x}{\beta}}dx \\ &= \frac{1}{\beta}\int\limits_{0}^{\infty}e^{\frac{\beta tx-x}{\beta}}dx \\ &= \frac{1}{\beta}\int\limits_{0}^{\infty}e^{\frac{-x(\beta 1-\beta t}{\beta}}dx \\ &= \frac{1}{\beta}(\frac{-\beta}{1-\beta t})e^{\frac{-x(1-\beta t}{\beta}}|_0^\infty \\ &= \frac{-1}{1-\beta t}e^{\frac{-x(1-\beta t}{\beta}}|_0^\infty \\ &= \frac{-1}{1-\beta t}\cdot 0-\frac{-1}{1-\beta t}e^0 \\ &= \frac{1}{1-\beta t}=(1-\beta t)^{-1} \\ \\ \\ M_X^{(1)}(t) &= -1(1-\beta t)^{-2}(-\beta) \\ &= \beta(1-\beta t)^{-2} \\ \\ \\ M_X^{(2)}(t) &= -2\beta(1-\beta t)^{-3}(-\beta) \\ &= 2\beta^2(1-\beta t)^{-3} \\ \\ \\ E(X) &= M_X^{(1)}(0) \\ &= \beta(1-\beta\cdot 0)^{-2} \\ &= \beta(1-0)^{-2} \\ &= \beta(1)^{-2} \\ &= \beta \\ \\ \\ E(X^2) &= M_X^{(2)}(0) \\ &= 2\beta^2(1-\beta\cdot 0)^{-3} \\ &= 2\beta^2(1-0)^{-3} \\ &= 2\beta^2(1)^{-3} \\ &= 2\beta^2 \\ \\ \\ \mu &= E(X) \\ &= \beta \\ \\ \\ \sigma^2 &= E(X^2) - E(X)^2 \\ &= 2\beta^2 - \beta^2 \\ &= \beta^2 \end{aligned}\]

14.5 Maximum Likelihood Estimator

Let \(x_1,x_2,\ldots,x_n\) be a random sample from an Exponential distribution with parameter \(\beta\).

14.5.1 Likelihood Function

\[\begin{aligned} L(\theta) &= L(x_1,x_2,\ldots,x_n|\theta) \\ &= f(x_1|\theta)f(x_2|\theta)\cdots f(x_n|\theta)\\ &= \frac{1}{\theta}\exp\bigg\{-\frac{x_1}{\theta}\bigg\} \cdot\frac{1}{\theta}\exp\bigg\{-\frac{x_n}{\theta}\bigg\} \cdots\frac{1}{\theta}\exp\bigg\{-\frac{x_n}{\theta}\bigg\} \\ &= \frac{1}{\theta^n}\exp\bigg\{-\frac{1}{\theta}\sum\limits_{i=1}^{n}x_i\bigg\} \end{aligned}\]

14.5.2 Log-likelihood Function

\[\begin{aligned} \ell(\theta) &= \ln(L(\theta)) \\ &= \ln(1)-n\ln(\theta)-\frac{1}{\theta}\sum\limits_{i=1}^{n}x_i \\ &= 0-n\ln(\theta)-\theta^{-1}\sum\limits_{i=1}^{n}x_i \\ &= -n\ln(\theta)-\theta^{-1}\sum\limits_{i=1}^{n}x_i \end{aligned}\]

14.5.3 MLE for \(\beta\)

\[\begin{aligned} \frac{d\ell(\beta)}{d\beta} &= -\frac{n}{\beta}+\beta^2\sum\limits_{i=1}^{n}x_i \\ \\ \\ 0 &= -\frac{n}{\beta}+\beta^2\sum\limits_{i=1}^{n}x_i \\ \Rightarrow\frac{n}{\beta} &= \beta^2\sum\limits_{i=1}^{n}x_i \\ \Rightarrow\frac{n\beta^2}{\beta} &= \sum\limits_{i=1}^{n}x_i \\ \Rightarrow n\beta &= \sum\limits_{i=1}^{n}x_i \\ \Rightarrow \beta &= \frac{1}{n}\sum\limits_{i=1}^{n}x_i \end{aligned}\]

So \(\hat\beta=\frac{1}{n}\sum\limits_{i=1}^{n}x_i\) is the maximum likelihood estimator for \(\beta\).

14.6 Theorems for the Exponential Distribution

14.6.1 Validity of the Distribution

\[ \int\limits_{0}^{\infty}\frac{1}{\beta}e^{\frac{-x}{B}}dx = 1 \]

Proof:

\[\begin{aligned} \int\limits_{0}^{\infty}\frac{1}{\beta}e^{\frac{-x}{\beta}}dx &= -e^{\frac{-x}{\beta}}\Big|_0^\infty \\ &= -e^{\frac{-\infty}{\beta}}-(-e^{\frac{-0}{\beta}}) \\ &= e^{\frac{0}{\beta}}-e^{\frac{-\infty}{\beta}} \\ &= 1-0 \\ &= 1 \end{aligned}\]

14.6.2 Sum of Exponential Random Variables

Let \(X_1,X_2,\ldots,X_n\) be independent random variables from an Exponential distribution with parameter \(\beta\), i.e. \(X_i\sim\)Exponential\((\beta)\). Let \(Y=\sum\limits_{i=1}^{n}X_i\). Then \(Y\sim\)Gamma\((n,\beta)\).

Proof:

\[\begin{aligned} M_Y(t) &= E(e^{tY}) \\ &= E(e^{t(X_1+X_2+\cdots+X_n}) \\ &= E(e^{tX_1}e^{tX_2}\cdots e^{tX_n}) \\ &= (1-\beta t)^{-1}(1-\beta t)^{-1}\cdots(1-\beta t)^{-1} \\ &= (1-\beta t)^{-n} \end{aligned}\]

Which is the mgf for a Gamma random variable with parameters \(n\) and \(\beta\). Thus \(Y\sim\)Gamma\((n,\beta)\).