31 Poisson Distribution

31.1 Probability Mass Function

A random variable is said to have a Poisson distribution with parameter \(\lambda\) if its probability mass function is:

\[p(x)=\left\{ \begin{array}{ll} \frac{\lambda^xe^{-\lambda}}{x!}, & x=0,1,2,...\\ 0 & otherwise \end{array}\right. \]

31.2 Cumulative Mass Function

\[P(x)=\left\{ \begin{array}{lll} e^{-\lambda}\sum\limits_{i=0}^{x}\frac{\lambda^i}{i!}, & x=1,2,3,...\\ 0 & otherwise \end{array}\right. \]

A recursive form of the cdf can be derived and has some usefulness in computer applications. With it, one need only initiate the first value and additional cumulative probabilities can be calculated. It is derived as follows:

\[\begin{aligned} P(X=x+1) &= \frac{e^{-\lambda}\lambda^{x+1}}{(x+1)!} \\ &= \frac{\lambda}{x+1}\frac{e^{-\lambda}\lambda^x}{x!} \\ &= \frac{\lambda}{x+1}P(X=x) \end{aligned}\]

The graphs on the left and right show a Poisson probability cumulative distribution function, respectively, for $\lambda=3$.

Figure 31.1: The graphs on the left and right show a Poisson probability cumulative distribution function, respectively, for \(\lambda=3\).

31.3 Expected Values

\[\begin{aligned} E(X) &= \sum\limits_{x=0}^{\infty}x\frac{\lambda^xe^{-\lambda}}{x!} \\ &= e^{-\lambda}\sum\limits_{x=0}^{\infty}x\frac{\lambda^x}{x!} \\ &= e^{-\lambda}\Big(0\frac{\lambda^0}{0!}+1\frac{\lambda^1}{1!} +2\frac{\lambda^2}{2!}+3\frac{\lambda^3}{3!}+\cdots\Big) \\ &= e^{-\lambda}\Big(0+\lambda^1+\frac{\lambda^2}{1!}+\frac{\lambda^3}{2!}+\cdots\Big) \\ &= \lambda e^{-\lambda}\Big(\lambda^0+\frac{\lambda^1}{1!}+\frac{\lambda^2}{2!}+\cdots\Big) \\ &= \lambda e^{-\lambda}\Big(\frac{\lambda^0}{0!}+\frac{\lambda^1}{1!} +\frac{\lambda^2}{2!}+\cdots\Big) \\ &= \lambda e^{-\lambda}e^\lambda \\ &= \lambda e^(-\lambda+\lambda) \\ &= \lambda \end{aligned}\]

  1. Taylor Series Expansion: \(e^x=\frac{x^0}{0!}+\frac{x^1}{1!} +\frac{x^2}{2!}+\cdots =1+\frac{x^1}{1!}+\frac{x^2}{2!}+\cdots\)

\[\begin{aligned} E(X^2) &= \sum\limits_{x=0}^{\infty}x\frac{\lambda^xe^{-\lambda}}{x!} \\ &= e^{-\lambda}\sum\limits_{x=0}^{\infty}x\frac{\lambda^x}{x!} \\ &= e^{-\lambda}\Big(0^2\frac{\lambda^0}{0!}+1^2\frac{\lambda^1}{1!} +2^2\frac{\lambda^2}{2!}+3^2\frac{\lambda^3}{3!}+\cdots\Big) \\ &= \lambda e^{-\lambda}\Big(\frac{\lambda^0}{1} +2\frac{\lambda^1}{1!}+3\frac{\lambda^2}{2!}+\cdots\Big) \\ &= \lambda e^{-\lambda}\sum\limits_{x=0}^{\infty}(x+1)\frac{\lambda^x}{x!} \\ &= \lambda e^{-\lambda}\sum\limits_{x=0}^{\infty} \Big(x\frac{\lambda^x}{x!}+\frac{\lambda^x}{x!}\Big) \\ &= \lambda e^{-\lambda}\Big(\sum\limits_{x=0}^{\infty}x\frac{\lambda^x}{x!} +\sum\limits_{x=0}^{\infty}\frac{\lambda^x}{x!}\Big) \\ &= \lambda\sum\limits_{x=0}^{\infty}x\frac{\lambda^xe^{-\lambda}}{x!} +\lambda\sum\limits_{x=0}^{\infty}\frac{\lambda^xe^{-\lambda}}{x!} \\ &= \lambda E(X)+\lambda \\ &= \lambda^2+\lambda\\ \\ \\ \mu &= E(X) \\ &= \lambda \\ \\ \\ \sigma^2 &= E(X^2) - E(X) \\ &= \lambda^2 + \lambda - \lambda^2 \\ &= \lambda \end{aligned}\]

31.4 Moment Generating Function

\[\begin{aligned} M_X(t) &= E(e^{tX}) \\ &= \sum\limits_{x=0}^{\infty}e^{tx}\frac{\lambda^xe^{-\lambda}}{x!} \\ &= \sum\limits_{x=0}^{\infty}\frac{(\lambda e^{tx})^xe^{-\lambda}}{x!} \\ &= e^{-\lambda}\sum\limits_{x=0}^{\infty}\frac{(\lambda e^{tx})^x}{x!} \\ &= e^{-\lambda}\Big[\frac{(\lambda e^t)^0}{0!}+\frac{(\lambda e^t)^1}{1!}+ \frac{(\lambda e^t)^2}{2!}+\cdots\Big] \\ &= e^{-\lambda}e^{\lambda e^t} \\ &= e^{(\lambda e^t-\lambda)} \\ &= e^{\lambda(e^t-1)} \end{aligned}\]

  1. Taylor Series Expansion: \(e^x=\frac{x^0}{0!}+\frac{x^1}{1!} +\frac{x^2}{2!}+\cdots =1+\frac{x^1}{1!}+\frac{x^2}{2!}+\cdots\)

\[\begin{aligned} M_X^{(1)}(t) &= e^{\lambda(e^t-1)}\lambda e^t=\lambda e^te^{\lambda(e^t-1)} \\ \\ \\ M_X^{(2)}(t) &= (e^{\lambda(e^t-1)}(\lambda e^t)+(e^{\lambda(e^t-1)}\lambda e^t)(\lambda e^t) \\ &= \lambda e^t[e^{\lambda e^t-1)}+e^{\lambda(e^t-1)}\lambda e^t] \\ &= \lambda e^t[e^{\lambda(e^t-1)}(1+\lambda e^t)] \\ \\ \\ E(X) &= M_X^{(1)}(0) \\ &= \lambda e^0e^{\lambda(e^0-1)}\lambda e^t=\lambda \\ \\ \\ E(X^2) &= M_X^{(2)}(0) \\ &= \lambda e^0[e^{\lambda(e^0-1)}(1+\lambda e^0)] \\ &= \lambda e^0[e^{\lambda(e^0-1)}(1+\lambda e^0] \\ &= \lambda(1+\lambda) \\ &= \lambda+\lambda^2 \\ \\ \\ \mu &= E(X) \\ &= \lambda \\ \sigma^2 &= E(X^2)-E(X) \\ &= \lambda+\lambda^2-\lambda^2 \\ &= \lambda \end{aligned}\]

31.5 Maximum Likelihood Estimator

Let \(x_1,x_2,\ldots,x_n\) be a random sample drawn from a Poisson Distribution with parameter \(\lambda\).

31.5.1 Likelihood Function

\[\begin{aligned} L(\theta) &= L(x_1,x_2,\ldots,x_n|\theta) \\ &= p(x_1|\theta)p(x_2|\theta)\cdots p(x_n|\theta) \\ &= \frac{e^{-\lambda}\lambda^{x_1}}{x_1!}\cdot\frac{e^{-\lambda}\lambda^{x_2}}{x_2!} \cdot \ \cdots\ \cdot \frac{e^{-\lambda}\lambda^{x_n}}{x_n!} \\ &= \frac{e^{-n\lambda}\lambda^{\sum\limits_{i=1}^{n}x_i}}{\prod\limits_{i=1}^{n}x_1!} \end{aligned}\]

31.5.2 Log-likelihood

\[\begin{aligned} \ell(\lambda) &= \ln(L(\lambda)) \\ &= \ln\Big[\frac{e^{-n\lambda}\lambda^{\sum\limits_{i=1}^{n}x_i}}{\prod\limits_{i=1}^{n}x_1!}\Big] \\ &= \ln(e^{-n\lambda})+\ln\Big(\lambda^{\sum\limits_{i=1}^{n}x_i}\Big) - \ln\Big(\prod\limits_{i=1}^{n}x_1!\Big) \\ &= -n\lambda + \sum\limits_{i=1}^{n}x_i\ln\Big(\lambda)-\ln(\prod\limits_{i=1}^{n}x_1!\Big) \end{aligned}\]

31.5.3 MLE for \(\lambda\)

\[\begin{aligned} \frac{d\ell}{d\lambda} &= -n - \frac{\sum\limits_{i=1}^{n}x_i}{\lambda} - 0 \\ &= \frac{\sum\limits_{i=1}^{n}x_i}{\lambda} - n \\ \\ \\ 0 &= \frac{\sum\limits_{i=1}^{n}x_i}{\lambda}-n\\ \Rightarrow \frac{\sum\limits_{i=1}^{n}x_i}{\lambda} &= n\\ \Rightarrow \sum\limits_{i=1}^{n}x_i &= n\lambda\\ \Rightarrow \frac{\sum\limits_{i=1}^{n}x_i}{n} &= \lambda \end{aligned}\]

so \(\displaystyle \hat\lambda=\frac{\sum\limits_{i=1}^{n}x_i}{n}\) is the Maximum Likelihood Estimator for \(\lambda\).

31.6 Theorems for the Poisson Distribution

31.6.1 Derivation of the Poisson Distribution

Suppose \(X\) is a Binomial random variable in all respects but has an infinite (non-fixed) number of trials, each with probability of success \(p\).

Then the pdf of \(X\) is \(P(x)=\frac{e^{-\lambda}\lambda^x}{x!}\)

Proof:

For an infinite number of trials we take \(\lim\limits_{n\rightarrow\infty}{n\choose x}p^x(1-p)^{n-x}\).

By rewriting \(p\) in terms of \(\mu\ (\mu=np\ \Rightarrow p=\frac{\mu}{n})\) we get

\[\begin{aligned} \lim\limits_{n\rightarrow\infty}{n\choose x}\frac{\mu}{n}^x(1-\frac{\mu}{n})^{n-x} &= \lim\limits_{n\rightarrow\infty}{n\choose x} \frac{\mu}{n}^x(1-\frac{\mu}{n})^n(1-\frac{\mu}{n})^{-x} \\ &= \lim\limits_{n\rightarrow\infty}\bigg(\frac{n(n-1)\cdots(n-x+1)(n-x)!}{x!(n-x)!}\bigg) \mu^x\frac{1}{n^x}(1-\frac{\mu}{n})^n(1-\frac{\mu}{n})^{-x} \\ &= \lim\limits_{n\rightarrow\infty}\bigg(\frac{n(n-1)\cdots(n-x+1)}{x!}\bigg) \mu^x\frac{1}{n^x}(1-\frac{\mu}{n})^n(1-\frac{\mu}{n})^{-x} \\ &= \frac{\mu^x}{x!}\lim\limits_{n\rightarrow\infty}\bigg(\frac{n(n-1)\cdots(n-x+1)}{n^x}\bigg) \frac{1}{n^x}(1-\frac{\mu}{n})^n(1-\frac{\mu}{n})^{-x} \\ ^{[1]}&= \frac{\mu^x}{x!}\lim\limits_{n\rightarrow\infty}\bigg(\frac{n(n-1)\cdots(n-x+1)}{n^x}\bigg) \lim\limits_{n\rightarrow\infty}\frac{1}{n^x}(1-\frac{\mu}{n})^n \lim\limits_{n\rightarrow\infty}(1-\frac{\mu}{n})^{-x} \\ &= \frac{\mu^x}{x!}\cdot 1\cdot e^{-\mu}\cdot 1 \\ &= \frac{e^{-\mu}\mu^x}{x!} \end{aligned}\]

  1. \(\lim\limits_{n\rightarrow\infty}(1-\frac{x}{n})^n=e^-x\)

Traditionally, we use \(\lambda\) in place of \(\mu\) for the Poisson distribution, giving us the desired result. (Wackerly, III, and Scheaffer 2002, 125)

31.6.2 Validity of the Distribution

\[\sum\limits_{x=0}^{\infty}\frac{e^{-\lambda}\lambda^x}{x!} = 1\]

Proof:

\[\begin{aligned} \sum\limits_{x=0}^{\infty}\frac{e^{-\lambda}\lambda^x}{x!} &= e^{-\lambda}\sum\limits_{x=0}^{\infty}\frac{\lambda^x}{x!} \\ &= e^{-\lambda}\bigg(\frac{\lambda^0}{0!}+ \frac{\lambda^1}{1!}+\frac{\lambda^2}{2!}+\cdots\bigg) \\ &= e^{-\lambda}\cdot e^\lambda \\ &= e^0 \\ &= 1 \end{aligned}\]

  1. Taylor Series Expansion: \(e^x=\frac{x^0}{0!}+\frac{x^1}{1!}+\frac{x^2}{2!}+\cdots\)

31.6.3 Sum of Poisson Random Variables

Let \(X_1,X_2,\ldots,X_n\) be independent random variables from a Poisson distribution with parameter \(\lambda_i,\ i=1,2,\ldots,n\); that is, \(X_i\sim\)Poisson\((\lambda_i)\).

Let \(Y=\sum\limits_{i=1}^{n}X_i\).

Then \(Y\sim\)Poisson\((\sum\limits_{i=1}^{n}\lambda_i)\).\

Proof:

\[\begin{aligned} M_Y(t) &= E(e^{tY}) \\ &= E(e^{t(X_1+X_2+\cdots+X_n)} \\ &= E(e^{tX_1}e^{tX_2}\cdots e^{tX_n}) \\ &= E(e^{tX_1})E(e^{tX_2})\cdots E(e^{tX_n}) \\ &= e^{\lambda_1(e^t-1)}e^{\lambda_2(e^t-1)}\cdots e^{\lambda_n(e^t-1)} \\ &= e^{(\lambda_1+\lambda_2+\cdots+\lambda_n)(e^t-1)} \\ &= e^{(e^{t-1})\sum\limits_{i=1}^{n}\lambda_i} \end{aligned}\]

Which is the mgf of a Poisson random variable with parameter \(\sum\limits_{i=1}^{n}\lambda_i\). Thus \(Y\sim\)Poisson\((\sum\limits_{i=1}^{n}\lambda_i)\).

References

Wackerly, Dennis D., William Mendenhall III, and Richard L. Scheaffer. 2002. Mathematical Statistics with Applications. 6th ed. Duxbury.