9 Chi-Square Distribution

9.1 Probability Distribution Function

A random variable \(X\) is said to have a Chi-Square Distribution with parameter \(\nu\) if its probability distribution function is

\[f(x) = \left\{ \begin{array}{ll} \frac{x^{\frac{\nu}{2}-1}e^{-\frac{x}{2}}} {2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} & 0<x,\ 0<\nu\\ 0 & otherwise \end{array} \right. \]

\(\nu\) is commonly referred to as the degrees of freedom.

9.2 Cumulative Distribution Function

The cumulative distribution function for the Chi-Square Distribution cannot be written in closed form. It’s integral form is expressed as \[ F(x) = \left\{ \begin{array}{ll} \displaystyle\int\limits_{0}^{x} \frac{t^{\frac{\nu}{2}-1}e^{-\frac{t}{2}}} {2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} dt & 0<x,\ 0<\nu\\\\ 0 & otherwise \end{array} \right. \]

The graphs on the top and bottom depict the Chi-Square probability distribution and cumulative distribution functions, respectively, for $\nu=4,7,10$.  As $\nu$ gets larger, the distribution becomes flatter with thicker tails.

(#fig:ChiSquare_Distribution)The graphs on the top and bottom depict the Chi-Square probability distribution and cumulative distribution functions, respectively, for \(\nu=4,7,10\). As \(\nu\) gets larger, the distribution becomes flatter with thicker tails.

9.3 Expected Values

\[\begin{aligned} E(X) &= \int\limits_{0}^{\infty}x\frac{x^{\frac{\nu}{2}-1}e^{-\frac{x}{2}}} {2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})}dx \\ &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \int\limits_{0}^{\infty}x\cdot x^{\frac{\nu}{2}-1}e^{-\frac{x}{2}}dx \\ &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \int\limits_{0}^{\infty}x^{\frac{\nu}{2}}e^{-\frac{x}{2}}dx \\ ^{[1]} &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \Big[\Gamma\Big(\frac{\nu}{2}+1\Big)2^{\frac{\nu}{2}+1}\Big] \\ &= \frac{\Gamma(\frac{\nu}{2}+1)2^{\frac{\nu}{2}+1}} {2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \\ &= \frac{\frac{\nu}{2}\Gamma(\frac{\nu}{2})2^{\frac{\nu}{2}+1}} {2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \\ &= \frac{2\nu}{2} \\ &= \nu \end{aligned}\]

  1. \(\int\limits_{0}^{\infty}x^{\alpha-1}e^{-\frac{x}{\beta}}dx = \beta^\alpha\Gamma(\alpha)\)

\[\begin{aligned} E(X^2) &= \int\limits_{0}^{\infty}x^2\frac{x^{\frac{\nu}{2}-1}e^{-\frac{x}{2}}} {2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})}dx \\ &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \int\limits_{0}^{\infty}x^2\cdot x^{\frac{\nu}{2}-1}e^{-\frac{x}{2}}dx \\ &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \int\limits_{0}^{\infty}x^{\frac{\nu}{2}+1}e^{-\frac{x}{2}}dx \\ ^{[1]} &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \Big[\Gamma(\frac{\nu}{2}+2)2^{\frac{\nu}{2}+2}\Big] \\ &= \frac{\Gamma\Big(\frac{\nu}{2}+2\Big)2^{\frac{\nu}{2}+2}} {2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \\ &= \frac{(\frac{\nu}{2}+1)\Gamma(\frac{\nu}{2}+1)2^{\frac{\nu}{2}+2}} {2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \\ &= \frac{\Big(\frac{\nu}{2}+1\Big)\frac{\nu}{2}\Gamma(\frac{\nu}{2})2^{\frac{\nu}{2}+2}} {2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \\ &= \Big(\frac{\nu}{2}+1\Big)\frac{\nu}{2}\cdot 2^2=2\Big(\frac{\nu}{2}+1\Big)\nu \\ &= (\nu+2)\nu=\nu^2+2\nu \end{aligned}\]

  1. \(\int\limits_{0}^{\infty}x^{\alpha-1}e^{-\frac{x}{\beta}}dx = \beta^\alpha\Gamma(\alpha)\)

\[\begin{aligned} \mu &= E(X) \\ &= \nu \\ \\ \\ \sigma^2 &= E(X^2)-E(X)^2 \\ &= \nu^2+2\nu-\nu^2 \\ &= 2\nu \end{aligned}\]

9.4 Moment Generating Function

\[\begin{aligned} M_X(t) &= E(e^{tX}) \\ &= \int\limits_{0}^{\infty}e^{tx} \frac{x^{\frac{\nu}{2}-1}e^{-\frac{x}{2}}} {2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})}dx \\ &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \int\limits_{0}^{\infty}e^{tx}\cdot x^{\frac{\nu}{2}-1}e^{-\frac{x}{2}}dx \\ &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \int\limits_{0}^{\infty}x^{\frac{\nu}{2}-1} e^{tx}e^{-\frac{x}{2}}dx \\ &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \int\limits_{0}^{\infty}x^{\frac{\nu}{2}-1} e^{tx-\frac{x}{2}}dx \\ &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \int\limits_{0}^{\infty}x^{\frac{\nu}{2}-1} e^{\frac{2tx}{2}-\frac{x}{2}}dx \\ &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \int\limits_{0}^{\infty}x^{\frac{\nu}{2}-1} e^{-\frac{2tx-x}{2}}dx \\ &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \int\limits_{0}^{\infty}x^{\frac{\nu}{2}-1} e^{-x\frac{-2t+1}{2}}dx \\ &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \int\limits_{0}^{\infty}x^{\frac{\nu}{2}-1} e^{-x\frac{1-2t}{2}}dx \\ &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \int\limits_{0}^{\infty}x^{\frac{\nu}{2}-1} e^{\frac{-x}{\frac{2}{1-2t}}}dx \\ ^{[1]} &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \Big[\Big(\frac{2}{1-2t}\Big)^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})\Big]\\ &= \frac{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} {2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})(1-2t)^{\frac{\nu}{2}}} \\ &= \frac{1}{(1-2t)^{\frac{\nu}{2}}} \\ &= (1-2t)^{-\frac{\nu}{2}} \end{aligned}\]

  1. \(\int\limits_{0}^{\infty}x^{\alpha-1}e^{-\frac{x}{\beta}}dx = \beta^\alpha\Gamma(\alpha)\)

\[\begin{aligned} M_X^{(1)}(t) &= -\frac{\nu}{2}(1-2t)^{-\frac{\nu}{2}-1}(-2) \\ &= \frac{2\nu}{2}(1-2t)^{-\frac{\nu}{2}-1} \\ &= \nu(1-2t)^{-\frac{\nu}{2}-1} \\ \\ \\ M_X^{(2)}(t) &= (-\frac{\nu}{2}-1)\nu(1-2t)^{-\frac{\nu}{2}-2}(-2) \\ &= (\frac{2\nu}{2}+2)\nu(1-2t)^{-\frac{\nu}{2}-2} \\ &= (\nu+2)\nu)(1-2t)^{-\frac{\nu}{2}-2} \\ &= (\nu^2+2\nu)(1-2t)^{-\frac{\nu}{2}-2}\\ \\ \\ M_X^{(1)}(0) &= \nu(1-2\cdot 0)^{-\frac{\nu}{2}-1} \\ &= \nu(1-0)^{-\frac{\nu}{2}-1} \\ &= \nu(1)^{-\frac{\nu}{2}-1} \\ &= \nu \\ M_X^{(2)}(0) &= (\nu^2+2\nu)(1-2\cdot 0)^{-\frac{\nu}{2}-2} \\ &= (\nu^2+2\nu)(1-0)^{-\frac{\nu}{2}-2} \\ &= (\nu^2+2\nu)(1)^{-\frac{\nu}{2}-2} \\ &= (\nu^2+2\nu) \\ \\ \\ E(X) &= M_X^{(1)}(0) \\ &= \nu\\ \\ \\ E(X^2) &= M_X^{(2)}(0) \\ &= (\nu^2+2\nu) \\ \\ \\ \mu &= E(X) \\ &= \nu \\ \sigma^2 &= E(X^2)-E(X)^2 \\ &= \nu^2+2\nu-\nu^2 \\ &= 2\nu \end{aligned}\]

9.5 Maximum Likelihood Function

Let \(x_1,x_2,\ldots,x_n\) be a random sample from a Chi-square distribution with parameter \(\nu\).

9.5.1 Likelihood Function

\[\begin{aligned} L(\theta) &= f(x_1|\theta) f(x_2|\theta) \cdots f(x_n|\theta) \\ &= \frac{x_1^{\nu/2-1}e^{-x_1/2}}{2^{\nu/2}\Gamma\big(\frac{\nu}{2}\big)} \cdot \frac{x_2^{\nu/2-1}e^{-x_2/2}}{2^{\nu/2}\Gamma\big(\frac{\nu}{2}\big)} \cdots \frac{x_n^{\nu/2-1}e^{-x_n/2}}{2^{\nu/2}\Gamma\big(\frac{\nu}{2}\big)} \\ &= \prod\limits_{i=1}^{n}\frac{x_i^{\nu/2-1}e^{-x_i/2}} {2^{\nu/2}\Gamma\big(\frac{\nu}{2}\big)} \\ &= \bigg(2^{\nu/2}\Gamma\Big(\frac{\nu}{2}\Big)\bigg) \prod\limits_{i=1}^{n}x_i^{\nu/2-1}e^{-x_i/2} \\ &= \bigg(2^{\nu/2}\Gamma\Big(\frac{\nu}{2}\Big)\bigg) \cdot \exp\bigg\{ \sum\limits_{i=1}^{n}\frac{x_i}{2} \bigg\} \cdot \prod\limits_{i=1}^{n}x_i^{\nu/2-1} \\ &= \bigg(2^{\nu/2}\Gamma\Big(\frac{\nu}{2}\Big)\bigg) \cdot \exp\bigg\{ \frac{1}{2}\sum\limits_{i=1}^{n}x_i \bigg\} \cdot \prod\limits_{i=1}^{n}x_i^{\nu/2-1} \end{aligned}\]

9.5.2 Log-likelihood Function

\[\begin{aligned} \ell(\theta) &= \ln\big(L(\theta)\big) \\ &= \ln\Bigg[ \bigg(2^{\nu/2}\Gamma\Big(\frac{\nu}{2}\Big)\bigg) \cdot \exp\bigg\{ \frac{1}{2}\sum\limits_{i=1}^{n}x_i \bigg\} \cdot \prod\limits_{i=1}^{n}x_i^{\nu/2-1} \Bigg] \\ &= \ln\Bigg[ \bigg( 2^{\nu/2}\Gamma \Big( \frac{\nu}{2} \Big) \bigg) \Bigg] + \ln\Bigg( \exp\bigg\{ \frac{1}{2}\sum\limits_{i=1}^{n}x_i \bigg\} \Bigg) + \ln\bigg(\prod\limits_{i=1}^{n}x_i^{\nu/2-1}\bigg) \\ &= -n \ln\bigg( 2^{\nu/2}\Gamma \Big( \frac{\nu}{2} \Big) \bigg) + \frac{1}{2}\sum\limits_{i=1}^{n}x_i + \bigg( \frac{\nu}{2}-1 \bigg) \ln\bigg( \prod\limits_{i=1}^{n}x_i \bigg) \\ &= -n\bigg( \ln(2^{\nu/2}) + \Gamma\Big(\frac{\nu}{2}\Big) \bigg) + \frac{1}{2}\sum\limits_{i=1}^{n}x_i + \bigg( \frac{\nu}{2}-1 \bigg) \sum\limits_{i=1}^{n}\ln x_i \\ &= -n\bigg(\frac{\nu}{2} \ln 2 + \ln \Gamma\Big( \frac{\nu}{2} \Big) \bigg) + \frac{1}{2}\sum\limits_{i=1}^{n}x_i + \bigg( \frac{\nu}{2}-1 \bigg) \sum\limits_{i=1}^{n}\ln x_i \\ &= -\frac{n\nu}{2} \ln 2 - n\ln \Gamma\Big( \frac{\nu}{2} \Big) + \frac{1}{2}\sum\limits_{i=1}^{n}x_i + \bigg( \frac{\nu}{2}-1 \bigg) \sum\limits_{i=1}^{n}\ln x_i \end{aligned}\]

9.5.3 MLE for \(\nu\)

\[\begin{aligned} \frac{d\ell}{d\nu} &= -\frac{n}{2} \ln 2 - \frac{n}{\Gamma\big(\frac{\nu}{2}\big)} \Gamma^\prime\Big(\frac{\nu}{2}\Big) \cdot \frac{1}{2} + 0 + \frac{1}{2} \sum\limits_{i=1}^{n}\ln x_i \\ &= -\frac{n}{2} \ln 2 - \frac{n}{2\Gamma\big(\frac{\nu}{2}\big)} \Gamma^\prime\Big(\frac{\nu}{2}\Big) + \frac{1}{2} \sum\limits_{i=1}^{n}\ln x_i \\ \\ \\ 0 &= -\frac{n}{2} \ln 2 - \frac{n}{2\Gamma\big(\frac{\nu}{2}\big)} \Gamma^\prime\Big(\frac{\nu}{2}\Big) + \frac{1}{2} \sum\limits_{i=1}^{n}\ln x_i\\ \Rightarrow \frac{n}{2} \ln 2 - \frac{1}{2}\sum\limits_{i=1}^{n}\ln x_i &= -\frac{n}{2\Gamma\big(\frac{\nu}{2}\big)} \Gamma^\prime\Big(\frac{\nu}{2}\Big)\\ \Rightarrow n\ln 2 - \sum\limits_{i=1}^{n}\ln x_i &= -\frac{n}{\Gamma\big(\frac{\nu}{2}\big)} \Gamma^\prime\Big(\frac{\nu}{2}\Big)\\ \Rightarrow \frac{\sum\limits_{i=1}^{n}\ln x_i - n\ln 2}{n} &= \frac{\Gamma^\prime\big(\frac{\nu}{2}\big)}{\Gamma\big(\frac{\nu}{2}\big)} \end{aligned}\]

Due to the complexity of the Gamma function in this equation, no solution can be developed for \(\nu\) in closed form. Thus, we have to rely on numerical methods to obtain a solution to the equation and find the maximum likelihood estimator.

9.6 Theorems for the Chi-Square Distribution

9.6.1 Validity of the Distribution

\[ \int\limits_{0}^{\infty}\frac{x^{\frac{\nu}{2}-1}e^{-\frac{x}{2}}} {2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})}dx = 1 \]

Proof:

\[\begin{aligned} \int\limits_{0}^{\infty}\frac{x^{\frac{\nu}{2}-1}e^{-\frac{x}{2}}} {2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})}dx &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \int\limits_{0}^{\infty}x^{\frac{\nu}{2}-1}e^{-\frac{x}{2}}dx \\ ^{[1]} &= \frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})} \Big[2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})\Big] \\ &= \frac{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})}{2^{\frac{\nu}{2}} \Gamma(\frac{\nu}{2})} \\ &= 1 \end{aligned}\]

  1. \(\int\limits_{0}^{\infty}x^{\alpha-1}e^{-\frac{x}{\beta}}dx = \beta^\alpha\Gamma(\alpha)\)

9.6.2 Sum of Chi-Square Random Variables

Let \(X_1 , X_2 , \ldots , X_n\) be independent Chi-Square random variables with parameter \(\nu_i\), that is \(X_i\sim\chi^2(\nu_i),\ i=1,2,\ldots,n\).

Suppose \(Y = \sum\limits_{i=1}^{n}X_i\). Then \(Y\sim\chi^2(\sum\limits_{i=1}^{n}\nu_i)\).

Proof:

\[\begin{aligned} M_Y(t) &= E(e^{tY}=E(e^{t(X_1+X_2+\cdots+X_n}) \\ &= E(e^{tX_1}e^{tX_2}\cdots e^{tX_n}) \\ &= E(e^{tX_1})E(e^{tX_2})\cdots E(e^{tX_n}) \\ &= (1-2t)^{-\frac{\nu_1}{2}}(1-2t)^{-\frac{\nu_2}{2}}\cdots (1-2t)^{-\frac{\nu_n}{2}} \\ &= (1-2t)^{\sum\limits_{i=1}^{n}\nu_i} \end{aligned}\]

Which is the mgf of a Chi-Square random variable with parameter \(\sum\limits_{i=1}^{n}\nu_i\).
Thus \(Y\sim\chi^2\bigg(\sum\limits_{i=1}^{n}\nu_i\bigg)\).

9.6.3 Multiple of a Chi-Square Random Variable

Let \(X\) be a Chi-Square random variable with parameter \(\nu\), that is \(X\sim\chi^2(\nu),\ i=1,2,\ldots,n\).

Suppose \(Y = c \cdot X\). Then \(Y\sim Gamma(\frac{\nu}{2}, 2 \cdot c)\).

Proof:

\[\begin{aligned} M_Y(t) &= E(e^{tY})=E(e^{tcX}) \\ &= (1-2tc)^{-\frac{\nu}{2}} \\ &= (1-2c \cdot t)^{-\frac{\nu}{2}} \end{aligned}\]

Which is the mgf of a Gamma distributed variable with parameters \(\alpha = \frac{\nu}{2}\) and \(\beta = 2c\). Thus, \(Y\sim Gamma(\frac{\nu}{2}, 2 \cdot c)\).

9.6.4 Square of a Standard Normal Random Variable

If \(Z\sim N(0,1)\), then \(Z^2\sim\chi^2(1)\).

Proof: \[\begin{aligned} M_{Z^2}(t) &= E(e^{tZ^2}) \\ &= \int\limits_{-\infty}^{\infty}e^{tz^2}\frac{1}{\sqrt{2\pi}} e^{-\frac{z^2}{2}}dz \\ &= \frac{1}{\sqrt{2\pi}}\int\limits_{-\infty}^{\infty}e^{tz^2} e^{-\frac{z^2}{2}}dz \\ &= \frac{1}{\sqrt{2\pi}}\int\limits_{-\infty}^{\infty} e^{-\frac{z^2}{2}(-2t+1)}dz \\ &= \frac{1}{\sqrt{2\pi}}\int\limits_{-\infty}^{\infty} e^{-\frac{z^2}{2}(1-2t)}dz \\ ^{[1]} &= \frac{2}{\sqrt{2\pi}}\int\limits_{0}^{\infty} e^{-\frac{z^2}{2}(1-2t)}dz \\ ^{[2]} &= \frac{2}{\sqrt{2\pi}}\int\limits_{0}^{\infty}e^{-u} \frac{\sqrt{2}u^{-\frac{1}{2}}}{2(1-2t)^{\frac{1}{2}}}du \\ &= \frac{2\sqrt{2}}{2\sqrt{2\pi}(1-2t)^{\frac{1}{2}}} \int\limits_{0}^{\infty}e^{-u}u^{-\frac{1}{2}}du \\ &= \frac{2\sqrt{2}}{2\sqrt{2\pi}(1-2t)^{\frac{1}{2}}} \int\limits_{0}^{\infty}u^{\frac{1}{2}-1}e^{-u}du \\ ^{[3]} &= \frac{1}{\sqrt{\pi}(1-2t)^{\frac{1}{2}}}\Gamma(\frac{1}{2}) \\ &= \frac{\sqrt{\pi}}{\sqrt{\pi}(1-2t)^{\frac{1}{2}}} \\ &= \frac{1}{(1-2t)^{\frac{1}{2}}}=(1-2t)^{-\frac{1}{2}} \\ \end{aligned}\]

  1. \(\int\limits_{-\infty}^{\infty}f(x)dx = 2\int\limits_{0}^{\infty}f(x)dx\) when f(x) is an even function ()
  2. Let \(u=\frac{z^2}{2}(1-2t) \ \ \ \ \Rightarrow z=\frac{\sqrt{2}u^{\frac{1}{2}}}{(1-2t)^{\frac{1}{2}}}\)
        So \(dz=\frac{\sqrt{2}u^{-\frac{1}{2}}} {2(1-2t)^{\frac{1}{2}}}\)
  3. \(\int\limits_{0}^{\infty}x^{\alpha-1}e^{-\frac{x}{\beta}}dx =\beta^\alpha\Gamma(\alpha)\)

Which is the mgf of a Chi-Square random variable with 1 degree of freedom. Thus \(Z^2\sim\chi^2(1)\).