The statistic edit

Let ${\displaystyle L}$  be the likelihood function which depends on a univariate parameter ${\displaystyle \theta }$  and let ${\displaystyle x}$  be the data. The score ${\displaystyle U(\theta )}$  is defined as

${\displaystyle U(\theta )={\frac {\partial \log L(\theta \mid x)}{\partial \theta }}.}$ 

The Fisher information is^[6]

${\displaystyle I(\theta )=-\operatorname {E} \left[\left.{\frac {\partial ^{2}}{\partial \theta ^{2}}}\log f(X;\theta )\,\right|\,\theta \right]\,,}$ 

where ƒ is the probability density.

The statistic to test ${\displaystyle {\mathcal {H}}_{0}:\theta =\theta _{0}}$  is ${\displaystyle S(\theta _{0})={\frac {U(\theta _{0})^{2}}{I(\theta _{0})}}}$ 

which has an asymptotic distribution of ${\displaystyle \chi _{1}^{2}}$ , when ${\displaystyle {\mathcal {H}}_{0}}$  is true. While asymptotically identical, calculating the LM statistic using the outer-gradient-product estimator of the Fisher information matrix can lead to bias in small samples.^[7]

Note on notation edit

Note that some texts use an alternative notation, in which the statistic ${\displaystyle S^{*}(\theta )={\sqrt {S(\theta )}}}$  is tested against a normal distribution. This approach is equivalent and gives identical results.

As most powerful test for small deviations edit

${\displaystyle \left({\frac {\partial \log L(\theta \mid x)}{\partial \theta }}\right)_{\theta =\theta _{0}}\geq C}$ 

where ${\displaystyle L}$  is the likelihood function, ${\displaystyle \theta _{0}}$  is the value of the parameter of interest under the null hypothesis, and ${\displaystyle C}$  is a constant set depending on the size of the test desired (i.e. the probability of rejecting ${\displaystyle H_{0}}$  if ${\displaystyle H_{0}}$  is true; see Type I error).

The score test is the most powerful test for small deviations from ${\displaystyle H_{0}}$ . To see this, consider testing ${\displaystyle \theta =\theta _{0}}$  versus ${\displaystyle \theta =\theta _{0}+h}$ . By the Neyman–Pearson lemma, the most powerful test has the form

${\displaystyle {\frac {L(\theta _{0}+h\mid x)}{L(\theta _{0}\mid x)}}\geq K;}$ 

Taking the log of both sides yields

${\displaystyle \log L(\theta _{0}+h\mid x)-\log L(\theta _{0}\mid x)\geq \log K.}$ 

The score test follows making the substitution (by Taylor series expansion)

${\displaystyle \log L(\theta _{0}+h\mid x)\approx \log L(\theta _{0}\mid x)+h\times \left({\frac {\partial \log L(\theta \mid x)}{\partial \theta }}\right)_{\theta =\theta _{0}}}$ 

and identifying the ${\displaystyle C}$  above with ${\displaystyle \log(K)}$ .

Relationship with other hypothesis tests edit

If the null hypothesis is true, the likelihood ratio test, the Wald test, and the Score test are asymptotically equivalent tests of hypotheses.^[8][9] When testing nested models, the statistics for each test then converge to a Chi-squared distribution with degrees of freedom equal to the difference in degrees of freedom in the two models. If the null hypothesis is not true, however, the statistics converge to a noncentral chi-squared distribution with possibly different noncentrality parameters.

A more general score test can be derived when there is more than one parameter. Suppose that ${\displaystyle {\widehat {\theta }}_{0}}$  is the maximum likelihood estimate of ${\displaystyle \theta }$  under the null hypothesis ${\displaystyle H_{0}}$  while ${\displaystyle U}$  and ${\displaystyle I}$  are respectively, the score vector and the Fisher information matrix. Then

${\displaystyle U^{T}({\widehat {\theta }}_{0})I^{-1}({\widehat {\theta }}_{0})U({\widehat {\theta }}_{0})\sim \chi _{k}^{2}}$ 

asymptotically under ${\displaystyle H_{0}}$ , where ${\displaystyle k}$  is the number of constraints imposed by the null hypothesis and

${\displaystyle U({\widehat {\theta }}_{0})={\frac {\partial \log L({\widehat {\theta }}_{0}\mid x)}{\partial \theta }}}$ 

and

${\displaystyle I({\widehat {\theta }}_{0})=-\operatorname {E} \left({\frac {\partial ^{2}\log L({\widehat {\theta }}_{0}\mid x)}{\partial \theta \,\partial \theta '}}\right).}$ 

This can be used to test ${\displaystyle H_{0}}$ .

The actual formula for the test statistic depends on which estimator of the Fisher information matrix is being used.^[10]

In many situations, the score statistic reduces to another commonly used statistic.^[11]

In linear regression, the Lagrange multiplier test can be expressed as a function of the F-test.^[12]

When the data follows a normal distribution, the score statistic is the same as the t statistic.^{[clarification needed]}

When the data consists of binary observations, the score statistic is the same as the chi-squared statistic in the Pearson's chi-squared test.

• Fisher information
• Uniformly most powerful test
• Score (statistics)
• Sup-LM test

• Buse, A. (1982). "The Likelihood Ratio, Wald, and Lagrange Multiplier Tests: An Expository Note". The American Statistician. 36 (3a): 153–157. doi:10.1080/00031305.1982.10482817.
• Godfrey, L. G. (1988). "The Lagrange Multiplier Test and Testing for Misspecification : An Extended Analysis". Misspecification Tests in Econometrics. New York: Cambridge University Press. pp. 69–99. ISBN 0-521-26616-5.
• Ma, Jun; Nelson, Charles R. (2016). "The superiority of the LM test in a class of econometric models where the Wald test performs poorly". Unobserved Components and Time Series Econometrics. Oxford University Press. pp. 310–330. doi:10.1093/acprof:oso/9780199683666.003.0014. ISBN 978-0-19-968366-6.
• Rao, C. R. (2005). "Score Test: Historical Review and Recent Developments". Advances in Ranking and Selection, Multiple Comparisons, and Reliability. Boston: Birkhäuser. pp. 3–20. ISBN 978-0-8176-3232-8.