Multivariate t-distribution

Summary

In statistics, the multivariate t-distribution (or multivariate Student distribution) is a multivariate probability distribution. It is a generalization to random vectors of the Student's t-distribution, which is a distribution applicable to univariate random variables. While the case of a random matrix could be treated within this structure, the matrix t-distribution is distinct and makes particular use of the matrix structure.

Multivariate t
Notation
Parameters location (real vector)
scale matrix (positive-definite real matrix)
(real) represents the degrees of freedom
Support
PDF
CDF No analytic expression, but see text for approximations
Mean if ; else undefined
Median
Mode
Variance if ; else undefined
Skewness 0

Definition edit

One common method of construction of a multivariate t-distribution, for the case of   dimensions, is based on the observation that if   and   are independent and distributed as   and   (i.e. multivariate normal and chi-squared distributions) respectively, the matrix   is a p × p matrix, and   is a constant vector then the random variable   has the density[1]

 

and is said to be distributed as a multivariate t-distribution with parameters  . Note that   is not the covariance matrix since the covariance is given by   (for  ).

The constructive definition of a multivariate t-distribution simultaneously serves as a sampling algorithm:

  1. Generate   and  , independently.
  2. Compute  .

This formulation gives rise to the hierarchical representation of a multivariate t-distribution as a scale-mixture of normals:   where   indicates a gamma distribution with density proportional to  , and   conditionally follows  .

In the special case  , the distribution is a multivariate Cauchy distribution.

Derivation edit

There are in fact many candidates for the multivariate generalization of Student's t-distribution. An extensive survey of the field has been given by Kotz and Nadarajah (2004). The essential issue is to define a probability density function of several variables that is the appropriate generalization of the formula for the univariate case. In one dimension ( ), with   and  , we have the probability density function

 

and one approach is to use a corresponding function of several variables. This is the basic idea of elliptical distribution theory, where one writes down a corresponding function of   variables   that replaces   by a quadratic function of all the  . It is clear that this only makes sense when all the marginal distributions have the same degrees of freedom  . With  , one has a simple choice of multivariate density function

 

which is the standard but not the only choice.

An important special case is the standard bivariate t-distribution, p = 2:

 

Note that  .

Now, if   is the identity matrix, the density is

 

The difficulty with the standard representation is revealed by this formula, which does not factorize into the product of the marginal one-dimensional distributions. When   is diagonal the standard representation can be shown to have zero correlation but the marginal distributions are not statistically independent.

A notable spontaneous occurrence of the elliptical multivariate distribution is its formal mathematical appearance when least squares methods are applied to multivariate normal data such as the classical Markowitz minimum variance econometric solution for asset portfolios.[2]

Cumulative distribution function edit

The definition of the cumulative distribution function (cdf) in one dimension can be extended to multiple dimensions by defining the following probability (here   is a real vector):

 

There is no simple formula for  , but it can be approximated numerically via Monte Carlo integration.[3][4][5]

Conditional Distribution edit

This was developed by Muirhead [6] and Cornish.[7] but later derived using the simpler chi-squared ratio representation above, by Roth[1] and Ding.[8] Let vector   follow a multivariate t distribution and partition into two subvectors of   elements:

 

where  , the known mean vectors are   and the scale matrix is  .

Roth and Ding find the conditional distribution   to be a new t-distribution with modified parameters.

 

An equivalent expression in Kotz et. al. is somewhat less concise.

Forming first an intermediate distribution  , the explicit conditional distribution renders as:

 

where

  Effective degrees of freedom, augmented by the disused variables.
  is the conditional mean of  
  is the Schur complement of  ; the conditional covariance.
  is the squared Mahalanobis distance of   from   with scale matrix  
 

Copulas based on the multivariate t edit

The use of such distributions is enjoying renewed interest due to applications in mathematical finance, especially through the use of the Student's t copula.[9]

Elliptical Representation edit

Constructed as an elliptical distribution,[10] take the simplest centralised case with spherical symmetry and no scaling,  , then the multivariate t-PDF takes the form

 

where   and   = degrees of freedom as defined in Muirhead[6] section 1.5. The covariance of   is

 

The aim is to convert the Cartesian PDF to a radial one. Kibria and Joarder,[11] define radial measure   and, noting that the density is dependent only on r2, we get

 

which is equivalent to the variance of  -element vector   treated as a univariate heavy-tail zero-mean random sequence with uncorrelated, yet statistically dependent, elements.

Radial Distribution edit

  follows the Fisher-Snedecor or   distribution:

 

having mean value  .  -distributions arise naturally in tests of sums of squares of sampled data after normalization by the sample standard deviation.

By a change of random variable to   in the equation above, retaining  -vector  , we have   and probability distribution

 

which is a regular Beta-prime distribution   having mean value  .

Cumulative Radial Distribution edit

Given the Beta-prime distribution, the radial cumulative distribution function of   is known:

 

where   is the incomplete Beta function and applies with a spherical   assumption.

In the scalar case,  , the distribution is equivalent to Student-t with the equivalence  , the variable t having double-sided tails for CDF purposes.

The radial distribution can also be derived via a straightforward coordinate transformation from Cartesian to spherical. A constant radius surface at   with PDF   is an iso-density surface. Given this density value, the quantum of probability on a shell of surface area   and thickness   at   is  .

The enclosed  -sphere of radius   has surface area  , and substitution into   shows that the shell has element of probability   which is equivalent to radial density function

 

which further simplifies to   where   is the Beta function.

Changing the radial variable to   returns the previous Beta Prime distribution  

To scale the radial variables without changing the radial shape function, define scale matrix   , yielding a 3-parameter Cartesian density function, ie. the probability   in volume element   is

 

or, in terms of scalar radial variable  ,

 

Radial Moments edit

The moments of all the radial variables , with the spherical distribution assumption, can be derived from the Beta Prime distribution. If   then  , a known result. Thus, for variable   we have

 

The moments of   are

 

while introducing the scale matrix   yields

 

Moments relating to radial variable   are found by setting   and   whereupon

 

Linear Combinations and Affine Transformation edit

Full Rank Transform edit

This closely relates to the multivariate normal method and is described in Kotz and Nadarajah, Kibria and Joarder, Roth, and Cornish. Starting from a somewhat simplified version of the central MV-t pdf:  , where   is a constant and   is arbitrary but fixed, let   be a full-rank matrix and form vector  . Then, by straightforward change of variables

 

The matrix of partial derivatives is   and the Jacobian becomes  . Thus

 

The denominator reduces to

 

In full:

 

which is a regular MV-t distribution.

In general if   and   has full rank   then

 

Marginal Distributions edit

This is a special case of the rank-reducing linear transform below. Kotz defines marginal distributions as follows. Partition   into two subvectors of   elements:

 

with  , means  , scale matrix  

then  ,   such that

 
 

If a transformation is constructed in the form

 

then vector  , as discussed below, has the same distribution as the marginal distribution of   .

Rank-Reducing Linear Transform edit

In the linear transform case, if   is a rectangular matrix  , of rank   the result is dimensionality reduction. Here, Jacobian   is seemingly rectangular but the value   in the denominator pdf is nevertheless correct. There is a discussion of rectangular matrix product determinants in Aitken.[12] In general if   and   has full rank   then

 
 

In extremis, if m = 1 and   becomes a row vector, then scalar Y follows a univariate double-sided Student-t distribution defined by   with the same   degrees of freedom. Kibria et. al. use the affine transformation to find the marginal distributions which are also MV-t.

  • During affine transformations of variables with elliptical distributions all vectors must ultimately derive from one initial isotropic spherical vector   whose elements remain 'entangled' and are not statistically independent.
  • A vector of independent student-t samples is not consistent with the multivariate t distribution.
  • Adding two sample multivariate t vectors generated with independent Chi-squared samples and different   values:   will not produce internally consistent distributions, though they will yield a Behrens-Fisher problem.[13]
  • Taleb compares many examples of fat-tail elliptical vs non-elliptical multivariate distributions

Related concepts edit

  • In univariate statistics, the Student's t-test makes use of Student's t-distribution
  • The elliptical multivariate-t distribution appears to arise spontaneously in linearly constrained least squares solutions involving multivariate normal source data, for example the Markowitz global minimum variance solution in financial portfolio analysis.[14][15][2] which addresses an ensemble of normal random vectors or a random matrix. It does not arise in ordinary least squares (OLS) or multiple regression with fixed dependent and independent variables which problem tends to produce well-behaved normal error probabilities.
  • Hotelling's T-squared distribution is a distribution that arises in multivariate statistics.
  • The matrix t-distribution is a distribution for random variables arranged in a matrix structure.

See also edit

References edit

  1. ^ a b Roth, Michael (17 April 2013). "On the Multivariate t Distribution" (PDF). Automatic Control group. Linköpin University, Sweden. Archived (PDF) from the original on 31 July 2022. Retrieved 1 June 2022.
  2. ^ a b Bodnar, T; Okhrin, Y (2008). "Properties of the Singular, Inverse and Generalized inverse Partitioned Wishart Distribution" (PDF). Journal of Multivariate Analysis. 99 (Eqn.20): 2389–2405.
  3. ^ Botev, Z.; Chen, Y.-L. (2022). "Chapter 4: Truncated Multivariate Student Computations via Exponential Tilting.". In Botev, Zdravko; Keller, Alexander; Lemieux, Christiane; Tuffin, Bruno (eds.). Advances in Modeling and Simulation: Festschrift for Pierre L'Ecuyer. Springer. pp. 65–87. ISBN 978-3-031-10192-2.
  4. ^ Botev, Z. I.; L'Ecuyer, P. (6 December 2015). "Efficient probability estimation and simulation of the truncated multivariate student-t distribution". 2015 Winter Simulation Conference (WSC). Huntington Beach, CA, USA: IEEE. pp. 380–391. doi:10.1109/WSC.2015.7408180.
  5. ^ Genz, Alan (2009). Computation of Multivariate Normal and t Probabilities. Lecture Notes in Statistics. Vol. 195. Springer. doi:10.1007/978-3-642-01689-9. ISBN 978-3-642-01689-9. Archived from the original on 2022-08-27. Retrieved 2017-09-05.
  6. ^ a b Muirhead, Robb (1982). Aspects of Multivariate Statistical Theory. USA: Wiley. pp. 32–36 Theorem 1.5.4. ISBN 978-0-47 1-76985-9.
  7. ^ Cornish, E A (1954). "The Multivariate t-Distribution Associated with a Set of Normal Sample Deviates". Australian Journal of Physics. 7: 531–542. doi:10.1071/PH550193.
  8. ^ Ding, Peng (2016). "On the Conditional Distribution of the Multivariate t Distribution". The American Statistician. 70 (3): 293–295. arXiv:1604.00561. doi:10.1080/00031305.2016.1164756. S2CID 55842994.
  9. ^ Demarta, Stefano; McNeil, Alexander (2004). "The t Copula and Related Copulas" (PDF). Risknet.
  10. ^ Osiewalski, Jacek; Steele, Mark (1996). "Posterior Moments of Scale Parameters in Elliptical Sampling Models". Bayesian Analysis in Statistics and Econometrics. Wiley. pp. 323–335. ISBN 0-471-11856-7.
  11. ^ Kibria, K M G; Joarder, A H (Jan 2006). "A short review of multivariate t distribution" (PDF). Journal of Statistical Research. 40 (1): 59–72. doi:10.1007/s42979-021-00503-0. S2CID 232163198.
  12. ^ Aitken, A C - (1948). Determinants and Matrices (5th ed.). Edinburgh: Oliver and Boyd. pp. Chapter IV, section 36.
  13. ^ Giron, Javier; del Castilo, Carmen (2010). "The multivariate Behrens–Fisher distribution". Journal of Multivariate Analysis. 101 (9): 2091–2102. doi:10.1016/j.jmva.2010.04.008.
  14. ^ Okhrin, Y; Schmid, W (2006). "Distributional Properties of Portfolio Weights". Journal of Econometrics. 134: 235–256.
  15. ^ Bodnar, T; Dmytriv, S; Parolya, N; Schmid, W (2019). "Tests for the Weights of the Global Minimum Variance Portfolio in a High-Dimensional Setting". IEEE Trans. on Signal Processing. 67 (17): 4479–4493.

Literature edit

  • Kotz, Samuel; Nadarajah, Saralees (2004). Multivariate t Distributions and Their Applications. Cambridge University Press. ISBN 978-0521826549.
  • Cherubini, Umberto; Luciano, Elisa; Vecchiato, Walter (2004). Copula methods in finance. John Wiley & Sons. ISBN 978-0470863442.
  • Taleb, Nassim Nicholas (2023). Statistical Consequences of Fat Tails (1st ed.). Academic Press. ISBN 979-8218248031.

External links edit

  • Copula Methods vs Canonical Multivariate Distributions: the multivariate Student T distribution with general degrees of freedom
  • Multivariate Student's t distribution