Journal cover Journal topic
Nonlinear Processes in Geophysics An interactive open-access journal of the European Geosciences Union
Journal topic

Journal metrics

Journal metrics

  • IF value: 1.129 IF 1.129
  • IF 5-year value: 1.519 IF 5-year 1.519
  • CiteScore value: 1.54 CiteScore 1.54
  • SNIP value: 0.798 SNIP 0.798
  • SJR value: 0.610 SJR 0.610
  • IPP value: 1.41 IPP 1.41
  • h5-index value: 21 h5-index 21
  • Scimago H index value: 48 Scimago H index 48
Volume 16, issue 1 | Copyright
Nonlin. Processes Geophys., 16, 57-64, 2009
https://doi.org/10.5194/npg-16-57-2009
© Author(s) 2009. This work is distributed under
the Creative Commons Attribution 3.0 License.

  06 Feb 2009

06 Feb 2009

Information theoretic measures of dependence, compactness, and non-gaussianity for multivariate probability distributions

A. H. Monahan1 and T. DelSole2 A. H. Monahan and T. DelSole
  • 1School of Earth and Ocean Sciences, University of Victoria, Victoria, BC, Canada and Canadian Institute for Advanced Research Earth System Evolution Program, Canada
  • 2Department of Atmospheric, Oceanic, and Earth Sciences, George Mason University, Fairfax, Virginia, and Center for Ocean-Land-Atmosphere Studies, Calverton, Maryland, USA

Abstract. A basic task of exploratory data analysis is the characterisation of "structure" in multivariate datasets. For bivariate Gaussian distributions, natural measures of dependence (the predictive relationship between individual variables) and compactness (the degree of concentration of the probability density function (pdf) around a low-dimensional axis) are respectively provided by ordinary least-squares regression and Principal Component Analysis. This study considers general measures of structure for non-Gaussian distributions and demonstrates that these can be defined in terms of the information theoretic "distance" (as measured by relative entropy) between the given pdf and an appropriate "unstructured" pdf. The measure of dependence, mutual information, is well-known; it is shown that this is not a useful measure of compactness because it is not invariant under an orthogonal rotation of the variables. An appropriate rotationally invariant compactness measure is defined and shown to reduce to the equivalent PCA measure for bivariate Gaussian distributions. This compactness measure is shown to be naturally related to a standard information theoretic measure of non-Gaussianity. Finally, straightforward geometric interpretations of each of these measures in terms of "effective volume" of the pdf are presented.

Publications Copernicus
Download
Citation
Share