Nonlinear Processes in Geophysics www.nonlin-processes-geophys.net 1023-5809 1607-7946 13 4 2006 10.5194/npg-13-449-2006 http://www.nonlin-processes-geophys.net/13/449/2006/ http://www.nonlin-processes-geophys.net/13/449/2006/npg-13-449-2006.html http://www.nonlin-processes-geophys.net/13/449/2006/npg-13-449-2006.pdf 449 466 2006-09-12 Statistical characteristics of surrogate data based on geophysical measurements V. Venema victor.venema@uni-bonn.de S. Bachner H. W. Rust C. Simmer Meteorologisches Institut, Universität Bonn, Germany Potsdam Institute for Climate Impact Research, Germany In this study, the statistical properties of a range of measurements are compared with those of their surrogate time series. Seven different records are studied, amongst others, historical time series of mean daily temperature, daily rain sums and runoff from two rivers, and cloud measurements. Seven different algorithms are used to generate the surrogate time series. The best-known method is the iterative amplitude adjusted Fourier transform (IAAFT) algorithm, which is able to reproduce the measured distribution as well as the power spectrum. Using this setup, the measurements and their surrogates are compared with respect to their power spectrum, increment distribution, structure functions, annual percentiles and return values. It is found that the surrogates that reproduce the power spectrum and the distribution of the measurements are able to closely match the increment distributions and the structure functions of the measurements, but this often does not hold for surrogates that only mimic the power spectrum of the measurement. However, even the best performing surrogates do not have asymmetric increment distributions, i.e., they cannot reproduce nonlinear dynamical processes that are asymmetric in time. Furthermore, we have found deviations of the structure functions on small scales. Bardet, J.-M., Lang, G., Oppenheim, G., Philippe, A., and Taqqu, M. S.: Generators of long-range dependent processes: A survey, in: Theory and Applications of Long-Range Dependence, edited by: Oppenheim, G., Taqqu, M. S., and Doukha, P., Birkhäuse, Boston, USA, 579–623, 2003. Beran, J.: Statistics for Long-Memory Processes, Monographs on Statistics and Applied Probability, Chapman & Hall, New York, USA, 1994. Beran, J., Bhansali, R. J., and Ocker, D.: On unified model selection for stationary and nonstationary short- and long-memory autoregressive processes, Biometrika, 85, 921–934, 1998. Bisaglia, L.: Model selection for long-memory models, Quaderni di Statistica, 4, 2002. Brockwell, P. J. and Davis, R. A.: Time series: theory and methods, Springer series in statistics, Springer, Berlin, Germany, 1991. Christakos, G.: Random field models in earth sciences, Academic Press, San Diego, USA, 474 p., ISBN: 0-12-174230-X, 1992. Davis, A., Marshak, A., Cahalan, R., and Wiscombe, W.: The LANDSAT scale break in stratocumulus as a three-dimensional radiative transfer effect: implications for cloud remote sensing, J. Atmos. Sci., 54, 241–260, 1997. Davis, A. B., Marshak, A., Gerber, H., and Wiscombe, W. J.: Horizontal structure of marine boundary layer clouds from centimeter to kilometer scales, J. Geophys. Res., 104, 6123–6144, 1999. Coles, S.: An introduction to statistical modeling of extreme values, Springer series in statistics, Springer, Berlin, Germany, 2001. DWD, Deutscher Wetterdienst: http://www.dwd.de/en/FundE/Klima/KLIS, Offenbach, Germany, 2006. GRDC, Global Runoff Data Centre of the Federal Institute of Hydrology: http://grdc.bafg.de, Koblenz, Germany, 2006. Evans, K. F. and Wiscombe, W. J.: An algorithm for generating stochastic cloud fields from radar profile statistics, Atmos. Res., 72, 263–289, 2004. Frisch, U.: Turbulence, the legacy of A.N. Kolmogorov, Cambridge University Press, UK, ISBN: 0521457130, 1995. Gençai, R., Selçuk, F., and Ulugülyagci, A.: EVIM: A Software Package for Extreme Value Analysis in MATLAB, Studies in Nonlinear Dynamics and Econometrics 5, 213–239, 2001. Kugiumtzis, D.: Test your surrogate data before you test for nonlinearity, Phys. Rev. E, 60, 2808–2816, 1999. Hannan, E. J. and Quinn, B. G.: The determination of the order of an autoregression, J. Roy. Statist. Soc., 41, 190–195, 1979. Haslett, J. and Raftery, A. E.: Space-time modelling with long-memory dependence: Assessing Ireland's wind power resource, J. Appl. Statist., 38, 1–50, 1989. Kallache, M., Rust, H., and Kropp, J.: Trend Assessment: Applications for Hydrology and Climate, Nonlin. Processes Geophys., 2, 201–210, 2005. Kantz, H. and Schreiber, T.: Nonlinear time series analysis, Cambridge Nonlinear Science Series No. 7, Cambridge University Press, Cambridge UK, 1997. Koutsoyiannis, D.: The Hurst phenomenon and fractional Gaussian noise made easy, Hydrol. Sci. J., 47, 573–595, 2002. Kuhn, Th.: The structure of scientific revolutions, Chicago University Press, USA, 1963. Kylling, A., Webb, A. R., Kift, R., et al.: Spectral actinic flux in the lower troposphere: measurement and 1-D simulations for cloudless, broken cloud and overcast situations, Atmos. Chem. Phys., 5, 1975–1997, 2005. Lehmann, A.: Die Säkulare Klimareihe von Potsdam, in: Klimastatusbericht 2001, DWD, Offenbach a.M., Germany, 227–239, 2001. Lohre, M., Sibbertsen P., and Könnig, T.: Modeling water flow of the Rhine River using seasonal long memory, Water Resour. Res., 39, 1132, doi:10.1029/2002WR001697, 2003. Hurst, H. E.: Long-term storage capacity of reservoirs, Trans. Am. Soc. Civil Eng., 116, 770–799, 1951. Mandelbrot, B. B.: The Fractal Geometry of Nature, W H Freeman, New York, USA, ISBN: 0716711869, 460 p., 1982. Masters, F. and Gurley, K. R.: Non-Gaussian Simulation: Cumulative Distribution Function Map-Based Spectral Correction, J. Eng. Mech., 1418–1428, 2003. Montanari, A., Rosso, R., and Taqqu, M. S.: Fractional differenced ARIMA models applied to hydrologic time series: Identification, estimation and simulation, Water Resour. Res., 33, 1035, 97WR00043, 1997. Molz, F. J., Rajaram, H., and Lu, S.: Stochastic fractal-based models of heterogeneity in subsurface hydrology: origins, applications, limitations, and future research questions, Rev. Geophys., 42, RG1002/2004, 2004. Pincus, R., Hannay, C., and Evans, K. F.: The accuracy of determining three-dimensional radiative transfer effects in cumulus clouds using ground-based profiling instruments, J. Atmos. Sc., 62, 2284–2293, 2005. Radziejewski, M., Bardossy, A., and Kundzewicz, S.: Detection of change in river flow using phase randomization, Hydrol. Sci. J., 45, 547–558, 2000. Rust, H.: http://www.pik-potsdam.de/~hrust/tools.html, 2006. Scheirer, R. and Macke, A.: On the accuracy of the independent column approximation in calculating the downward fluxes in the UVA, UVB and PAR spectral ranges, J. Geophys. Res., 106, 14 301–14 312, 2001. Scheirer, R. and Schmidt, S.: CLABAUTAIR: a new algorithm for retrieving three-dimensional cloud structure from airborne microphysical measurements, Atmos. Chem. Phys., 5, 2333–2340, 2005. Schreiber, T. and Schmitz, A.: Improved surrogate data for nonlinearity tests, Phys. Rev. Lett, 77, 635–638, 1996. Schreiber, Th.: Constrained randomization of time series data, Phys. Rev. Lett., 80, 2105–2108, 1998. Schreiber, T. and Schmitz, A.: Surrogate time series, Physica D, 142, 346–382, 2000. Theiler, J., Eubank, S., Longtin, A., Galdrikian, B., and Farmer, J. D.: Testing for nonlinearity in time series: the method of surrogate data, Physica D, 58, 77–94 1992. Theiler, J. and Prichard, D.: Constrained-realization Monte-Carlo method for hypothesis testing, Physica D, 94, 221–235, 1996. Timmer, J. and König, M.. On generating power law noise, A&A, 300, 707–710, 1995. Turcotte, D. L.: Fractals and Chaos in Geology and Geophysics, Cambridge University Press, U.K., ISBN: 0521567335, 412, 1997. Venema, V., Meyer, S., Gimeno García, S., Kniffka, A., Simmer, C., Crewell, S., Löhnert, U., Trautmann, U., and Macke, A.: Surrogate cloud fields generated with the Iterative Amplitude Adapted Fourier Transform algorithm, Tellus 58A, 104–120, 2006a. Venema, V., Ament, F., and Simmer, C.: A Stochastic Iterative Amplitude Adjusted Fourier Transform Algorithm with improved accuracy, Nonlin. Processes Geophys., 13, 321–328,, 2006b. Venema, V.: http://www.meteo.uni-bonn.de/mitarbeiter/venema/themes/surrogates/iaaft/statistics, 2006. Venema, V.: An evolutionary search algorithm to generate 3D cloud fields with measured cloud boundary statistics, report, http://www.meteo.uni-bonn.de/victor/articles/2003/2003_report_cloud_boundaries_evol_search.pdf, 2003. Wilson, J., Lovejoy, S., and Schertzer, D.: Continuous multiplicative cascade models of rain and clouds. Scaling, fractals and non-linear variability in geophysics, edited by: Schertzer, D. and Lovejoy, S., 185–208, Kluwer, Dordrecht, The Netherlands, 1991.