Nonlinear Processes in Geophysics www.nonlin-processes-geophys.net 1023-5809 1607-7946 10 6 2003 10.5194/npg-10-477-2003 http://www.nonlin-processes-geophys.net/10/477/2003/ http://www.nonlin-processes-geophys.net/10/477/2003/npg-10-477-2003.html http://www.nonlin-processes-geophys.net/10/477/2003/npg-10-477-2003.pdf 477 491 0000-00-00 A comparison of assimilation results from the ensemble Kalman Filter and a reduced-rank extended Kalman Filter X. Zang P. Malanotte-Rizzoli Department of Earth, Atmospheric and Planetary Sciences, Massachusetts Institute of Technology, USA The goal of this study is to compare the performances of the ensemble Kalman filter and a reduced-rank extended Kalman filter when applied to different dynamic regimes. Data assimilation experiments are performed using an eddy-resolving quasi-geostrophic model of the wind-driven ocean circulation. By changing eddy viscosity, this model exhibits two qualitatively distinct behaviors: strongly chaotic for the low viscosity case and quasi-periodic for the high viscosity case. In the reduced-rank extended Kalman filter algorithm, the model is linearized with respect to the time-mean from a long model run without assimilation, a reduced state space is obtained from a small number (100 for the low viscosity case and 20 for the high viscosity case) of leading empirical orthogonal functions (EOFs) derived from the long model run without assimilation. Corrections to the forecasts are only made in the reduced state space at the analysis time, and it is assumed that a steady state filter exists so that a faster filter algorithm is obtained. The ensemble Kalman filter is based on estimating the state-dependent forecast error statistics using Monte Carlo methods. The ensemble Kalman filter is computationally more expensive than the reduced-rank extended Kalman filter.The results show that for strongly nonlinear case, chaotic regime, about 32 ensemble members are sufficient to accurately describe the non-stationary, inhomogeneous, and anisotropic structure of the forecast error covariance and the performance of the reduced-rank extended Kalman filter is very similar to simple optimal interpolation and the ensemble Kalman filter greatly outperforms the reduced-rank extended Kalman filter. For the high viscosity case, both the reduced-rank extended Kalman filter and the ensemble Kalman filter are able to significantly reduce the analysis error and their performances are similar. For the high viscosity case, the model has three preferred regimes, each with distinct energy levels. Therefore, the probability density of the system has a multi-modal distribution and the error of the ensemble mean for the ensemble Kalman filter using larger ensembles can be larger than with smaller ensembles.