Statistical optimization for passive scalar transport: maximum entropy production vs maximum Kolmogorov-Sinay entropy

We derive rigorous results on the link between the principle of maximum entropy production and the principle of maximum Kolmogorov-Sinai entropy using a Markov model of the passive scalar diffusion called the Zero Range Process. We show analytically that both the entropy production and the Kolmogorov-Sinai entropy seen as functions of f admit a unique maximum denoted fmaxEP and fmaxKS. The behavior of these two maxima is explored as a function of the system disequilibrium and the system resolution N. The main result of this article is that fmaxEP and fmaxKS have the same Taylor expansion at _rst order in the deviation of equilibrium. We find that fmaxEP hardly depends on N whereas fmaxKS depends strongly on N. In particular, for a fixed difference of potential between the reservoirs, fmaxEP (N) tends towards a non-zero value, while fmaxKS (N) tends to 0 when N goes to infinity. For values of N typical of that adopted by Paltridge and climatologists we show that fmaxEP and fmaxKS coincide even far from equilibrium. Finally, we show that one can find an optimal resolution N_ such that fmaxEP and fmaxKS coincide, at least up to a second order parameter proportional to the non-equilibrium uxes imposed to the boundaries.


I. INTRODUCTION
A major difficulty in the modeling of nonlinear geophysical or astrophysical processes is the taking into account of all the relevant degrees of freedom.For example, fluid motions obeying Navier-Stokes equations usually require of the order of N = Re 9/4 modes to faithfully describe all scales between the injection scale and the dissipative scale (?).In atmosphere, or ocean, where the Reynolds number exceeds 10 9 , this amount to N = 10 20 , a number too large to be handled by any existing computers (?).The problem is even more vivid in complex systems such as planetary climate, where the coupling of lito-bio-cryo-sphere with ocean and atmosphere increases the number of degrees of freedom beyond any practical figure.This justifies the long historical tradition of parametrization and statistical model reduction, to map the exact equations describing the system onto a set of simpler equations involving few degrees of freedom.The price to pay is the introduction of free parameters, describing the action of discarded degrees of freedom, that needs to be prescribed.When the number of free parameters is small, their prescription can be successfully done empirically through calibrating experiments or by a posteriori tuning (?).When the number of parameters is large, such as in climate models where it reaches several hundreds (?), such empirical procedure is inapplicable, because it is impossible to explore the whole parameter space.In that respect, it is of great interest to explore an alternative road to parametrization via application of a statistical optimization principle, such as minimizing or maximizing of a suitable cost functional.As discussed by (?) and (?), this strategy usually leads to closed reduced equations with adjustable parameters in the closure appearing as weights in the cost functional and can be computed explicitly.A famous example in climate is given by a principle of maximum entropy production (MEP) that allowed (?) to derive the distribution of heat and clouds at the Earth surface with reasonable accuracy, without any parameters and with a model of a dozen of degrees of freedom (boxes).Since then, refinements of Paltrige model have been suggested to increase its generality and range of prediction (?).MEP states that a stationary nonequilibrium system chooses its final state in order to maximize the entropy production as is explain in (?).Rigorous justifications of its application have been searched using e.g.information theory (?) without convincing success.More recently, we have used the analogy of the climate box model of Paltridge with the asymmetric exclusion Markov process (ASEP) to establish numerically a link between the MEP and the principle of maximum Kolmogorov-Sinai entropy (MKS)(?).The MKS principle is a relatively new concept which extends the classical results of equilibrium physics (?).This principle applied to Markov Chains provides an approximation of the optimal diffusion coefficient in transport phenomena (?) or simulates random walk on irregular lattices (?).It is therefore a good candidate for a physically relevant cost functional in passive scalar modeling.
The goal of the present paper is to derive rigorous results on the link between MEP and MKS using a Markov model of the passive scalar diffusion called the Zero Range Process (?).We find that there exists an optimal resolution N * such that both maxima coincide to second order in the distance from equilibrium.The application of this principle to passive scalar transport parametrization is therefore expected to provide both the value of the optimal flux, and of the optimal number of degrees of freedom (resolution) to describe the system.This suggests that the MEP and MKS principle may be unified when the Kolmogorov-Sinai entropy is defined on opportunely coarse grained partitions.

II. FROM PASSIVE SCALAR EQUATION TO ZRP MODEL
The equation describing the transport of a passive scalar like temperature in a given velocity field u(x, t) reads: with appropriate boundary conditions, or equivalently, in non-dimensionnal form: where κ, Re and P r are respectively the molecular diffusivity, the Reynolds and the Prandtl number.To solve this equation, one must know both the velocity field and the boundary conditions, and use as many number of modes as necessary to describe all range of scales up to the scales at which molecular diffusivity takes place i.e. roughly (ReP r) 3/2 modes, where Re is the Reynolds number of the convective flow, and P r is its Prandtl number.In geophysical flows, this number is too large to be handled even numerically (?).Moreover, in typical climate studies, the velocity flow is basically unknown as it must obey a complicated equation involving the influence of all the relevant climate components.In order to solve the equation, one must necessarily prescribe the heat flux f = −uT + κ∇T .The idea of Paltridge was then to discretize the passive scalar equation in boxes and prescribe the heat flux f i(i+1) between boxes i and i+1 by maximizing the associated thermodynamic entropy production Ṡ = i f i(i+1) ( 1 Ti+1 − 1 Ti ).Here, we slightly modify the Paltridge discretization approximation to make it amenable to rigorous mathematical results on Markov Chains.For simplicity, we stick to a one dimensional case (corresponding to boxes varying only in latitude) and impose the boundary conditions through two reservoirs located at each end of the chain (mimicking the solar heat flux at pole and equator).We consider a set of N boxes that can contain an arbitrary number n ∈ N of particles.We then allow transfer of particles in between two adjacent boxes via decorrelated jumps (to the right or to the left) following a 1D Markov dynamics governed by a coupling with the two reservoirs imposing a difference of chemical potential at the ends.The resulting process is called the Zero Range Process (?).The different jumps are described as follow.At each time step a particle can jump right with probability pw n or jump left with probability qw n where w n is a parameter depending of the number of particles inside the box.Physically it represents the interactions between particles.At the edges of the lattice the probability rules are different: At the left edge a particule can enter with probability α and exit with probability γw n whereas at the right edge a particle can exit with probability βw n and enter with probability δ.Choices of different w n give radically different behaviors.For example w n = 1 + b/n where b ≥ 0 described condensation phenomena (?) whereas w 1 = w et w n = 1 if n ≥ 2 has been used to modeled road traffic.We will consider in this article the particular case where w = 1 by convenience of calculation.Moreover without loss of generality we will take p ≥ q which corresponds to a particle flow from the left to the right and note f = p − q .After a sufficiently long time the system reaches a non-equilibrium steady state.The interest of this toy model is that it is simple enough so that exact computations are analytically tractable.
Taking the continuous limit of this process, it may be checked that the fugacity z, which is a quantity related to the average particle density (see 9 below), of stationary solutions of a system consisting of boxes of size 1 N follows the continuous equation (?) : corresponding to a stationary solution of a non-dimensional passive scalar equation with non-dimensionnal velocity f and a non-dimensionnal diffusivity 1 ReP r = 1 2N .Therefore, the fugacity of the Zero Range Process is a passive scalar obeying a convective-diffusion equation, with advection velocity controlled by the probability to jump to the right or to the left, and diffusivity controlled by the number of boxes: the larger the number of boxes (the finer the resolution) the smaller the diffusivity.This observation illuminates the well-known observation that the numerical diffusion of a discrete model of diffusion is inversely proportionnal to the resolution.The parameter f controls the regime: f = 0 corresponds to a purely conductive regime whereas the larger f the more convective the regime.In the sequel, we calculate the entropy production and the Kolmogorov-Sinai entropy function of f .These two quantities reach a maximum noted respectively f max EP and f max KS .The MEP principle (resp.the MKS principle) states that the system will choose f = f max EP (resp f = f max KS ).We will show first of all in this article that numerically f max EP ≈ f max KS even far from equilibrium for a number of boxes N roughly corresponding to the resolution taken by ? in his climate model.This result is similar to what we found for the ASEP model (?) and thus gives another example of a system in which the two principles are equivalent.Moreover we will see analytically that f max EP and f max KS have the same behavior in first order in the difference of the chemical potentials between the two reservoirs for N large enough.These results provide a better understanding of the relationship between the MEP and the MKS principles.

III. NOTATIONS AND USEFUL PRELIMINARY RESULTS
This Markovian Process is a stochastic process with a infinite number of states in bijection with N N .In fact, each state can be written n = (n 1 , n 2 , ...., n N ) where n i is the number of particule lying in site i.We call P n the stationary probability to be in state n.In order to calculate this probability it is easier to use a quantum formalism than the Markovian formalism as explained in the following articles (??).
The probability to find m particles in the site k is equal to: where Z k is the analogue of the grand canonical repartition function and z k is the fugacity between 0 and 1. Moreover We can show that the probability P over the states is the tensorial product of the probability p k over the boxes: Thus events (n k = m) and (n k = m ) for k = k are independent and so: So finally Moreover, with the Hamiltonian equation found from the quantum formalism we can find the exact values of z k function of the system parameters: and the flux of particles c: Finally, the stationary density is related to the fugacity by the relation: A.
Entropy Production For a system subject to internal forces X i and associated fluxes J i the macroscopic entropy production is well known (?) and takes the form: The Physical meaning of this quantity is a measure of irreversibility: the larger σ the more irreversible the system.
In the case of the zero range process irreversibility is created by the fact that p = q.We will parametrize this irreversibility by the parameter f = p − q and we will take p + q = 1.In the remaining of the paper, we take, without loss of generality, p ≤ q which corresponds to a flow from left to right.Moreover, the only flux to be considered is here the flux of particules c and the associated force is due to the gradient of the density of particules ρ : X = ∇ log ρ (?).Thus, when the stationary state is reached ie when c is constant: Thus, according to Eqs. ( 7), ( 8), ( 9) and (10) when N tends to +∞ we obtain: Because f ≥ 0 the entropy production is positive if and only if ρ 1 ≥ ρ N iff z 1 ≥ z N .This is physically coherent because fluxes are in the opposite direction of the gradient.We remark that if f = 0 then σ(f ) = 0.Moreover, when f increases ρ 1 (f ) decreases and ρ 2 (f ) increases till they take the same value.Thus it exists f , large enough, for which σ(f ) = 0. Between these two values of f the entropy production has at least one maximum.

B. Kolmogorov-Sinai Entropy
There are several ways to introduce the Kolmogorov-Sinai entropy which is a mathematical quantity introduced by Kolmogorov and developed by famous mathematician as Sinai and Billingsley (?).Nevertheless, for a Markov process we can give it a simple physical interpretation: the Kolmogorov-Sinai entropy is the time derivative of the Jaynes entropy (entropy over the path).
For a Markov Chain we have thus: where µ stat = µ istat i = 1...N is the stationary measure and where the p ij are the transition probabilities.
Thus the Kolmogorov-Sinai entropy takes the following form: For the Zero Range Process ,we show in appendix that it can be written as: IV. RESULTS We will start first by pointing to some interesting properties of f max EP and f max KS , then by presenting numerical experiments on the ZRP model and finally concluding with some analytical computations.
Let us first note that for N ,α,β,γ,δ fixed the entropy production as well as the Kolmogorov-Sinai entropy seen as functions of f admit both a unique maximum.When N tends to infinity and f = 0, using Eq.( 7) (i.e. the symmetric case), we find that z 1 = α γ and z N = δ β .Thus, the system is coupled with two reservoirs with respective chemical potential α γ (left) and δ β (right).For α γ = δ β the system is out of equilibrium.We assume, without loss of generality, z 1 ≥ z N which corresponds to a flow from left to right.As a measure of deviation from equilibrium we take s = z 1 −z N : the larger s, the more density fluxes we expect into the system.First we remark that f max EP hardly depends on N whereas f max KS depends strongly on N .This is easily understood because σ depends only on z 1 and z N whereas h KS depends on all the z i .Moreover, the profile of the z i depends strongly on N .In particular, for a fixed difference of potential between the reservoirs , f max EP (N ) tends towards a non-zero value, while f max KS (N ) tends to 0 when N goes to infinity.Moreover, f max EP and f max KS coincide even far from equilibrium for N corresponding to the choice of ?N ≈ 10 ∼ 100.For N fixed, as large as one wants, and for all , as small as one wants, it exists ν such that for all s ∈ [0; ν] |f max EP − f max KS | ≤ .These observations are confirmed by the results presented in Figures 1 and 3 where EP and KS are calculated using Eq. ( 7) and ( 15) for s = 0.13 and three different partitions: N = 20 N = 100 et N = 1000.The figure shows that f max EP and f max KS coincide with good approximation for N = 20 and N = 100.But then when N increases f max KS (N ) tends to 0 whereas f max EP (N ) tends to a non-zero value.
In Figure 2 we represent the Entropy Production (top) and KS Entropy (bottom) function of f for N = 1000 and for three value of s: s = 0.13; s = 0.2; s = 0.04.This supports the claim that for N fixed, we could tried different values of s such that s ∈ [0; ν] |f max EP − f max KS | ≤ .Figure 3 shows that ∆f max is minimum when the system is close to equilibrium whereas the further the system is from equilibrium (when s increases) the more ∆f max increases.Moreover the optimal resolution where f max EP and f max KS coincide is approximately 10 ∼ 100.Then ∆f max is maximum at N = 500 and s = 0.05.∆f max is obvously linear in s, for small values of s, but the behaviour with N is more complicated.
Such numerical investigations suggest to understand why f max KS (N ) and f max EP (N ) have different behaviors function of N , and why for N large enough f max KS and f max EP have the same behavior of first order in the deviation from equilibrium measured by the parameter s.We will see that we can get a precise answer to such questions by doing calculations and introducing a sort of Hydrodynamics approximation.

A. Taylor expansion
From Eq. ( 15) it is apparent that f max KS depends on N whereas from Eq. ( 10) we get that f max EP hardly depends on N .Indeed there is a difference between f max EP and f max KS , i.e. a difference between the two principles for the Zero Range Process.Nevertheless, we have seen numerically that there is a range of N , namely N ≈ 10 ∼ 100 for which the maxima fairly coincide.Using Eqs.(15) (7) (11) we compute analytically the Taylor expansion of f max EP and f max KS in s.We will show the main result: f max EP and f max KS have the same Taylor expansion in first order in s for N large enough.Their Taylor expansions are different up to the second order in s but it exists an N , i.e. a resolution, such that f max EP and f max KS coincident up to the second order.Let us start by computing f max KS .It does not depend of the constant terms of h KS in Eq.( 15) and therefore we need only concern ourselves with : (16) Using Eq.( 7), the expression of H(f, N, α, γ, β, δ) takes an easy form.To simplify the calculations, we restrict the space of parameter by assuming α + γ = 1 and β + δ = 1 and we parametrize the deviation from equilibrium by the parameter s = α − δ.Moreover let's note a = 1 N .Thus, we have H(f, N, α, γ, β, δ) = H(f, a, α, s).In order to know the Taylor expansion to the first order in s of f max KS we develop H(f, a, α, s) up to the second order in f ; i.e. we have H(f, a, α, s) = C + Bf + Af 2 + o(f 2 ) then we find f max KS = −B/2A that we will develop in power of s.This is consistent if we assume f a.After some tedious but straightforward calculations, we get at the first order in s and so, We repeat the same procedure starting from Eq.( 11) and we obtain: Thus, since a = 1 N 1 the behaviour of f max KS (s) and f max EP (s) is the same for s small enough.We remark that we can strictly find the same result by solving the hydrodynamics continuous approximation given by Eq. ( 3).This equation is a classical convection-diffusion equation.We remark that, by varying f , we change the convective behavior: f = 0 corresponds to a purely diffusive regime whereas by increasing f we enhance the role of convection.If the system is near equilibrium then f max EP ≈ f max KS ≈ 0 and the system is purely diffusive.When the system is out of equilibrium f max EP and f max KS are different from 0 and corresponds to an (optimal) trade-off between purely diffusive and convective behavior.One can verify this numerically: We first calculate the exact values of the Entropy Production function of f using Eq. ( 7) and the Kolmogorov-Sinai Entropy function of f using Eqs.(7) (15).Then we approximate these two curves with a cubic spline approximation in order to find f max EP and f max KS .In order to find the optimal resolution N * we can go one step further by expanding f max EP and f max KS up to the second order in s: Thus, f max EP and f max KS coincide in second order in s iff a satisfies the quadratic equation: This equation has a unique positive solution because the leading coefficient is positive for s small enough (4α − 6α 2 + 6α 3 − 4s + 3α 2 s) ≥ 0 and the constant term is negative −(1 − α) ≤ 0. We remark that the optimal resolution N * = 1 a * depends on the parameters of the system namely on the degree of non-equilibrium.This fact can be the explanation for two well known issues in climate/weather modeling.First, it explains that, when downgrading or upgrading the resolution of convection models, the relevant parameters must be changed as they depend on the grid size.Second, it suggests that if the resolution is well tuned to represent a particular range of convective phenomena, it might fail in capturing the dynamics out of this range: since finer grids are needed to better represent deep convection phenomena, the deviations between model and observations observed in the distribution of extreme convective precipitation may be due to an inadequacy of the grid used.

V. CONCLUSION
We have shown how a simple 1D Markov Process, the Zero Range Process, can be used to obtain rigorous results on the problem of parametrization of the passive scalar transport problem, relevant to many geophysical applications including temperature distribution in climate modeling.Using this model, we have derived rigorous results on the link between a principle of maximum entropy production and the principle of maximum Kolmogorov-Sinai entropy using a Markov model of the passive scalar diffusion called the Zero Range Process.The Kolmogorov-Sinai entropy seen as function of the convective velocity admit a unique maximum.We show analytically that both have the same Taylor expansion at the first order in the deviation from equilibrium.The behavior of these two maxima is explored as a function of the resolution N (equivalent to the number of boxes, in the box approximation).We found that for a fixed difference of potential between the reservoirs , the maximal convective velocity predicted by the maximum entropy production principle tends towards a non-zero value, while the maximum predicted using Kolmogorov-Sinai entropy tends to 0 when N goes to infinity.For values of N typical of those adopted by climatologists (N ≈ 10 ∼ 100), we show that the two maxima nevertheless coincide even far from equilibrium.Finally, we show that there is an optimal resolution N * such that the two maxima coincide to second order in s, a parameter proportional to the non-equilibrium fluxes imposed to the boundaries.The fact that the optimal resolution depends on the intensity of the convective phenomena to be represented, points to new interesting research avenues, e.g. the introduction of convective models with adaptive grids optimized with maximum entropy principles on the basis of the convective phenomena to be represented.On another hand, the application of this principle to passive scalar transport parametrization is therefore expected to provide both the value of the optimal flux, and of the optimal number of degrees of freedom (resolution) to describe the system.It would be interesting to apply it to more realistic passive scalar transport problem, to see if it yield to model that can be numerically handled (i.e.corresponding to a number of bow that is small enough to be handled by present computers).In view of applications to atmospheric convection, it would be interesting to apply this procedure to the case of an active scalar, coupled with a Navier-Stokes equation for the velocity.In such a case, the role of f will be played by the turbulent subgrid Reynolds stresses.The heat fluxes and N * will be fixed by the coarse-graining length, and the optimization procedure will in principle provide the optimum subgrid Reynolds stresses at a given resolution N .Moreover, by imposing coincidence of MKS and MEP, one could get both the Reynolds stresses, heat fluxes and the optimum resolution.Moreover, on a theoretical side, it will be interesting to study whether for general dynamical systems, there exists a smart way to coarse grain the Kolmogorov-Sinai entropy such that its properties coincide with the thermodynamic entropy production.This will eventually justify the use of the MEP principle and explain the deviations as well as the different representations of it due to the dependence of the dynamic (Kolmogorov Smirnov, Tsallis, Jaynes) entropies on the kind of partition adopted.

FIG. 1 .
FIG. 1. Entropy Production calculate using 11 (left) and KS Entropy calculate using 7 and 15 (right) function of f for s = 0.13 and respectively N = 20 N = 100 et N = 1000

FIG. 4 .
FIG. 4. fmax EP (left) and fmax KS (right) function of s for α = 0.5 and N = 100.We remark than fmax KS and fmax EP have both a linear behaviour with slope respectively 0.48 and 0.49 which is really close to 1 4α = 0.5

N 1 (
z k + a(1 − z k )) and by deriving subject to a we show that: