Parameter estimation for a generalized Gaussian distribution

Citation for published version (APA):Beelen, T. G. J., & Dohmen, J. J. (2015). Parameter estimation for a generalized Gaussian distribution. (CASA-report; Vol. 1540). Technische Universiteit Eindhoven.

Document status and date:Published: 01/12/2015

Document Version:Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers)

Please check the document version of this publication:

• A submitted manuscript is the version of the article upon submission and before peer-review. There can beimportant differences between the submitted version and the official published version of record. Peopleinterested in the research are advised to contact the author for the final version of the publication, or visit theDOI to the publisher's website.• The final author version and the galley proof are versions of the publication after peer review.• The final published version features the final layout of the paper including the volume, issue and pagenumbers.Link to publication

General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.

• Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal.

If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, pleasefollow below link for the End User Agreement:www.tue.nl/taverne

Take down policyIf you believe that this document breaches copyright please contact us at:openaccess@tue.nlproviding details and we will investigate your claim.

Download date: 28. Aug. 2021

EINDHOVEN UNIVERSITY OF TECHNOLOGY Department of Mathematics and Computer Science

CASA-Report 15-40 December 2015

Parameter estimation for a generalized Gaussian distribution

by

T.G.J. Beelen, J.J. Dohmen

Centre for Analysis, Scientific computing and Applications Department of Mathematics and Computer Science Eindhoven University of Technology P.O. Box 513 5600 MB Eindhoven, The Netherlands ISSN: 0926-4507

Parameter Estimation for a Generalized GaussianDistribution

T.G.J. Beelen (Eindhoven University of Technology),J.J. Dohmen (NXP Semiconductors)

December 22, 2015

1 General Introduction

In 2014 several companies, institutes and universities in Europe started a joinedproject named ‘CORTIF’ (Coexistence Of RF Transmissions In the Future). Seethe website http://cortif.xlim.fr for more details.The CORTIF partners in the Netherlands are NXP Semiconductors (NXP-NL),IMEC, Technolution, and Eindhoven University of Technology, departments EEand Mathematics & Computer Science (CASA group). In 2015 CASA started aclose co-operation with NXP-NL on the subject of so-called ‘trimmed distribu-tions’. The underlying background is as follows.As might be known NXP is a semiconductor company designing and producing alarge variety of integrated circuits (ICs) for a wide range of products. Of course,NXP aims at a high yield as well as good IC-performance. Therefore, most of theproduced ICs are measured and checked against the specifications. For some RFapplications a specific output signal of produced ICs can be measured. In case ofout-of-spec that IC can be ‘tuned’ by adapting a built-in resistor such that the ICdoes meet the specs eventually .Clearly, when looking at the statistical distribution of a large amount of such‘tuned’ ICs, this distribution will not be standard Gaussian anymore. In fact, thecorresponding probability density function has a more flat shape than in case ofstandard Gaussian. In order to optimize the yield it would be benificial to have astatistical model for the observed distribution. One of the promising approaches isto use the so-called generalized Gaussian distribution function and to estimate its3 defining parameters using the maximum likelihood technique.In this report we will investigate this approach in detail and propose a numericalfast and reliable method for computing these parameters.

1

2 Abstract

In this report we analyze and solve the problem of computing a reliable estimationof the parameters of a generalized gaussian distribution function. It is well-knownthat this problem can be formulated as a maximization problem of the maximumlikelihood function associated with the distribution function. Next this can easilybe transformed to finding the zero of a function g(β ) where β is the unknownexponent parameter of the distribution. The evaluation of g(β ) requires a fewsummations of a (large) number of randomly generated values xi from a generalizedGaussian distribution. This will be indicated by g(β ) = g(β ; xi).Among other approaches, this zero can be computed numerically using Newton’smethod. However, we show that this approach is far from trivial since g(β ) behavesin an extreme way near its zero (if existing). In fact, g(β ) is rather steep at one sideof its zero, but is almost completely flat and often (very) close to zero at the otherside. Hence, we include an extensive and detailed analysis of all formulas involvedfor a full understanding of the behaviour of g(β ).We propose a new procedure consisting of several runs where per run (k) a differentset of random values x(k)i is taken for computing the zero β (k) of g(β ; x(k)i ). Thenthe average of all these zeros β (k) is considered as a good estimate of the exponentparameter β .Finally, we show that this procedure indeed gives reliable results when applied toempirical data.

2

3 Mathematical problem description

The starting point is the paper ”Generalized normal distribution” in [1]. The prob-lem is to find a numerical reliable and accurate method for estimating the parame-ters of the following probability density function

f (x) =β

2αΓ(1/β )exp(−

(|x−µ|)

α

)β

) (1)

In the figures 1 and 2 the probability density function (1) and the cumulative densityfunction cd f are shown for α = 1 and a few values of the parameter β . It is easilyseen that in case α = 1 the major part of the x-values where tol < cd f (x)< 1− tolis in the range (−2,2) for 2≤ β ≤ 8. Here tol is a relatively small positive number,say tol ≤ 10−3.

Figure 1: Generalized Gaussian density functions with µ = 0 and α = 1.

We assume N points xi be given in the interval [U ,V ]. The points xi can be ran-domly distributed corresponding to (1).The likelihood function corresponding to f (x) is given by

L =N

∏i=1

f (xi) =

(β

2αΓ(1/β )

)N

exp

(−

N

∑i=1

(|xi−µ|)

α

)β)

= (2α)−Nβ

N(Γ(1/β )−Nexp

(−

N

∑i=1

(|xi−µ|)

α

)β) (2)

3

Figure 2: Generalized Gaussian cumulative density function with µ = 0 and α = 1.

The log 1 maximum likelihood function L is given by

L = log(L ) =−Nlog(2α)+Nlog(β )−Nlog((Γ(1/β ))−N

∑i=1

(|xi−µ|)

α

)β

(3)

In figure 3 the function L is shown for µ = 0 and a few values of α and β .The necessary conditions to maximize L are

∂L∂α

= 0,∂L∂β

= 0 and∂L∂ µ

= 0. (4)

In the rest of this document we assume that µ is known. Hence, we can ignore thecondition ∂L

∂ µ= 0.

However, for completeness, the formula for ∂L∂ µ

are given in appendix ??.

The condition ∂L∂α

= 0 gives

α = α =

(β

N

N

∑i=1|xi−µ|β

)1/β

(5)

The condition ∂L∂β

= 0 gives

∂L∂β

=1β+

Ψ(1/β )

β 2 − 1N

N

∑i=1

∣∣xi−µ

α

∣∣β log∣∣xi−µ

α

∣∣= 0 (6)

The typical behaviour of ∂L/∂β is shown in the figures (4) and (5) below. We see1Throughout this document we will indicate the ‘ln’-function by ‘log’.

4

Figure 3: The log maximum likelihood function L.

from figure (5) that ∂L/∂β for β ∈ [6.3,6.4] which corresponds to α ≈ 4.55 usingformula (5).It is easily seen from these figures that solving ∂L/∂β = 0 is not straigthforward,especially in case α > 2. This suggests that further investigation of ∂L/∂β isneeded. We will proceed with this in the next sections. First we mention theapproach in [1] for computing the maximum likelihood function L. Their mainidea is taking expression (6) for ∂L/∂β = 0 and next substituting α given in (5).Then, after multiplying with the factor β/N, we get

g(β ) = 0 where (7)

g(β ) = 1+Ψ(1/β )

β− ∑

Ni=1 |xi−µ|β log|xi−µ|

∑Ni=1 |xi−µ|β

+log(

β

N ∑Ni=1 |xi−µ|β

)β

(8)

5

Figure 4: The function ∂L/∂β .

Equation (7) is solved using Newton-Raphson’s method with

g′(β ) =− Ψ(1/β )

β 2 − Ψ′(1/β )

β 3 +1

β 2

− ∑Ni=1 |xi−µ|β (log|xi−µ|)2

∑Ni=1 |xi−µ|β

+

(∑

Ni=1 |xi−µ|β log|xi−µ|

∑Ni=1 |xi−µ|β

)2

+∑

Ni=1 |xi−µ|β log|xi−µ|

β ∑Ni=1 |xi−µ|β

−log(

β

N ∑Ni=1 |xi−µ|β

)β 2

(9)

The typical behaviour of the functions g(β ) and g′(β ) is shown in figure (6).RemarkThe zero β0 in figure 6 is computed by applying Newton-Raphson’s method usingg(β ) and g′(β ) to the interval x = [−3 : 0.01 : 5]. As usual, µ = 0. So, we have

βk+1 = βk−g(βk)/g′(βk), k ≥ 1 and β1 = 3. (10)

We used as stopping criterion:

|βk+1−βk|< eabs + erel ∗ |βk| with eabs = 1.e−15, erel = 1.e−6. (11)

6

Figure 5: The function ∂L/∂β where ∂L/∂β = 0 for β0 ≈ 6.3 and a(β0)≈ 4.55.

The numerical results are shown in the table in figureEnd of Remark

It should be noticed that this method is sensitive to the choice of the interval forthe x-values. This is shown in figure 8. Notice that this figure suggests that thefunction g(β ) has no zeros at all when the x-values are in the interval [-3 : 0.01 :3]. If so, then clearly the Newton-Raphson method will fail.In Appendix C it is shown that

limβ→∞

g(β ) = 0 (12)

4 Behaviour of the function α(β )

For convenience, we recall formula (5) for the function α(β )

α(β ) = αsum(β ) =

(β

N

N

∑i=1|xi−µ|β

)1/β

(13)

7

Figure 6: The functions g(β ) and g′(β ). The marker denotes β0 ≈ 6.4205 withg(β0) = 0 and a(β0)≈ 4.5637. Recall also figure 5

.

The typical behaviour of α(β ) is depicted in figure 9. We see that the functionα(β ) is monotonically increasing for β > 0.It is shown in (40) that the function αsum(β ) can be approximated by

αanalyt(β )≈M(

β

β +1

)1/β

, N→ ∞

where M = max1≤i≤N |xi−µ|(14)

When evaluating formulas (13) and (14) numerically (with µ = 0, |xi+1− xi| =0.001,−4 ≤ xi ≤ 4) we found |αsum(β )− αanalyt(β )| < 5e−4 for all 0 < β ≤ 10.See also figure 10. Furthermore, it is shown in Appendix D.1 that

limβ→0

α(β ) = 0 and

limβ→∞

α(β ) = M where M = max1≤i≤N |xi−µ|(15)

Notice that α(β )< M for all β > 0 and that α(β ) is quite sensitive to the choiceof the endpoint of the x-interval.

4.1 Asymptotic behaviour of g(β )

We split the function g(β ) given by (8) into 3 parts as follows:

g(β ) = g1(β )−P(β |µ)+Q(β |µ) where (16)

8

Figure 7: Table: Newton-Raphson results for g(β ) for the x-interval [−3 : 0.01 : 5].

g1(β ) = 1+Ψ(1/β )

β, (17)

P(β |µ) = ∑Ni=1

∣∣xi−µ∣∣β log

∣∣xi−µ∣∣

∑Ni=1

∣∣xi−µ∣∣β and (18)

Q(β |µ) =log( β

N ∑Ni=1

∣∣xi−µ∣∣β )

β(19)

Recalling (100)) and (5) we notice that g1(β ) = β ∗F(β ) and Q(β |µ) = log(a).It is shown in Appendix C that

limβ→∞

g1(β ) = 0, limβ→∞

P(β |µ) = log(M), and limβ→∞

Q(β |µ) = log(M),

where M = max1≤i≤N

|xi−µ|(20)

Thus,lim

β→∞

g(β ) = 0 (21)

5 Numerical experiments

5.1 Procedure for parameter estimation of an empirical pdf

We will now consider a pdf obtained from numerical simulations or from measure-ments in practice. Such a pdf will be called an empirical pdf. We assume that thispdf can be modelled by a generalized Gaussian pdf where its parameters α and β

are still unknown, but that µ is known. We will determine these parameters using

9

Figure 8: The function g(β ) for a number of x-intervals with different lengths.

the formulas given earlier. This is globally done using the procedure describedbelow. We assume the data of the pdf to be modelled are given by the set

{(x0i,y0i)|1≤ i≤ N} where

x0i ∈ [X0,min,X0,max], y0i ∈ [Y0,min,Y0,max],

mean µ0 = Σx0i/N

(22)

1. Transform the given data such that their mean is equal to 0:(set xi = x0i−µ0, 1≤ i≤ N , and set µ = 0)

2. Determine the empirical pdf E pd f based on the given data:(use, for example, the Matlab function ksdensity.m)

3. Compute the cumulative density function Ecd f corresponding to E pd f(for example, by numerical integration of E pd f using the trapezoid rule)

4. Generate a set of random variables xrnd,i using Ecd f(see Appendix I )

5. Compute the parameter β as the zero β0 of the function g(x) in (8), usingxrnd,i

10

Figure 9: The function α(β ) for β > 0 with limβ→∞

α(β ) =V = max1≤i≤N |xi−µ|.

6. Compute the parameter α = α(β0) with formula (5)

Let us now comment on this global procedure.

• Since the xrnd,i are random values, they are not sorted in general.Therefore, the magnitudes of the successive values |xrnd,i− µ|β may varyconsiderably, and might cause numerical inaccuracy when computing thesum Σ|xrnd,i−µ|β , as required in the computation of g(x).So, we suggest to first sort the values xrnd,i to increasing order and then touse them in further calculations.

• As said before (see section 3) the computation of g(x) is sensitive to the end-point M = max1≤i≤N |xrnd,i| of the x-interval. So, the computed zero β0 ofg(x) and next also α(β0) strongly depend on M.

Therefore, we propose to modify the procedure as follows

– First, carry out the steps 1 up to 3 as before

– Secondly, repeat the steps 4 and 5 a number of times k = 1, · · · ,K, andsaving the valuesin run k, step 4 :

∗ min(k)rnd = min1≤ i≤N{x(k)rnd,i} and max(k)rnd = max1≤ i≤N{x(k)rnd,i},

11

Figure 10: Difference between αsum(β ) and αanalyt(β ).

in run k, step 5:

∗ β(k)0 and α(k) = α(β

(k)0 )

– Finally, computethe mean α = 1

K Σα(k)0 , the mean β = 1

K Σβ(k)0 and

rndmin = min1≤k≤Kmin(k)rnd , rndmax = max1≤k≤Kmax(k)rnd

Remarks

• Recall that the computation of α involves the computation of Σ|xi− µ |β .This should be done as accurate as possible and preferrably not be sensitiveto the choice of random interval points.

• Notice that the interval Irnd = [rndmin, rndmax ] is the interval where all gen-erated random values xk

rnd,i obtained in all runs k are located.

12

• Therefore, an alternative approach for α might be calculating α = α(β )on the interval [rndmin, rndmax ], equidistantly subdivided by R (�N) points.However, it turned out that this α-value was much larger than that obtainedabove, and gave a worse approximation of the empirical pdf (see also nextsection).

• An alternative for the equidistant subdivision of Irnd might be using all therandom points x(k)rnd,i, after having sorted them in increasing order. Clearly,this requires saving the generated values in each run k.

5.2 Application of the estimation procedure

We want to apply the described procedure to an empirical pdf obtained when simu-lating measurements of a voltage output signal of an industrial IC design. The pdfis calculated using the Matlab function ksdensity.m ansd is shown in figure 11.

Figure 11: An empirical pdf obtained from industrial data.

By numerical integration of this pdf (using the trapezoidal method) we find theempirical cdf as in figure 12. Notice that in this figure we hardly can recognizethe curvatures due to the local peaks in the corresponding pdf. See Appendix Gshowing a similar figure in a standard Gaussian case where the peaks in the pdf areeven more pronounced than in fig.11. Once having the empirical cdf (Ecd f ) avail-able, we start the procedure described above. In the figures below a few typicalexamples are given.Per example we we have chosen 20 runs, and per run we generated 1000 randomnumbers distributed conform the Ecdf. The final values β and α for β and α are

13

Figure 12: An empirical cdf obtained by numerical integration.

found by averaging all computed β k and αk = φα(βk), respectively. Similarly for

µ .In order to check the fit of the generalized Gaussian pdf = GG(x |µ,α,β ) de-fined by the computed α, β , µ we show this pdf and the original empirical pdfsimultaneously in figure 14. Moreover, in figure 15 we also add the plots ofGG(x | µ, α−10%, β ) and GG(x | µ, α +10%, β ) in order to see the influence of a10% change in α to the deviation of generalized Gaussian pdf from the E pd f .Clearly, the curve corresponding to α fits best the Epdf.This statement can also be checked quantatively, using the MSE-criterion:

MSE =1N

N

∑i=1|E pd f (xi)−GG(xi |µ,α,β ) | 2 (23)

We chose N=100 equidistant points xi in the given x-interval and computed theMSE-values, see table in figure 16.

14

Figure 13: Computation of 50 β ′s using random values from the Ecd f .

Figure 14: The original Epdf and the estimated generalized Gaussian pdfGG(x | µ, β ).

15

Figure 15: The original Epdf and 3 estimated generalized Gaussian pdf’s.

Figure 16: The MSE-values for the 3 estimated generalized Gaussian pdf’s.

16

A Approximation of a sum 1N ∑

Ni=1 f (xi) by an integral

ProblemConsider the interval I = [U,V ] with 0≤U <V ]. Let f (x) be an integrable func-tion on this interval and let {xi |1≤ i≤ N} be a finite set of points in I .Our goal is to find an approximation of the sum 1

N ∑ f (xi) in terms of an analyticalintegral, in case N→ ∞.

Approach to problem solvingWe have xi ∈ [U,V ] with U ≤ 0 < V , for all 1 ≤ i ≤ N. We can assume that thesexi are ordered such that x1 = U, · · · ,xN = V . Furthermore, we assume that the xi

are equidistant on [U,V ]. In this case we can derive some results concerning theapproximation of certain summations involving xi by an integral. A more detailedand more accurate analysis is given in a next section. In the next appendix we con-sider the situation where the xi are not equidistantly but randomly distributed.

Let m = min1≤i≤N |xi| , M = max1≤i≤N |xi|.Clearly, m < M (indeed, if m = M then all xi are equal to M which is not the case).Let vi =

xiM . Then xi = Mvi and 0≤ m

M ≤ |vi| ≤ 1 .Since the xi are assumed to be equidistant we see that |xi+1− xi| is independent ofi. Hence, we can define ∆x = |xi+1− xi| and ∆v = |vi+1− vi|= ∆x

M .Let x = 0 be in the interval [xk, xk+1], so xk ≤ 0 < xk+1 or xk < 0≤ xk+1. Then

∆x = |xk+1− xk|= |xk|+ |xk+1| ≥ 2m (24)

Let L =V −U . Then we have N−1 intervals [xi, xi+1], so

∆x =L

N−1(25)

Combining (24) and (25) we find 0≤ mM ≤

L2M(N−1) . Thus,

mM

= O(1N), N→ ∞ (26)

Since the xi are equidistant on [U,V ], we see that the vi are equidistant on [ mM ,1]≈

[0,1], Hence,

∆v≈ 1(N−1)

and1

N∆v≈ 1, N→ ∞ (27)

Let f (|vi|) be an integrable function of |vi| defined on the interval [0,1] and let N

17

be sufficiently large.Then, using (27) we have

1N ∑ f (|vi|) =

1N∆v ∑

(f (|vi|)∆v

)≈∑

(f (|vi|)∆v

)≈

1∫m/M

f (v)dv =( 1∫

0

−m/M∫0

)f (v)dv, if N→ ∞

(28)

Recalling (26) and provided thatm/M∫

0f (v)dv = O( 1

N ), formula (28) simplifies to

1N ∑ f (|vi|)≈

1∫0

f (v)dv+O(1N), N→ ∞ (29)

Note:For use later, we also mention that in case N→ ∞(m

M

)β+1= O(

1N), 1−

(mM

)β+1= 1+O(

1N),

log(

1−(m

M

)β+1)= O(

1N),(m

M

)β+1log(m

M

)= O(

1N), for all β > 0

(30)

Outline of the proof for the last formula in (30):Consider the function H(x) = xβ+1log(x) = xh(x) with h(x) = xβ log(x)→ 0 asx→ 0, using x = e−βy. Hence, for each β > 0 there exists a constant C = C(β )such that |h(x)| ≤ 1 for all x and |H(x)| ≤ x if −log(x)>C(β ).Consequently, H(x) = O(x), x→ 0End of Note

18

A.1 A few examples

Example 1: f (x) = xβ , x≥ 0.

We have∫ m/M

0 f (x)dx = 1β+1

(mM

)β+1= O( 1

N ), see (26).Applying (29) we have (in case N→ ∞)

1N ∑ |vi|β ≈

∫ 1

0f (v)dv =

1β +1

(31)

Hence,1N ∑ |xi|β = Mβ 1

N ∑ |vi|β )≈Mβ 1β +1

(32)

Hence, using (30),

1β

log( 1

N ∑ |xi|β)=

1β

log(Mβ

N ∑ |vi|β)= log(M)+

1β

log( 1

N ∑ |vi|β)

≈ log(M)+1β

log( 1

β +1+O(

1N

)≈ log(M)+

1β

(log( 1

β +1

)+O(

1N)) (33)

Example 2: f (x) = xβ log(x), x≥ 0.Straightforward intgeration gives

m/M∫0

f (v)dv =

m/M∫0

vβ log(v)dv

=1

(β +1)

(mM

)β+1{

log(mM)− 1

β +1

}= O(

1N)

(34)

and ∫ 1

0f (v)dv =

1∫0

vβ log(v)dv =− 1(β +1)2 (35)

19

Hence,

1N ∑ |vi|β log|vi| ≈ −

1(β +1)2 , N→ ∞

and

1N ∑ |xi|β log|xi|=

Mβ

N

N

∑i=1

(|vi|)β log(M|vi|)

= Mβ

(log(M)

1N ∑ |vi|β +

1N ∑ |vi|β log|vi|

)≈Mβ

( log(M)

β +1− 1

(β +1)2

), N→ ∞

(36)

Example 3: log(

α(β ))

.We recall the formula for α(β ):

α(β ) =

(β

N

N

∑i=1|xi−µ|β

)1/β

(37)

As before, we can assume µ = 0 without loss of generality. Then

log(a(β )) =1β

log(β )+1β

log

(1N

N

∑i=1|xi|β

)(38)

Substitution of (33) into (38), we find

log(a(β )) = log(M)+1β

log(

β

β +1

)+O(

1N)

≈ log(M)+1β

log(

β

β +1

), N→ ∞

(39)

Hence,

a(β )≈M(

β

β +1

)1/β

, N→ ∞ (40)

A.2 An accurate approximation of the function g(β )

In this section we include a more accurate integral approximation of the sum, givenby Dr. A. J.E.M. Janssen, Eindhoven University of Technology, see [5].We recall the formula for the function g(β ):

g(β ) = g1(β )−P(β )+Q(β ) where (41)

20

g1(β ) = 1+Ψ(1/β )

β, (42)

P(β ) =∑

Ni=1

∣∣xi∣∣β log

∣∣xi∣∣

∑Ni=1

∣∣xi∣∣β (43)

and

Q(β ) =log( β

N ∑Ni=1

∣∣xi∣∣β )

β(44)

Here Ψ is the digamma function, Ψ(z) = Γ′(z)/Γ(z) as given in [4], Ch. 6.The xi, i = 1, ...,N are in an interval [m,M] with 0 < m < M and are defined by

xi = m+(i−1/2)M−m

N, i = 1, ...,N (45)

Using the midpoint integration rule we approximate

N

∑i=1

∣∣xi∣∣β =

N

∑i=1

(m+(i−1/2)

M−mN

)β

≈ NM−m

M∫m

uβ du =N

M−m

(Mβ+1−mβ+1

β +1

) (46)

Notice that it follows from (46) that

1N

N

∑i=1

∣∣xi∣∣β ≈ Mβ

β +1

(1− (m/M)β+1

1− (m/M)

)→ Mβ

β +1as m→ 0 (47)

Similarly,

N

∑i=1

∣∣xi∣∣β log(xi)≈

NM−m

M∫m

uβ log(u)du

=N

M−m1

β +1

([uβ+1log(u)

]M

m−

M∫m

uβ du)

=N

M−m1

β +1

(Mβ+1log(M)−mβ+1log(m)−Mβ+1−mβ+1

β +1

)(48)

21

We find from (46) and (48) that

P(β )≈N

M−m1

β+1

(Mβ+1log(M)−mβ+1log(m)− Mβ+1−mβ+1

β+1

)N

M−m

(Mβ+1−mβ+1

β+1

)=

Mβ+1log(M)−mβ+1log(m)

Mβ+1−mβ+1 − 1β +1

(49)

and

Q(β )≈ 1β

log(

β

NN

M−mMβ+1−mβ+1

β +1

)=

1β

log(

β

β +1Mβ+1−mβ+1

M−m

) (50)

Notice thatP(β )≈ log(M)− 1

β +1as m→ 0 (51)

and

Q(β )≈ log(M)+1β

log(

β

β +1

)as m→ 0 (52)

By combining (41), (49) and (50) we get

g(β )≈ 1+Ψ(1/β )

β

−(Mβ+1log(M)−mβ+1log(m)

Mβ+1−mβ+1 − 1β +1

)+

1β

log(

β

β +1Mβ+1−mβ+1

M−m

) (53)

Finally, by combining (51), (52), (53) and letting m→ 0 we find

g(β )≈(

1+Ψ(1/β )

β

)+( 1

β +1+

1β

log( β

β +1))

as m→ 0(54)

A.3 Behaviour of g(β ) as β ↘ 0

Let us now investigate the behaviour of g(β ) as β ↘ 0.To this end, we first consider formula (42) for g1(β ).We have (see [4], Eq. 6.3.18)

Ψ(z) = log(z)− 12z− 1

12z2 +O(z−4), z→ ∞ (55)

22

Hence,

Ψ(1/β ) = log(1/β )− 12

β +O(β 2), β ↘ 0 (56)

and1+

1β

Ψ(1/β ) =12+

1β

log(1/β )+O(β ), β ↘ 0 (57)

Combining (53) and (57) we obtain

g(β )≈ 12+

1β

log(1/β )

−(Mβ+1log(M)−mβ+1log(m)

Mβ+1−mβ+1 − 1β +1

)+

1β

log(

β

β +1Mβ+1−mβ+1

M−m

) (58)

or equivalently,

g(β )≈ 12− 1

βlog(1+β )

−(Mβ+1log(M)−mβ+1log(m)

Mβ+1−mβ+1 − 1β +1

)+

1β

log(Mβ+1−mβ+1

M−m

) (59)

Let us now return to expression (49) for P(β ) in the case β ↘ 0.

P(β )≈ Mβ+1log(M)−mβ+1log(m)

Mβ+1−mβ+1 − 1β +1

→ M log(M)−mlog(m)

M−m−1

(60)

23

Furthermore, we have

Mβ+1−mβ+1

M−m= Mβ

(1− (m/M)β+1

1− (m/M)

)= Mβ

(1− (m/M)

)+((m/M)− (m/M)β+1

)1− (m/M)

= Mβ

(1+(

mM)1− (m/M)β

1− (m/M)

)= Mβ

(1+(

mM)1− e−β log(m/M)

1− (m/M)

)= Mβ

(1+(

mM)1− (1+β log(m/M)+(O)(β 2))

1− (m/M)

)= Mβ

(1− (

mM)β log(m/M)

1− (m/M)+O(β 2)

)= Mβ

(1−β

mlog(m/M)

M−m+O(β 2)

)

(61)

Hence,

1β

log(Mβ+1−mβ+1

M−m

)=

1β

log[Mβ

(1−β

mlog(m/M)

M−m+O(β 2)

)]= log(M)− mlog(m/M)

M−m+O(β )

=M log(M)−mlog(m)

M−m+O(β )

(62)

Combining (50) and (62) we find

Q(β )≈ 1β

log(

β

β +1

)+

1β

log(Mβ+1−mβ+1

M−m

)=

1β

log(

β

β +1

)+

M log(M)−mlog(m)

M−m+O(β )

(63)

Notice that in case m→ 0 formula (63) reduces to

Q(β )≈ log(M)+1β

log(

β

β +1

)+O(β ) (64)

24

We conclude from (59), (60), (61), and (62) that

g(β )≈ 12−[ 1

βlog(1+β )

]β=0

−(M log(M)−mlog(m)

M−m−1)+

M log(M)−mlog(m)

M−m

=12−[ 1

β(β +O(β 2)

]β=0

+1 =12

as β ↘ 0

(65)

B Approximation of a sum 1N ∑

Ni=1 f (xi) for random xi

In this section we will discuss the approximation of a sum 1N ∑

Ni=1 f (xi) in case the

xi are random variables. We refer to [6], Ch. 5.4.Here we only consider the case that the xi correspond to the generalized Gaussiandensity function

p(x |α,β ,µ) =β

2αΓ(1/β )exp(−∣∣∣∣x−µ

α

∣∣∣∣β),with α > 0, β > 0

(66)

We assume µ = 0 and we will denote p(x |α,β ,µ = 0) simply by pα,β (x). Thus,

pα,β (x) = c(α,β )exp(−∣∣∣ xα

∣∣∣β), with c(α,β ) =β

2αΓ(1/β )(67)

Let {xi |1≤ i≤ N} be a set of random variables, where each xi corresponds to thedensiy function (67).Then we have

1N

N

∑1

xi ≈ E {x}=∞∫−∞

x pα,β (x)dx (68)

Notice that (68) gives E {x}= 0 since µ = 0 was assumed.

Let h(x) be an integrable function over R.As stated in [6], Ch. 5.4, Eq. (5-28) we have

1N

N

∑1

h(xi)≈ E {h(x)}=∞∫−∞

h(x) pα,β (x)dx (69)

25

We now consider two special cases. Below we will use the notation: ‘formula(x)→ [x = y]→ = formula(y)’ when the variable x will be replaced by y.

Case 1: h(x) = h1(x) = |x|β

1N

N

∑1|xi|β ≈ E {h1(x)}

= c(α,β )

∞∫−∞

|x|β exp(−∣∣∣ xα

∣∣∣β)dx = 2c(α,β )

∞∫0

xβ exp(−( x

α

)β)dx

→ [x = α t, dx = α dt ]→

= 2c(α,β )αβ+1∞∫

0

tβ exp(−tβ )dt

→ [ t = u1/β , dt = (1/β )u1/β−1du ]→

= 2c(α,β )αβ+1

β

∞∫0

u1/β e−udu = 2c(α,β )αβ+1

βΓ(1/β +1)

= 2β

2αΓ(1/β )

αβ+1

β

( 1β

Γ(1/β ))=

αβ

β

(70)Remark 1We used the identity Γ(z+1) = zΓ(z) for z > 0 with z = 1

β, see [4], Eq.(6.1.15).

Remark 2 By carefully inspecting the formulas in (70) we see

2c(α,β )αβ+1∞∫

0

tβ exp(−tβ )dt =αβ

β(71)

Remark 3Consider the case β = 2. Then we have from (70) and the fact that E {x}= 0

1N

N

∑1|xi|2 ≈ E {x2}= E {x2}−

(E {x}

)2=

12

α2 (72)

Notice that this formula is consistent with the fact that for β = 2 the generalizedpdf (67) equals the standard Gaussian pdf with µ = 0 and σ2 = α2/2.

End of Case 1

26

Case 2: h(x) = h2(x) = |x|β log|x|

1N

N

∑1|xi|β log|xi| ≈ E {h2(x)}

= c(α,β )

∞∫−∞

|x|β log|x|exp(−∣∣∣ xα

∣∣∣β)dx

= 2c(α,β )

∞∫0

xβ log(x)exp(−( x

α

)β)dx

→ [x = α t, dx = α dt ]→

= 2c(α,β )αβ+1∞∫

0

tβ log(α t)exp(−tβ )dt

= 2c(α,β )αβ+1{

log(α)

∞∫0

tβ exp(−tβ )dt +Kβ

}

with Kβ =

∞∫0

tβ log(t)exp(−tβ )dt

(73)

We haveKβ → [ tβ = u, t = u1/β , dt = (1/β )u1/β−1du ]→

= (1/β2)

∞∫0

u1/β log(u)e−udu(74)

Combining (73), (74), (71), and (67) we find

E {h2(x)}=αβ

βlog(α)+2c(α,β )αβ+1 Kβ

=αβ

βlog(α)+

β

Γ(1/β )α

β Kβ

(75)

End of Case 2

27

Recalling formula (43) for P(β ) and using the results (70), (75), (74) we have

P(β ) =1N ∑ |xi|β log|xi|

1N ∑ |xi|β

≈ E {h2(x)}E {h1(x)}

=

αβ

βlog(α)+ β

Γ(1/β )αβ Kβ

αβ/β

= log(α)+β 2

Γ(1/β )Kβ

= log(α)+1

Γ(1/β )

∞∫0

u1/β log(u)e−udu

(76)

Recall that Q(β ) = (1/β )log((β/N)∑ |xi|β

), see (44).

Then by using (70) we find

Q(β )≈ (1/β )log(

β E {h1(x)})

= (1/β )log(

βαβ

β

)= log(α)

(77)

We can conclude from (76) and (77) that

P(β )−Q(β )≈ 1Γ(1/β )

∞∫0

u1/β log(u)e−udu (78)

Hence, we have the following approximation for the function g(β ) defined in (41)

g(β ) = 1+Ψ(1/β )

β−(

P(β )−Q(β ))

≈ 1+Ψ(1/β )

β− 1

Γ(1/β )

∞∫0

u1/β log(u)e−udu(79)

RemarkClearly, this equation merely depends on β (as should be), despite the fact that inthe above derivation of the formulas for P(β ) and Q(β ) we have used the densityfunction pαβ that depends on β as well as on α .End of Remark

28

C Analysis of the behaviour of g(β )

For convenience, we recall the formula (41) for g(β ).

g(β ) = g1(β )−P(β )+Q(β ) where

g1(β ) = 1+Ψ(1/β )

β, and

P(β ) =∑

Ni=1

∣∣xi∣∣β log

∣∣xi∣∣

∑Ni=1

∣∣xi∣∣β , Q(β ) =

log( β

N ∑Ni=1

∣∣xi∣∣β )

β

(80)

C.1 Behaviour of g1(β )

We have g1(β ) = 1+ Ψ(1/β )β

, see figure 17.

Figure 17: The function g1(β ).

Notice thatg′1(β ) =− 1

β 2

{Ψ(

1β)+

1β

Ψ′(

1β)},

g′1(β∗) = 0, withβ

∗ ≈ 4.628,

g′1(β∗)< 0, for 0 < β < β

∗,

g′1(β∗)> 0, for β > β

∗

(81)

29

Figure 18: The function g′1(β ). The figure at right is a zoom-in of that at left.

As shown in Appendix F, eqs.(131), (132) we have

g′1(1/β ) =−β

{βΨ(β )+β

2Ψ′(β )

}→ 0 as β → 0 (82)

Thus,lim

β→∞

g′1(β ) = 0. (83)

The function g′1(β ) is shown in figure 18.We can conclude from (81) that

g1(β ) is monotonically decreasing for 0 < β < β∗,

g1(β ) is monotonically increasing for β > β∗,

g1(β ) has a minimum g1(β∗)≈−0.05 withβ

∗ ≈ 4.628,

g1(β ) = 0 for β = β0 ≈ 2.166 (by inspecting Fig.17)

g1(β )> 0 for 0 < β < β0 ,

g1(β )→ ∞ as β → 0 (since g′1(β )< 0 for 0 < β < β0 < β∗)

(84)

30

As shown in Appendix F, eq. (132) we have limx→0

xΨ(x)=−1, or limβ→∞

Ψ(1/β )β

=−1.

Hence,lim

β→∞

g1(β ) = 0. (85)

C.2 Behaviour of Q(β )

First notice that by comparing (44) and (38) we have

Q(β ) = log(a). (86)

Recalling (44) and (52) we have

Q(β ) =1β

log(

β

N

N

∑i=1

∣∣xi∣∣β)≈ log(M)+

1β

log(

β

β +1

)with M = max1≤i≤N |xi|

(87)

Hence,lim

β→∞

Q(β ) = log(M) and limβ→0

Q(β ) =−∞ (88)

C.3 Behaviour of P(β )

Recalling (43) and (51) we have

P(β ) =∑

Ni=1

∣∣xi∣∣β log

∣∣xi∣∣

∑Ni=1

∣∣xi∣∣β ≈ log(M)− 1

β +1(89)

Hence,lim

β→∞

P(β ) = log(M) and limβ→0

P(β ) = log(M)−1 (90)

C.4 Behaviour of P(β )−Q(β )

It is easily seen from (87) and (89) that

P(β )−Q(β )≈−( 1

β +1+

1β

log( β

β +1))

→ 0 as β → ∞

(91)

31

C.5 Behaviour of g(β )

Finally, by combining (41), (42) and (91) we have

g(β ) = g1(β )−(

P(β )−Q(β ))

≈(

1+1β

Ψ(1/β ))+( 1

β +1+

1β

log( β

β +1)) (92)

Hence, using (85), (91) and (65),

limβ→∞

g(β ) = 0 and limβ→0

g(β ) = 1/2 (93)

The behaviour of g(β ) is shown in figure 19.

Figure 19: The behaviour of function g(β ) including β → 0.Here g(β0) = 0 with β0 ≈ 9.68

32

D Asymptotics

D.1 Asymptotic behaviour of a(β )

We will prove that1) lim

β→∞

a(β )≈M where M = max1≤i≤N |xi| and

2) limβ→0

a(β ) = 0

Proof of Part 1It is easily seen from (38) that

limβ→∞

log(a(β ))≈ log(M), or equivalently

limβ→∞

a(β )≈M.(94)

End of Part 1

Proof of Part 2We introduce

T (β ) = log(1N ∑ |xi|β ) (95)

Using l′Hopital′s rule and the fact that limβ→0

T (β ) = 0 we find

limβ→0

1β

(log(

1N ∑ |xi|β )

)= lim

β→0

T (β )β

= limβ→0

T ′(β ) =

limβ→0

1N ∑ |xi|β log|xi|

1N ∑ |xi|β

=1N ∑ log|xi|

(96)

Recalling (38) , using (96) and limβ→0

1β

log(β ) = limy→∞

(−yey) =−∞ we find

log(

a(β ))=

1β

log(β )+1β

log

(1N

N

∑i=1|xi|β

)→−∞ as β → 0 (97)

Hence,

limβ→0

a(β ) = limβ→0

(β

N ∑ |xi|β)1/β

= 0 (98)

End of Part 2

33

E Analysis of ∂L/∂β

In this section we want to analyze te behaviour of ∂L/∂β as given (6), i.e.,

∂L∂β

=1β+

Ψ(1/β )

β 2 +1N

N

∑i=1

∣∣xi−µ

α

∣∣β log∣∣xi−µ

α

∣∣ = F(β )+S(β ) where

(99)

F(β ) =1β+

Ψ(1/β )

β 2 and (100)

S(β | µ,α) =1N

N

∑i=1

∣∣xi−µ

α

∣∣β log∣∣xi−µ

α

∣∣ (101)

Notice that F(β ) depends on β only. Recall that Ψ(x) is the logaritmic derivativeof the well-known Γ-function.We want to answer the

Question:In which situations has the equation ∂L/∂β = 0 a solution for β > 0, α ≥ 1 ?

Therefore, we will analyze the functions F(β ) and S(β ) given in (100), (101) sep-arately.

We will make the following assumptions:

• β > 0, α ≥ 1

• The real points xi (i = 1, N) are located in the interval [U, V ]

• In general, the xi are random values corresponding to the probability function(1)

• For analysis purposes we choose xi to be nonrandom, i.e., equidistantly inthe interval [U, V ]

• The values xi has mean µ = ∑Ni=1 xi

• The mean µ is in the interval (U, V ), so U < µ = 0 <V

• If xi = µ then these xi will be excluded (otherwise ∂L/∂β is not defined).

• Consequently, N denotes the number of points xi 6= µ

• Without loss of generality, we can take µ = 0

34

E.1 Analysis of F(β )

The first part of ∂L/∂β is denoted by

F(β ) =1β+

Ψ(1/β )

β 2 (102)

We recall (see [4], Ch.6)

• Ψ(x) is the digamma function: Ψ(x) = ddx ln(Γ(x)) = Γ′(x)

Γ(x) where

• Γ(x) is the well-known gamma function

Behaviour of the functions F(β ) and F ′(β )

The functions F(β ) and F ′(β ) are shown in figures 20 and 21.

Figure 20: F(β ) and F ′(β ) for 1≤ β ≤ 10, with limβ→∞

F(β ) = limβ→∞

F ′(β ) = 0.

For convenience, we mention a few properties of these functions. For a proof, seeAppendix F.

• F(β ) = 0 for β = β0 ≈ 2.13

• F(β )> 0 for β < β0 and F(β )< 0 for β > β0 (see figure 21).

35

Figure 21: F(β ) and F ′(β ) for 2≤ β ≤ 5

• In particular, F(β = 2)≈ 0.0091≈ 0.01

• F(β )→+∞ for β ↗ 0

• limβ→∞

F(β ) = 0

• limβ→∞

F ′(β ) = 0

• As shown in figure 21 and Appendix F we have

• F ′(β ) = 0 for β = β1 ≈ 3.38

• F ′(β )< 0 for β < β1 and F ′(β )> 0 for β > β1

• So, F(β ) is decreasing for 0 < β < β1, and increasing for β > β1

• and minβ>0 F(β ) = F(β1)≈−0.015

• Thus,0≤ F(β )/ 0.01 for 2≤ β ≤ β0,

−0.015/ F(β )< 0 for β > β0(103)

36

E.2 Analysis of S(β )

We will now start with analyzing the second part of ∂L/∂β , i.e.,

S(β | µ,α) =1N

N

∑i=1

∣∣xi−µ

α

∣∣β log∣∣xi−µ

α

∣∣, for β > 0. (104)

As mentioned before we may assume that µ = 0. Where needed we will indicatethe dependence of µ explicitly in the formula for S. The dependence on α will beomitted. So, in most cases we will write S(β ) or S(β | µ = 0).We introduce the function f (x) for β ≥ 0,α ≥ 1 as follows:

f (x) =

{ ( xα

)β log( x

α

)for x > 0

0 for x = 0(105)

Notice that limx→0

f (x) = 0. See Appendix H.

Then we can write S(β ) = S(β | µ = 0) as

S(β ) =1N ∑|xi|>0 f (|xi|) = S−(β )+S+(β ) where

S−(β ) =1N ∑xi<0 f (|xi|) =

1N ∑xi<0 f (−xi) =

1N ∑ yi>0

yi=−xi

f (yi)

S+(β ) =1N ∑xi>0 f (xi)

(106)

To simplify the analysis of S(β ) we assume that the points xi 6= 0, 1 ≤ i ≤ N aredistributed equidistantly over the interval [U, V ]. By assumption, µ = 0. More-over, we assume that µ is in [U,V ]. Hence, U < 0 and V > 0.Let L = V −U be the length of the interval [U,V ] and let ∆x = L/N be the dis-tance between two subsequent points xi and xi+1. By assumption xi 6= 0. So,µ ∈ (−∆x,∆x) and xi ∈ [U,−∆x] or xi ∈ [∆x,V ].If N is sufficiently large, then we have

∑|xi|>0 f (|xi|) ∆x =∫ V

Uf (x)dx+O(∆x) (107)

Hence,

S−(β ) =1N ∑xi<0 f (−xi) =

1N∆x ∑xi<0 f (−xi)∆x =

1L ∑xi<0 f (−xi)∆x

=1L

∫ −∆x

Uf (−x)dx+O(∆x)

=1L

∫ 0

Uf (−x)dx− 1

L

∫ 0

−∆xf (−x)dx+O(∆x)

=1L

∫ |U |0

f (x)dx− 1L

∫∆x

0f (x)dx+O(∆x)

(108)

37

and

S+(β ) =1N ∑xi>0 f (xi) =

1N∆x ∑xi>0 f (xi)∆x =

1L ∑xi>0 f (xi)∆x

=1L

∫ V

∆xf (x)dx+O(∆x)

=1L

∫ V

0f (x)dx− 1

L

∫∆x

0f (x)dx+O(∆x)

(109)

It is shown in Appendix H that∫∆x

0f (x)dx = O(∆x), ∆x→ 0 (110)

Thus,

S−(β ) =1L

∫ |U |0

f (x)dx+O(∆x), ∆x→ 0

S+(β ) =1L

∫ V

0f (x)dx+O(∆x), ∆x→ 0

S(β ) =1N ∑|xi|>0 f (|xi|) = S−(β )+S+(β )

=1L

(∫ |U |0

+∫ V

0

)f (x)dx+O(∆x), ∆x→ 0

(111)

Notice that if∫ V

0 f (x)dx < 0 for some α and β then S−(β ) might be nonnegativedue to the term O(∆x) in (111).We will use the notation S−(β )l0 to refer to its formula (111) with

∫ V0 f (x)dx < 0,

and similarly for S+(β ) and S(β ). Using standard calculus we find we find

S+(β ) =1L

∫ V

0f (x)dx+O(∆x)

=1L

Vβ +1

(Vα

)β {log(

Vα)− 1

β +1

}+O(∆x)

(112)

For simplicity, we introduce the function

H(β |α,V ) =1L

Vβ +1

(Vα

)β {log(

Vα)− 1

β +1

}(113)

Thus,S+(β ) = H(β |α,V )+O(∆x), ∆x→ 0 (114)

38

A similar expression can be given for S−(β ) by replacing V by |U | in (113), i.e.,

S−(β ) = H(β |α, |U |)+O(∆x), ∆x→ 0 (115)

Finally, combination of (111), (112), (115) gives

S(β ) = H(β |α,V )+H(β |α, |U |)+O(∆x), ∆x→ 0 (116)

Let us return to formula (113) for H(β |α,V ) to find out when S(β )< 0.For fixed α and V we introduce H(β ) = H(β |α,V ).Furthermore, let K =V/α and c = log(K), then

H(β ) =VL

Kβ

β +1

(log(K)− 1

β +1

)H ′(β ) =

VL

Kβ

(β +1)3

(c2(β +1)2−2c(β +1)+2

)=

VL

Kβ

(β +1)3

((c(β +1)−1)2 +1

)> 0 for all β ≥ 0

(117)

Thus, H(β ) is a monotonically increasing function for β ≥ 0 and H(β )> H(0) =VL

(log(V/α)−1

).

Clearly, sign(

H(β ))= sign

(log(V/α)− 1

β+1

)and

log(

Vα

)− 1

β +1> 0 if α <Ve−

1β+1

< 0 if α >Ve−1

β+1

(118)

andV/e≤Ve−

1β+1 <V for all β ≥ 0 (119)

See also figure 22.We can distinguish three cases:Case 1If α ≥V then α ≥V >Ve−

1β+1 for all β ≥ 0. Moreover, in this case 0 <V/α ≤ 1.

So, we can conclude

if α ≥ V then S+(β )l0 for all β ≥ 0 and

S+(β )→ 0 as β → ∞(120)

Note: if α is given by (5) then this case can not occur as indicated in secton 4.

39

Figure 22: The function f (β ) =Ve−1

β+1 with f (0) =V/e and limβ→∞

f (β ) =V .

Case 2If α ≤V/e then α ≤V/e <Ve−

1β+1 for all β > 0, and thus log

(Vα

)− 1

β+1 > 0 forall β > 0. Then it follows from (112)

if α ≤ V/e then S+(β )m0 for all β > 0 (121)

Case 3If V/e < α < V then (see figure 23) there exists a βV such that Ve−

1βV +1 = α and

Ve−1

β+1 < α for all 0≤ β < βV , with

βV =1

log(V/α)−1 (122)

Recalling (112) and (118) we can now conclude

if V/e <α <V then

H(β |α,V )< 0 and

S+(β )l0 for all 0≤ β < βV

(123)

In figure 23 the behaviour of H(β |α,V ) is shown for a few values of α .Clearly, by replacing V by |U | in the formulas above, similar results can be givenfor S−(β ):

if α ≥ |U| then S−(β )l0 for all β > 0;

if α ≤ |U|/e then S−(β )m0 for all β > 0;

if V/e < α < |U| then S−(β )l0 for all 0≤ β < β|U|

(124)

40

Figure 23: The function H(β |α,V ) approximating S+(β ). We have H(β |α,V ) =0for β = βV =

(log(V/α)−1−1

), indicated by a �.

The value β|U | is given by

β|U | =1

log(|U |/α)−1 (125)

Now we can combine the results above for S(β ) = S−(β )+S+(β ).To this end , we refer to figure 24 and we findCase 1If V = |U | then S(β )l0 for 0≤ β < βV, in case V/e < α <V .

Case 2If V > |U | (as shown in figure 24) then

if V/e < α < |U| then S(β )l0, 0 < β < βV

if α ≤ V/e or α ≥ |U| then S(β )m0 for all β > 0(126)

Case 3

41

Figure 24: The functions fV and f|U | defining the β -region where S(β )l0.

If V < |U | then interchange V and |U | in the previous formulas, i.e.,

if |U|/e < α < V then S(β )l0, 0 < β < β|U|

if α ≤ |U|/e or α ≥ V then S(β )m0 for all β > 0(127)

The formules (126) and (127) can be combined to

if max(|U|/e,V/e)< α < min(|U|,V) then

S(β )l0, 0 < β < min(β|U |, βV ) (128)

if α ≤max(|U|/e,V/e) or α ≥min(|U|,V) then

S(β )m0 for all β > 0(129)

42

F Properties of Γ(x), Ψ(x),Ψ′(x), F(β ) and F′(β )

In figure 25 the behaviour of Γ(x), Ψ(x) and Ψ′(x) is shown.

Figure 25: The functions Γ(x), Ψ(x) and Ψ′(x)

We need the following properties of Ψ, Ψ′ and Γ for a detailed analysis of F(β ).

F.1 Properties of Ψ′(x)

• Ψ(n)(x) = (−1)n+1n!∑∞k=0

1(x+k)n+1 (see [4], Eq. 6.4.10)

• In particular, Ψ′(x) = ∑∞k=0

1(x+k)2>0 for x > 0

• Ψ′(x) = 1x2 +∑

∞k=1

1(x+k)2 > 0, thus Ψ′(x)∼ 1

x2 for x→ 0

• Hence,limx→0

x2Ψ′(x) = 1 (130)

43

F.2 Properties of Ψ(x)

• Ψ(x) is monotonically increasing for x > 0 (since Ψ′(x)> 0)

• Ψ(3/2) =−γ−2ln(2)+2≈ 0 (see [4], Eq. 6.3.4)

• Ψ(x0) = 0 for x0 ≈ 1.460≈ 3/2 (see (see [4], Table 6.1)

• Ψ(x)< 0 for 0 < x < x0 and Ψ(x)> 0 for x > x0

• Ψ(1/2)≈−2, so Ψ(x)/−2 for 0 < x≤ 1/2 (see [4], Eq. 6.3.3)

• Ψ(1− x) = Ψ(x)+π cot(πx) (see [4], Eq. 6.3.7)

• So, limx→0

x2 Ψ(x) = limx→0

(x2[Ψ(1− x)−π cot(πx)]

)=− lim

x→0πx2 cot(πx)

• and πx2 cot(πx) = πx2 cos(πx)sin(πx) = xcos(πx) πx

sin(πx)

• Then limx→0

πx2 cot(πx) = limx→0

xcos(πx) πxsin(πx) = lim

y→0

yπ

cos(y) ysin(y) = 0

• Thus,limx→0

x2Ψ(x) = 0 (131)

• Similarly, limx→0

xΨ(x) =− limx→0

πxcot(πx)

• and πxcot(πx) = cos(πx) πxsin(πx) = cos(y) y

sin(y) , (y = πx)

• Thus,limx→0

xΨ(x) = limy→0

cos(y)y

sin(y)=−1 (132)

F.3 Properties of Γ(x)

• Γ(x)> 0 for x > 0

• Ψ(x) = ddx ln(Γ(x))

• Ψ(x)< 0, x < x0, Ψ(x0) = 0 and Ψ(x)> 0, x > x0

• Hence, ln(Γ(x)) has a unique minimum for x = x0 ≈ 3/2

• Since the ln-function is increasing, also Γ(x) has a unique minimum for x =x0

44

F.4 Behaviour of F(β ) and F ′(β )

For convenience, we duplicate figure 21 with F(β ) and F ′(β ) for 2≤ β ≤ 5.We have the following properties.

Figure 26: F(β ) and F ′(β ) for 2≤ β ≤ 5 and F ′(β ) = 0 for β ≈ 3.38.

• limβ→∞

F(β )= limβ→∞

(1β+ Ψ(1/β )

β 2

)= lim

β→∞

Ψ(1/β )β 2 = lim

x→0x2Ψ(x)= 0 (see (132))

• F ′(β ) =− 1β 2

(1+2Ψ(1/β )/β +Ψ′(1/β )/β 2

)• Let H(β ) = 1+2Ψ(1/β )/β +Ψ′(1/β )/β 2 or

• H(x) = 1+2xΨ(x)+ x2Ψ′(x) where x = 1/β

• Using (132) and (130), we see that limx→0

H(x) = 1+2∗ (−1)+1 = 0

• So, limβ→∞

F ′(β ) = limx→0

F ′(x) =− limx→0

x2H(x) = 0 (see also figure 27)

• Numerical experiments show the following properties

• F ′(β ) = 0 for β = β1 ≈ 3.38

45

• F ′(β )< 0 for β < β1 and F ′(β )> 0 for x > β1

• Thus, F(β ) is decreasing for 0 < β < β1 , increasing for β > β1 and

• minβ>0 F(β ) = F(β1)≈−0.015

• F(β )→+∞ for β ↗ 0

• F(β ) = 0 for β = β0 ≈ 2.13

• F(β )> 0 for β < β0 and F(β )< 0 for β > β0

• F(β = 2)≈ 0.0091≈ 0.01

• Thus, −0.015/ F(β )/ 0.01 for β ≥ 2

Figure 27: F ′(x) where x = 1/β and limx→0

F ′(x) = limβ→∞

F ′(1/β ) = 0.

46

G The sum of two Gaussian pdf’s and its cdf

The standard Gaussian probability density function (pdf) f (x) and its correspond-ing cumulative density function F(x) are given by the wel-known formulas

f (x) =1

σ√

2πexp(−|x−µ|2

2σ2

)(133)

and

F(x) =∫ x

−∞

f (t)dt =12

[1+ er f

(x−µ

σ√

2

)](134)

Both functions are well-known in literature, and are easily available via tables orby computer packages. However, when a pdf is given as a sum of a few pdf’s inanalytical form, then the computation of its corresponding cdf in analytical formwill cost more effort. In such cases a numerical approach can be helpful. As anexample, we give the result of a numerical computation of the cdf corresponding tothe sum of two standard Gaussian pdf’s, each with different mean µ and standarddeviation σ . See figures 28 and 29. Notice that when comparing both figures we

Figure 28: The pdf of the sum of 2 standard Gaussian density functions.

see that the pdf curve of (y1 + y2)/2 is more pronounced than the correspondingpdf.Remark: In the figure above we have used the notation N(µ, σ) instead of themore commonly used notation N(µ, σ2).

47

Figure 29: The cdf of the sum of 2 standard Gaussian density functions.

48

H Behaviour of f(x)

Consider the function

f (x) =( x

α

)β

log( x

α

)for x > 0 and β > 0, α ≥ 1

f (x) = 0 for x = 0(135)

Then we have (see also figure 30)

Figure 30: The function f (x) = (x/α)β log(x/α). Its minimum is marked by an *.

• f ′(x) = 1α

( xα

)β−1 (1+β log( x

α

))• f ′(x) = 0 for x = x = αe−1/β and f (x) =−1/(eβ )

• f ′(x)> 0 for x > α and f ′(x)< 0 for 0 < x < x

• f (x) has a minimum for x = x

49

• f (x) is monotonically increasing for x > x

• f (x) is monotonically decreasing for 0 < x < x

• f (α) = 0 with f′(α) = 1α

• Substituting x/α = exp(−y) in f (x) gives f (x(y)) =−ye−βy. Hence,

• limx→0

f (x) = 0

∫ P

0f (x)dx =

Pβ +1

f (P)− P(β +1)2

(Pα

)β

(136)

and ∫∆x

0f (x)dx =

∆xβ +1

f (∆x)− ∆x(β +1)2

(∆xα

)β

(137)

It can easily be seen from (137) and Appendix H that∫∆x

0f (x)dx = O(∆x), ∆x→ 0. (138)

50

I Generating a random number from an empirical pdf

Let Ecd f (x) denote an empirical cumulative density function computed by numer-ical integration of a given empirical pdf denoted by Epd f (x). We assume thatEcd f (x) is a monotonically increasing function. Let the coordinates {(xi,yi) | i = 1, . . . ,N}be located on the curve Ecd f (x). Thus, yi = Ecd f (xi) or equivalently, xi = E−1

cd f (yi)for all i.Let {rnd(i) | i = 1, . . . ,K} be a given set of uniformly on [0,1] distributed ran-dom values. We want to determine the set {xi | i = 1, . . . ,K} such that each xi =E−1

cd f (rnd(i)) as accurate as possible. This is done by linear interpolation as fol-lows. See figure 31.

Figure 31: Generating a random value corresponding to an empirical cdf, usinglinear interpolation.

Let r be an arbitrarily chosen value from the set {rnd(i) | i = 1, . . . ,K}.We introduce

y+ = min1≤i≤N

{yi | yi ≥ r} and

y− = max1≤i≤N

{yi | yi ≤ r}(139)

Notice that y+ > y− due to the assumed monotonicity of Ecd f (x).Let x+ and x− correspond to y+ and y−, respectively. Thus,

x+ = E−1cd f (y

+) and x− = E−1cdf(y

−) (140)

51

Clearly, x+ > x−.Moreover, we define

dy+ = |y+− r|, dy− = |y−− r|,dx+ = |x+− xr|, dx− = |x−− xr|,

dy = dy++dy−, dx = dx++dx−(141)

Notice that dy+ ≥ 0 and dy− ≥ 0, but not both of them can be equal to zero.If dy+ = 0 then r = y+ and dx+ = 0.If dy− = 0 then r = y− and dx− = 0Recalling fig(31) and using linear interpolation, we have

xr− x−

dx=

dy−

dy,

x+− xr

dx=

dy+

dyand

tan(α) =dydx

=dy+

dx+=

dy−

dx−

(142)

It is easily seen from (142) that

xr = x−+dx− = x+−dx+ (143)

Hence, we can define the value xr = E−1cd f (r) as follows:

xr =

{x+−dx+ if 0≤ dy+ < dy−

x−+dx− if 0≤ dy− < dy+(144)

This value xr can now be interpreted as a random value corresponding to the prob-ability function Epd f (x). By taking successively r = rnd(i), i = 1, . . . ,K and com-puting the correspondig xr-values as indicated above, we obtain a set of K valuesbeing randomly distributed conform the pdf Epd f (x).

52

J Conclusions

In this report we addressed the problem of estimating the paramters of a General-ized Gaussian distribution. We studied in detail a numerical method known fromliterature. We proposed a new numerical reliable and fast procedure to compute thedesired parameters. This procedure is based on a detailed analysis of all functionsinvolved in order to have fast and safe convergence of the used Newton-Raphsonmethod.

K Acknowledgements

The authors wish to thank Dr. E.J.W. ter Maten, Dr. M. Hochstenbach, Dr. A. DiBucchianico and Dr. A.J.E.M. Janssen for their stimulating discussions and usefulsuggestions.

References

[1] Wikipedia: ”Generalized normal distribution”, 2015.

[2] K Kokkinakis, A.K. Nandi: ”Exponent parameter estimation forgeneralized Gaussian probability density functions with applicationto speech modeling”. Signal Processing 85, pp. 1852–1858, 2005.www.ElsevierComputerScience.com.

[3] R. Krupinski, J. Purczynski: ”Approximated fast estimator for the shapeparameter of generalized Gaussian distribution”. Signal Processing 86,pp. 205–211,2006. www.ElsevierComputerScience.com.

[4] M. Abramowitz, I.A. Stegun: Handbook of Mathematical Functions WithFormulas, Graphs, and Mathematical Tables” National Bureau of Standards,Applied Mathematics Series-55

[5] A.J.E.M. Janssen, Eindhoven University of Technology, The Netherlands,“private communication”, October 2, 2015

[6] A. Papoulis, Probability, Random Variables, and Stochastic Processes,McGraw-Hill, 1965

53

Contents

1 General Introduction 1

2 Abstract 2

3 Mathematical problem description 3

4 Behaviour of the function α(β ) 74.1 Asymptotic behaviour of g(β ) . . . . . . . . . . . . . . . . . . . 8

5 Numerical experiments 95.1 Procedure for parameter estimation of an empirical pdf . . . . . . 95.2 Application of the estimation procedure . . . . . . . . . . . . . . 13

A Approximation of a sum 1N ∑

Ni=1 f (xi) by an integral 17

A.1 A few examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 19A.2 An accurate approximation of the function g(β ) . . . . . . . . . . 20A.3 Behaviour of g(β ) as β ↘ 0 . . . . . . . . . . . . . . . . . . . . 22

B Approximation of a sum 1N ∑

Ni=1 f (xi) for random xi 25

C Analysis of the behaviour of g(β ) 29C.1 Behaviour of g1(β ) . . . . . . . . . . . . . . . . . . . . . . . . . 29C.2 Behaviour of Q(β ) . . . . . . . . . . . . . . . . . . . . . . . . . 31C.3 Behaviour of P(β ) . . . . . . . . . . . . . . . . . . . . . . . . . 31C.4 Behaviour of P(β )−Q(β ) . . . . . . . . . . . . . . . . . . . . . 31C.5 Behaviour of g(β ) . . . . . . . . . . . . . . . . . . . . . . . . . . 32

D Asymptotics 33D.1 Asymptotic behaviour of a(β ) . . . . . . . . . . . . . . . . . . . 33

E Analysis of ∂L/∂β 34E.1 Analysis of F(β ) . . . . . . . . . . . . . . . . . . . . . . . . . . 35E.2 Analysis of S(β ) . . . . . . . . . . . . . . . . . . . . . . . . . . 37

F Properties of Γ(x), Ψ(x),Ψ′(x), F(β ) and F′(β ) 43F.1 Properties of Ψ′(x) . . . . . . . . . . . . . . . . . . . . . . . . . 43F.2 Properties of Ψ(x) . . . . . . . . . . . . . . . . . . . . . . . . . . 44F.3 Properties of Γ(x) . . . . . . . . . . . . . . . . . . . . . . . . . . 44F.4 Behaviour of F(β ) and F ′(β ) . . . . . . . . . . . . . . . . . . . 45

54

G The sum of two Gaussian pdf’s and its cdf 47

H Behaviour of f(x) 49

I Generating a random number from an empirical pdf 51

J Conclusions 53

K Acknowledgements 53

55

PREVIOUS PUBLICATIONS IN THIS SERIES:

Number Author(s) Title Month

15-36 15-37 15-38 15-39 15-40

K. Urbanowicz A.S. Tijsseling N. Kumar J.H.M. ten Thije Boonkkamp B. Koren C. Filosa J.H.M. ten Thije Boonkkamp W.J. IJzerman F.A. Radu K. Kumar J.M. Nordbotten I.S. Pop T.G.J. Beelen J.J. Dohmen

Work and life of Piotr Szymański Flux approximation scheme for the incompressible Navier-Stokes equations using local boundary value problems A new ray tracing method in phase space using α-

shapes A convergent mass conservative numerical scheme based on mixed finite elements for two-phase flow in porous media Parameter estimation for a generalized Gaussian distribution

Dec. ‘15 Dec. ‘15 Dec. ‘15 Dec. ‘15 Dec. ‘15

Ontwerp: de Tantes,

Tobias Baanders, CWI