Adaptive Choice of Thresholds and the
Bootstrap Methodology: an Empirical Study
∗
M. Ivette Gomes,
Fernanda Figueiredo
andM. Manuela Neves
Abstract
In this paper, we discuss an algorithm for the adaptive estimation of a positive extreme value index, γ, the primary parameter in Statistics of Extremes. Apart from classical extreme value index estimators, we suggest the consideration of associated second-order corrected-bias estimators, and propose the use of bootstrap computer-intensive methods for the adaptive choice of thresholds.
Keywords. Heavy tails; statistics of extremes; extreme value index; adaptive semi-parametric estimation; bias reduction; location/scale invariant estimation.
1
Introduction and outline of the paper
Heavy-tailed models appear often in practice in fields like Telecommunications, Insurance, Finance, Bibliometrics and Biostatistics. We shall deal with the estimation of a positive extreme value index (EVI), γ, the primary parameter in Statistics of Extremes. Apart from the classical Hill, moment and generalized-Hill semi-parametric estimators of γ, detailed in Section 2, we shall consider the associated classes of second-order reduced-bias estimators, based on an adequate estimation of generalized scale and shape second-order parameters, valid for a large class of heavy-tailed underlying parents, and appealing in the sense that we are able to reduce the asymptotic bias of a classical estimator without increasing its asymptotic variance. We shall call these estimators “classical-variance reduced-bias” (CVRB) estimators.
After the introduction, in Section 2, of a few technical details in the area of Extreme Value Theory (EVT), related with the EVI-estimators under consideration in this paper, we shall briefly discuss, in Section 3, the kind of second-order parameters’ estimation which enables the building of reduced-bias estimators with the same asymptotic variance of the associated classical estimator. After the discussion, in Section 4, of the asymptotic behaviour of the estimators under consideration, we propose and discuss in Section 5, and in the lines of [13], an algorithm for the adaptive estimation of a positive EVI, through the use of bootstrap computer-intensive methods. The algorithm is described for a classical EVI estimator and associated CVRB estimator, but it works similarly for the estimation of any other parameter of extreme events, like a high quantile, the probability of exceedance or the return period of a high level. Section 6 is entirely dedicated to the application of the algorithm to the analysis of environmental data, the number of hectares burned during all wildfires recorded in Portugal in the period 1999-2003.
2
The EVI-estimators under consideration
In the area of EVT, and for large values, a model F is said to be heavy-tailed whenever the right-tail function, F := 1 − F , is a regularly varying function with a negative index of regular variation, denoted −1/γ, i.e., if for all x > 0, there exists γ > 0, such that F (tx)/F (t) → x−1/γ, as t → ∞. If this holds, we use the notation F ∈ RV
−1/γ, and
we are working in the whole domain of attraction (for maxima) of heavy-tailed models, denoted DM(EVγ)γ>0. Equivalently, with U (t) := F← 1 − 1/t=inf {x : F (x) ≥ 1 − 1/t},
F ∈ DM(EVγ)γ>0 ⇐⇒ F ∈ RV−1/γ ⇐⇒ U ∈ RVγ, the so-called first-order condition. For
these heavy-tailed parents, given a sample Xn := (X1, . . . , Xn) and the associated sample
of ascending order statistics (o.s.’s), (X1:n ≤ · · · ≤ Xn:n), the classical EVI estimator is the
Hill estimator ([15]), Hk ≡ Hk,n:= 1 k k X i=1 {ln Xn−i+1:n− ln Xn−k:n}, (1)
the average of the k log-excesses over a high random threshold Xn−k:n, an intermediate o.s.,
i.e., with k such that
But the Hill-estimator Hk, in (1), reveals usually a high non-null asymptotic bias at
optimal levels, i.e., levels k where the mean squaed error (MSE) is minimum. This non-null asymptotic bias, together with a rate of convergence of the order of 1/√k, leads to sample paths with a high variance for small k, a high bias for large k, and a very sharp MSE pattern, as function of k. Recently, several authors have been dealing with bias reduction in the field of extremes (for an overview, see [17], Chapter 6, as well as the more recent paper, [4]). For technical details, we then need to work in a region slightly more restrict than DM(EVγ)γ>0.
In this paper, we shall consider parents such that, with γ > 0, ρ < 0, and β 6= 0,
U (t) = Ctγ 1 + A(t)/ρ + O(A2(t)), as t → ∞, A(t) =: γβtρ. (3) The most simple class of second-order minimum-variance reduced-bias (MVRB) EVI-estimators is the one in [3], used for a semi-parametric estimation of ln V aRp in [9]. This
class, here denoted H ≡ Hk, is the CVRB-estimator associated with the Hill estimator
H = Hk, in (1), and depends upon the estimation of the second-order parameters (β, ρ), in
(3). Its functional form is
Hk ≡ Hk,n, ˆβ, ˆρ:= Hk 1 − ˆβ(n/k)ρˆ/(1 − ˆρ), (4)
where ( ˆβ, ˆρ) is an adequate consistent estimator of (β, ρ). Algorithms for the estimation of (β, ρ) are provided, for instance, in [9] , and will be reformulated in Section 3 of this paper.
Apart from the Hill estimator, in (1), we suggest the consideration of two other classical estimators, valid for all γ ∈ R, but taken here exclusively for heavy tails, the moment ([5]) and the generalized-Hill ([1], [2]) estimators. The moment estimator (M) has the functional expression Mk≡ Mk,n := Mk,n(1)+ 211 − Mk,n(2)/(Mk,n(1))2− 1 −1 , (5) with Mk,n(j) := 1kPk i=1 ln Xn−i+1:n− ln Xn−k:n j , j ≥ 1 Mk,n(1) ≡ Hk, in (1). The generalized
Hill estimator (GH), based on Hk,n, in (1), is given by
GHk ≡ GHk,n := Hk,n+ 1 k k X i=1 ln Hi,n− ln Hk,n . (6)
The associated CVRB estimators have similar expressions, due to same dominant component of asymptotic bias of the estimators in (5) and (6), for a positive EVI. Denoting W either M or GH, and with the notation W for either M or GH, we get
Wk≡ Wk,n, ˆβ, ˆρ:= Wk 1 − ˆβ (n/k) ˆ ρ
In the sequel, we generally denote C any of the classical EVI-estimators, in (1), (5) and (6), and C the associated CVRB-estimator.
3
Estimation of second-order parameters
The estimation of γ, β and ρ at the same value k leads to a high increase in the asymptotic variance of CVRB-estimators Ck, ˆβ, ˆρ, which becomes σ2
C((1 − ρ)/ρ)
4
(see [16], among others). The external estimation of ρ at k1, but the estimation of γ and β at k = o(k1), slightly
decreases the asymptotic variance to σ2
C((1 − ρ)/ρ)
2
, stilll greater than σ2
C (see [7], among
others). The external estimation of both β and ρ at a level k1, and the the estimation
of γ at a level k = o(k1), or even k = O(k1), can lead to a CVRB estimator with an
asymptotic variance σ2
C, provided we choose adequately k1 (see [3], [10], [11]). Such a choice
is theoretically possible (see [12] and [4]), but under conditions difficult to guarantee in practice. As a compromise between theoretical and practical results, we have so far advised any choice k1 = n1−, with small. We shall consider here = 0.001.
Algorithm (second-order parameters’ estimation):
1. Given an observed sample (x1, . . . , xn), plot the observed values of ˆρτ(k), the most
simple estimator in [6], for the tuning parameters τ = 0 and τ = 1.
2. Consider { ˆρτ(k)}k∈K, with K = ([n0.995], [n0.999]), compute their median, denoted ητ,
and compute Iτ :=
P
k∈K( ˆρτ(k) − ητ) 2
, τ = 0, 1. Next choose the tuning parameter τ∗ = 0 if I0 ≤ I1; otherwise, choose τ∗ = 1.
3. Work with ˆρ ≡ ˆρτ∗ = ˆρτ∗(k1) and ˆβ ≡ ˆβτ∗ := ˆβρˆ
τ ∗(k1), with k1 = n
0.999, being ˆβ ˆ ρ(k),
the estimator in [7].
Remark 1. This algorithm leads usually to the tuning parameter τ = 0 whenever |ρ| ≤ 1 and τ = 1, otherwise. For details on this and similar algorithms, see [9].
4
Asymptotic distributional behaviour of the
estima-tors
In order to obtain a non-degenerate behaviour for any semi-parametric EVI-estimator, it is convenient to assume a second-order condition, measuring the rate of convergence in the first-order condition. Such a condition, valid for all x > 0, involves a parameter ρ ≤ 0, a rate function A, with |A| ∈ RVρ and is given by
lim
t→∞(U (tx)/U (t) − x
γ)/A(t) = xγ (xρ− 1)/ρ. (8)
In this paper, and mainly because of the reduced-bias estimators in (4) and (7), generally denoted Ck ≡ Ck,n, ˆβ, ˆρ, we shall assume that (3) holds. Then, (8) holds, with A(t) = γ βt
ρ.
Let Ck be the associated classical estimator of γ. Trivial adaptations of the results in
the above-mentioned papers (see also [14]) enable us to state, without proof, the following theorem, again for models with γ > 0.
Theorem 1. Assume that condition (8) holds, and let k = kn be an intermediate sequence,
i.e. (2) holds. Then, there exist a sequence ZC
k of asymptotically standard normal random
variables, σC > 0 and real numbers bC,1 such that Ck=dγ+σCZ
C k/
√
k+bC,1 A(n/k) (1+op(1)).
If we further assume that (3) holds, and estimate β and ρ consistently through ˆβ and ˆρ, in such a way that ˆρ − ρ = op(1/ ln n), we can guarantee that there exists a pair of real numbers
(bC,1, bC,2), with b C,1 = 0, such that Ck,n, ˆβ, ˆρ= dγ +σ CZ C k/ √ k +b C,1 A(n/k)+bC,2 A 2(n/k) (1+ op(1)).
As n → ∞, let k = kn be intermediate such that
√
k A(n/k) → λ, finite, the lev-els k where the MSE of Ck is minimum. Let ˆγk denote either Ck or Ck. Then, we
have √k ˆγk− γ
d
→ N ormal(λ bˆγ,1, σC2), even if we work with CVRB EVI-estimators.
If √k A(n/k) → ∞, with √kA2(n/k) → λ
A, finite, the levels k where the MSE of Ck is
minimum, √k Ck− γ d → Normal λAb C,2, σ 2 C. We have σ 2 H = γ 2, σ2 M = σ 2 GH = 1 + γ 2, bH,1 = 1/(1 − ρ), bM,1 = bGH,1 = (γ − γρ + ρ)/(γ(1 − ρ)2), b H,1 = bM ,1 = bGH,1 = 0.
Conse-quently, since bC,1 6= 0 whereas b
C,1 = 0, the C-estimators outperform the C-estimators for
0 ! 0.5 1 BIASC 2 ! BIASC2 ! VarC " VarC ! MSEC ! MSEC ! r = k /n
Figure 1: Comparative asymptotic variances (Var), squared bias (BIAS2) and mean squared errors (MSE) of a classical EVI-estimator and associated CVRB estimator.
5
The bootstrap methodology and adaptive
EVI-estimation
With AMSE standing for “asymptotic MSE”, and kbγ
0(n) := arg minkM SE(bγk), k0|bγ(n) := arg min k AM SE bγk (9) = arg min k σ2 C/k + b 2 C,1 A 2(n/k) (if bγ = C) σC2/k + b2 C,2 A 4(n/k) (if bγ = C) = kbγ 0(n)(1 + o(1)).
The bootstrap methodology can thus enable us to consistently estimate the optimal sample fraction (OSF), kbγ
0(n)/n, on the basis of a consistent estimator of k0|bγ(n), in (9), in
a way similar to the one used for classical EVI-estimation (see, for instance, [8]). We shall here use the most obvious auxiliary statistics, the statistics Tk|ˆγ ≡ Tk,n|ˆγ :=γb[k/2]−bγk, k = 2, . . . , n−1, which converge in probability to zero, for intermediate k, and have an asymptotic behaviour strongly related with the asymptotic behaviour of bγk. Indeed, under the
above-mentioned third-order framework in (3), we easily get:
Tk|ˆγ d = σˆγ P ˆ γ k √ k + bˆγ,1(2ρ− 1) A(n/k)(1 + o p(1)) (if bγ = C) bˆγ,2(22ρ− 1) A2(n/k)(1 + o p(1)) (if bγ = C)
with Pkγˆ asymptotically standard normal. Consequently, denoting k0|T(n) :=
arg minkAM SE(Tk), we have
k0|bγ(n) = k0|T(n) (1 − 2ρ)1−2ρ2 (1 + o(1)) (if b γ = C) (1 − 22ρ)1−4ρ2 (1 + o(1)) (if b γ = C).
5.1
How does the bootstrap methodology then work?
Given the sample Xn = (X1, . . . , Xn) from an unknown model F , and the functional
Tk,n =: φk(Xn), 1 < k < n, consider for any n1 = O(n1−), 0 < < 1, the bootstrap
sample X∗n1 = (X1∗, . . . , Xn∗1), from the model Fn∗(x) = n1 Pn
i=1I[Xi≤x], the empirical d.f.
associated to the available sample, Xn. Next, associate to the bootstrap sample the cor-responding bootstrap auxiliary statistic, Tk∗
1,n1 := φk1(X ∗ n1), 1 < k1 < n1. Then, with k0|T∗ (n1) = arg mink1AM SE T ∗ k1,n1, k∗0|T(n1) k0|T(n) =n1 n −1−c ρc ρ (1 + o(1)), c = 2 (if bγ = C) 4 (if bγ = C). (10)
Consequently, for another sample size n2, and for every α > 1,
k0|T∗ (n1) α k0|T∗ (n2) nα 1 nα n n2 −1−c ρc ρ =k0|T(n) α−1 (1 + o(1)).
It is then enough to choose n2 = n (n1/n)α, to have independence of ρ. If we put n2 = n21/n,
i.e., α = 2, we have k0|T∗ (n1)
2
/k∗0|T(n2) = k0|T(n)(1 + o(1)), as n → ∞. We are now
able to estimate kbγ
0(n), on the basis of any estimate ˆρ of ρ. With ˆk∗0T denoting the sample
counterpart of k0T∗ , and ˆρ the ρ-estimate in Step 3. of the algorithm, initiated in Section 3, we have the k0-estimate
ˆ kbγ 0(n; n1) := min n − 1, Cρˆ (ˆk∗0|T(n1))2/ˆk∗0|T([n21/n] + 1) + 1 , (11) with Cρˆ = 1 − 2c ˆρ/2 1−c ˆ2ρ , c given in (10).
Again, with bγ denoting either C or C, we proceed with the algorithm. Algorithm (cont.) (bootstrap adaptive estimation of γ):
5. Next, consider the sub-sample size n1 = n0.955 and n2 = [n21/n] + 1;
6. For l from 1 till B = 250, generate independently, from the empirical d.f. Fn∗(x) =
1 n
Pn
i=1I[Xi≤x], associated with the observed sample, B bootstrap samples (x
∗ 1, . . . , x
∗ n2)
and (x∗1, . . . , x∗n2, x∗n2+1, . . . , x∗n1), of sizes n2 and n1, respectively;
7. Denoting Tk,n∗ the bootstrap counterpart of Tk,n, obtain (t∗k,n1,l, t
∗
k,n2,l), 1 ≤
l ≤ B, the observed values of the statistic Tk,n∗
i, i = 1, 2, compute M SE∗(ni, k) = B1 PBl=1 t∗k,ni,l 2 , k = 1, 2, . . . , ni − 1, and obtain ˆk∗0T(ni) := arg min1≤k≤ni−1M SE ∗ (ni, k), i = 1, 2; 8. Compute ˆkbγ 0(n; n1) in (11); 9. Compute bγn,n∗ 1|T :=bγˆkbγ 0(n;n1).
6
An application to burned areas data
Most of the wildfires are extinguished within a short period of time, with almost negligible effects. However, some wildfires go out of control, burning hectares of land and causing significant and negative environmental and economical impacts. The data we analyse here consists of the number of hectares, exceeding 100 ha, burnt during wildfires recorded in Portugal during 14 years (1990-2003). The data (a sample of size n = 2627) do not seem to have a significant temporal structure, and we have used it as a whole, although we think also sensible, to try avoiding spatial heterogeneity, considering at least three different regions: the north, the centre and the south of Portugal (a study out of the scope of this note).
The box-plot and a histogram of the available data provide evidence on the heaviness of the right-tail. We shall next consider, for this type of data, the performance of the adaptive CVRB-EVI estimates H, in (4), which is minimum variance reduced-bias (MVRB). These MVRB estimators exhibit stabler sample paths than Hill estimators, as functions of k, and enable us to take a decision upon the estimates to be used, even with the help of any heuristic stability criterion. The algorithm in this paper enables us to adaptively estimate the OSF associated with the MVRB or CVRB estimates. For a sub-sample size n1 = [n0.955] = 1843, and B=250 bootstrap generations, we have got ˆkH0∗ = 1319 and the
bootstrap MVRB-EVI-estimate H∗ = 0.658, the value pictured in the following figure, jointly with the above-mentioned adaptive bootstrap Hill estimate, H∗ = 0.73. Note the fact that the MVRB EVI-estimators look practically “unbiased” for the data under analysis.
The MVRB-EVI estimators look practically "unbiased"
0.5 0.6 0.7 0.8 0.9 0 2800
!
H
!
H
!
k
!
H
*= 0.658
!
1319
!
157
!
H
*= 0.73
Figure 2: Estimates of the EVI, γ, through the Hill estimator, H, in (1), and the MVRB es-timator, H, in (4), for the burned areas under analysis, together with the bootstrap adaptive estimates H∗ and H∗.
A few practical questions may be raised under the set-up developed: How does the asymptotic method work for moderate sample sizes? Is the method strongly dependent on the choice of n1? What is the sensitivity of the method with respect to the choice of
ρ-estimate? Although aware of the need of n1 = o(n), what happens if we choose n1 = n?
Answers to these questions are expected not to be a long way from the ones given for classical estimation (see [8]), have lightly been addressed in [13], for reduced-bias estimation, but are out of the scope of this paper.
References
[1] Beirlant, J., Vynckier, P., Teugels, J.: Excess functions and estimation of the extreme-value index. Bernoulli 2, 293-318 (1996)
[2] Beirlant, J., Dierckx, G., Guillou, A.: Estimation of the extreme-value index and generalized quantile plots. Bernoulli 11:6, 949-970 (2005)
[3] Caeiro, F., Gomes, M.I., Pestana, D.D.: Direct reduction of bias of the classical Hill estimator. Revstat 3:2, 111-136 (2005)
[4] Caeiro, F., Gomes, M.I., Henriques-Rodrigues, L.: Reduced-bias tail index estimators under a third order framework. Commun. in Statist. – Theory and Methods. 38:7, 1019-1040 (2009)
[5] Dekkers, A.L.M., Einmahl, J.H.J., Haan, L. de: A moment estimator for the index of an extreme-value distribution. Ann. Statist. 17, 1833-1855 (1989)
[6] Fraga Alves, M.I., Gomes M.I., Haan, L. de: A new class of semi-parametric estimators of the second order parameter. Portugaliae Mathematica 60:2, 194-213 (2003)
[7] Gomes, M.I., Martins M.J.: “Asymptotically unbiased” estimators of the tail index based on external estimation of the second order parameter. Extremes 5:1, 5-31 (2002) [8] Gomes, M.I., Oliveira, O.: The bootstrap methodology in Statistical Extremes — choice
of the optimal sample fraction. Extremes 4:4, 331-358 (2001)
[9] Gomes, M.I., Pestana, D.D.: A sturdy reduced-bias extreme quantile (VaR) estimator. J. American Statistical Association, Vol. 102, no. 477, 280–292 (2007b)
[10] Gomes, M.I., Martins, M.J., Neves, M.M.: Improving second order reduced-bias tail index estimation. Revstat 5:2, 177-207 (2007)
[11] Gomes, M.I., Haan, L. de, Henriques-Rodrigues, L.: Tail Index estimation for heavy-tailed models: accommodation of bias in weighted log-excesses. J. Royal Statistical Society B70, Issue 1, 31-52 (2008)
[12] Gomes, M.I., Pestana, D.D., Caeiro, F.: A note on the asymptotic variance at optimal levels of a bias-corrected Hill estimator. Statist. and Probab. Letters 79:3, 295-303 (2009)
[13] Gomes, M.I., Mendon¸ca, S., Pestana, D.D.: Adaptive reduced-bias tail index and value-at-risk estimation. In: Sakalauskas, L., Skiadas, C., Zavadskas, E.K. (eds.) Applied Stochastic Models and Data Analysis – ASMDA 2009, IMI & VGTU Editions, 41-44 (2009)
[14] Haan, L. de, Ferreira, A.: Extreme Value Theory: an Introduction. Springer Sci-ence+Business Media, LLC, New York (2006)
[15] Hill, B. M.: A simple general approach to inference about the tail of a distribution. Ann. Statist. 3, 1163-1174 (1975)
[16] Peng, L., Qi, Y.: Estimating the first and second order parameters of a heavy tailed distribution. Australian and New Zealand J. of Statist. 46, 305-312 (2004)
[17] Reiss, R.-D. and Thomas, M., Statistical Analysis of Extreme Values with Applications to Insurance, Finance, Hydrology and Other Fields, 3rd edition, Birkh¨auser Verlag, Basel-Boston-Berlin (2007)
