DOI 10.1007/s00382-016-3346-6
Improvements in precipitation simulation over South America
for past and future climates via multi‑model combination
Maytê Duarte Leal Coutinho1 · Kellen Carla Lima1 · Cláudio Moisés Santos e Silva1
Received: 15 December 2015 / Accepted: 6 September 2016 © Springer-Verlag Berlin Heidelberg 2016
data were first subjected to principal component analysis (PCA) and the scores were used to perform the prediction. Keywords Regional models · Principal component regression · Convex combination · Ensemble average · Outliers
1 Introduction
Use of regional climate models (RCM) to perform weather and climate studies has been motivated by the possibility of simulating mesoscale processes, which are associated with the water cycle and ground surface features, such as orographic precipitation (Leung et al. 2004; Salathé et al. 2008).
Errors in the initial conditions can lead to large uncer-tainties in numerical weather prediction models (Zhu and Thorpe 2006). Other uncertainty sources are associated with deficiencies in the physical parameterizations used in the models (Krishnamurti et al. 2004; Van Lier et al. 2012). Thus, several studies have been performed aiming to minimize this uncertainty (Kumar et al. 2001; Tebaldi et al. 2005; Krishnamurti et al. 2009) and to facilitate the analysis of future climate behavior. Knowledge of the systemic errors caused by these uncertainties is criti-cal to improve forecasting systems, aiming to minimize errors and help meteorologists to make better forecasts. The idea of combination is based on the hypothesis that different models carry complementary information and combined simulation can take advantage of this infor-mation synergy (Stocker et al. 2010). Thus, combina-tion (ensemble) models allow simulating the probabil-ity of future events with greater reliabilprobabil-ity compared to Abstract Combining individual forecasts is one of the
practices used to improve weather prediction results. Identifying which combination of techniques results in a more accurate forecast is the subject of many compara-tive studies as well proposals for combined methods. Here we compare three combination techniques: (1) principal component regression (PCR), (2) convex combination by mean squared errors (MSE) and (3) ensemble average to combine six regional climate models of the Regional Cli-mate Change Assessment for the La Plata Basin Project (CLARIS-LPB) for variable rainfall in three regions: Ama-zon (AMZ), Northeastern Brazil (NEB) and La Plata Basin (LPB), for the past (1961–1990) and future (2071–2100) climates. The results indicate that the average RMSE val-ues showed improved representation of climate for LPB in some months, which is an important advance in climate studies. On the other hand, PCR presented greater accuracy (lower RMSE) than MSE in the AMZ and NEB regions. In winter months, both combinations presented lower RMSE results, mainly PCR in the three study regions. The correla-tion coefficient supports the results already found, namely, PCR obtained moderate to strong correlations, which were statistically significant at 5 % in both regions for all months, while MSE presented low to moderate correla-tions, which were statically significant at 5 % only in some months. Based on that, PCR achieved the best corrected forecast, as it was superior in forecasting precipitation due to the lower RMSE value. It is noteworthy that the PCR
* Maytê Duarte Leal Coutinho maytecoutinho@yahoo.com.br
1 Universidade Federal do Rio Grande do Norte – Programa de
Pós-Graduação em Ciências Climáticas, Natal, Rio Grande do Norte, Brazil
deterministic prediction with only one model (Mullen and Buizza 2001).
The combination of simulations results obtained via dynamic models to improve prediction was first discussed by Krishnamurti et al. (1999, 2000a, b) and it has been widely used (Chakraborty et al. 2007; Lenartz et al. 2010; Knutti et al. 2010). There are different techniques to com-bine models, such as the Bayesian method (Robertson et al. 2004), multiple regression by principal components (Da Silva and Silva 2014) or weighed average. In general, the weights are determined using the historical relationship between predictions and observations (Krishnamurti et al. 2000a, b).
Some studies have combined precipitation and tempera-ture simulated by RCM for South America (SA). In gen-eral, these studies have calculated the average of simula-tion to generate the ensemble (Sun et al. 2006; Solman et al. 2013; Sánchez et al. 2015). Concerning the uncer-tainties in simulating precipitation and air temperature, Solman et al. (2013) performed an analysis of the main biases combining seven RCM in the ambit of the CLARIS project. The authors showed biases of ±2 °C and ±20 % for temperature and precipitation, respectively, in different areas of SA. Sánchez et al. (2015) calculated the ensem-ble average of seven CLARIS RCM to evaluate the abil-ity to reproduce the main features of climate conditions in the La Plata Basin (LPB). They showed that the dispersion of changes in precipitation, even for the regions where the models agreed, was about 40 %, suggesting that even when the models agree on the future changes, there is still large uncertainty.
Richardson (2001) showed that the ensemble average and simple bias correction provide better predictions com-pared to the individual models. In 2001, the Center for Weather Forecasting and Climate Studies (CPTEC) began predicting global weather by ensemble average. The first evaluation of this combination scheme was released by Coutinho (1999). This study was based on the objective assessment of the ensemble average performance in rela-tion to control forecast for some cyclogenesis cases, frontal systems and cyclonic vortices that occurred over SA. His results showed that the ensemble average performed better than the control prediction for these events.
In the present work, we attempt to improve the ensem-ble average results by employing more robust techniques to combine models. More specifically, the main goal of this study is to examine linear combinations of the simulated rainfall with six RCM of the Regional Climate Change Assessment for La Plata Basin Project (CLARIS/LPB). The first combination is by multiple linear regression using principal components. Secondly, we used a convex combination (weighted average), where the inverse of the mean square error is the weighting factor. The analyses
were performed in three regions: Amazon (AMZ), North-eastern Brazil (NEB) and La Plata Basin (LPB), for the past (1961–1990) and future (2071–2100) climates. The article is organized as follows: Sect. 2—data and meth-odology; Sect. 3—results; Sect. 4—discussion; and Sect. 5—conclusions.
2 Materials and methods
2.1 Study regions
The overall area covers most of SA, as shown in Fig. 1. The three regions, AMZ, NEB, and LPB, were chosen because previous studies have indicated considerable changes in rainfall values projected by climate models up to the end of the twenty-first century. Additionally, the two Brazil-ian regions, Amazon (tropical) and Northeast (semiarid) are considered areas of climatic vulnerability (Meehl et al. 2007; Marengo et al. 2010).
2.2 Rainfall data
We used rainfall data from the Global Precipitation Clima-tology Center (GPCC), which are monthly data arranged in a horizontal grid of 0.5° × 0.5° between latitudes 50° S and 50° N. The data cover only the continental areas and were collected from 67,200 rain gauges distributed around the world (Schneider et al. 2008). The studied period was from 1961 to 1990, representing the past climate.
2.3 Regional models
We used results of numerical simulations from RCM within the CLARIS project: the Eta model from National Institute for Space Research (INPE—Instituto Nacional
de Pesquisas Espaciais); the Rossby Centre RCA regional climate model; the Promes model, from Universidad de Castilla-La Mancha; the RegCM3 model from the Inter-national Center for Theoretical Physics; Max-Planck-Institute for Meteorology regional model (REMO); and
Modele de Circulation Generale du Laboratoire de
Mete-orologie Dynamique (LMDZ). The RCM were driven
by two CMIP3 General Circulation Models (GCMs) for emission scenario SRES A1B, also based on the study of Coutinho et al. (2016).
These models were evaluated by Menéndez et al. (2010), Da Rocha et al. (2012) and Ferraz and Pedroso (2013). According to Solman et al. (2013), these RCM are consist-ent according to the weather and the biases vary by ±2 °C for temperature and ±20 % for precipitation. The GCM/ RCM simulation matrix formulated as part of the CLARIS-LPB project is shown in Table 1. The RCM used here have horizontal grid spacing of 0.5° latitude by 0.5° longitude. The meteorological variable used was rainfall (mm/month) for the past (1961–1990) and future (2071–2100) climates. 2.4 Combination of simulations
2.4.1 Multiple linear regression using principal components
Multiple linear regression (MLR) is a statistical technique consisting of finding a linear relation between a depend-ent variable and more than one independdepend-ent variable. The mathematical model is presented by the following equation:
(1)
Yi=β0+β1X1i+β2X2+ · · · +βmXm+εi
where: Yi is the dependent variable; X1,X2 and Xm are the
independent variables; β0 is the intercept; and β1,β2 and βm
are the multiple regression coefficients, estimated by the least squares method (Wilks, 2006); and εi is the error. The
challenge of MLR is to determine the coefficients βm relat-ing independent and dependent variables. To find this solu-tion, we rewrite Eq. 1 in matrix form as follows.
Multiplying matrix-X by matrix-B and adding matrix-ε gives the following equation in matrix form:
The least squares method is used to determine MLR coefficients, with the condition that the sum of the squared errors be minimized (Martens 1992; Roggo et al. 2007). Isolating the error in (1), we have:
Then the sum of the squared errors (SQE), shown in (4) and in matrix form (5), is smallest where the derivative with respect to matrix-B is equal to zero, as shown in (6). The MLR solution in (7) is found by isolating matrix-B (step not shown).
Y = Y1 .. . Yi = 1X11 Xm1 .. . ... ... 1 X1i Xmi · β0 .. . βi + ε1 .. . εi (2) Y = XB + ε (3) Yi−(β0+β1X1i+β2X2+ · · · +βmXmi) = εi (4) SQE = (Yi−(β0+β1X1i+β2X2+ · · · +βmXmi))2 (5) SQE = n i=1 εi2=εTε = (Y − XB)T ·(Y − XB) (6) ∂(SQE) ∂B =0
Table 1 Matrix of GCM/RCM combinations of regional climate change simulations over South America performed in the CLARIS-LPB pro-ject RCM GCM Past climate (1961–1990) Near future (2011–2040) Far future (2071–2100) Continous run (1961–2100) Eta
Chou and Lan (2012), Pesquero et al. (2010)
HadCM3 X X X
RegCM3
Pal et al. (2007), da Rocha et al. (2009)
HadCM3 X X X
REMO (Majewski 1991)
EC5OM X X X
Promes
Sánchez et al. (2007), Domínguez et al. (2010)
HadCM3 X
LMDZ Sadourny et al. (1995), (Li and Conil 2003) HadCM3 X
RCA
Samuelsson et al. (2010, 2011)
However, to solve Eq. 7, the matrix XTX might not
be invertible. In other words, it can be a singular matrix where some predictor variables are linear combinations of each other, causing multicollinearity of the inde-pendent variables, so no single least squares parameter is estimated. Application of linear models needs uncor-related independent variables, but this is not the case for simulated rainfall with different RCM (independent variables).
To solve this problem, principal components analysis (PCA) can be used. This reduces the number of independ-ent variables and simultaneously generates a set of orthog-onal functions which avoid the multicollinearity problem. Thus, the PCs of the explanatory variables form a new set of variables that bring the same information as the original variables, but without correlation (Lee et al. 2012). The use of PCA to fit a MLR model was initially suggested by Ken-dall (1957). In this procedure, the first step is to find the PC of matrix-Z or the matrix of predictor variables X, given by the following relation:
where P′ is an orthogonal matrix of dimension m × m (in
which m is the number of predictor variables), consisting of eigenvectors of the covariance matrix or correlation
matrix-X. Then, rewriting Eqs. (2) and (7), the following equation is produced:
The second step is to determine the matrices P and B containing, respectively, the weights for each simulation and regression coefficient of the model. The eigenvalues of matrix-P provide weights of each variable and are used as predictors to calculate the new array of PC (Znew) of new simulations (Xnew), given by:
Finally, the third step is to use the coefficient of matrix-B to find the PC, and then the prediction set (Zprev), obtained by the following relation:
The validation of the MLR model consists of follow-ing two assumptions (Wilks, 2006): (1) the residuals have random distribution around mean zero (homosce-dasticity) and normal distribution; and (2) the variance is homogeneous. (7) B = XTX −1 XTY (8) Z = P′X (9) Y = ZB + ε (10) B = ZTZ −1 ZTY (11) Znew=P′Xnew (12) Zprev=ZnewB
2.4.2 Convex combination (weighted average)
This method consists of combining the results of tions, through the distribution of weights for each simula-tion. The total sum of weights is equal to 1. In the present work, we used the inverse of mean squared error (MSE) to weight the convex combination. Thus, larger weights are attributed to simulations with lower MSE and vice versa (Phillips et al. 1992; Paiva et al. 2008).
With Po and Ps being, respectively, the observed and
simulated rainfall, the MSE is defined as follows:
For the weighted average, the following equation is applied:
1. The sum of the weights of each simulation result:
2. The weighted sum, given by the inverse of the MSE of each simulation:
Therefore, the weighted average (¯xi) is given by:
where N is the number of observations, Mi are the simula-tions with the models, and i = 1, 2, …, 6.
2.4.3 Ensemble average
The calculation the average forecast, considering that all members have the same probability of representing the future state of the atmosphere. The ensemble mean is given by:
where N is the number of observations, Mi are the simula-tions with the models, and i = 1, 2, …, 6.
(13) MSE = 1 N N i=1 (Ps−Po)2 (14) N i=1 1 MSEMi ×Mi = 1 MSEM1 ×M1+ 1 MSEM2 ×M2+ · · · + 1 MSEM6 ×M6 (15) N i=1 1 MSEMi = 1 MSEM1 + 1 MSEM2 + · · · + 1 MSEM6 (16) ¯ xi = N i=1 1 MSEMi ×Mi N i=1 1 MSEMi AVE = 1 N N i=1 Mi
A basic evaluation of the ensemble average model biases and change signals is given in the paper by Coppola et al. (2014).
2.5 Skill metric to assess performance of the ensemble methods
Various performance measures can be used to compare the results and their associated errors between observed and historical (simulated) data. In this study, we evaluated the performance of techniques to combine models from the monthly rainfall data of the GPCC for the past climate (1961–1990) with combinations by principal component (PCR), convex analysis (MSE) and ensemble average (AVE). The techniques were compared through boxplots, which allows analyzing the data symmetry, dispersion and the existence of outliers, and is especially suitable for com-paring two or more sets of data corresponding to the cat-egories of a quantitative variable, in this case precipitation.
2.5.1 Ratio of standard deviations (Ra)
The nearer the ratio of standard deviations is to one, the greater the similarity between those deviations. We ana-lyzed this by the following equations:
where Po are the observations and ¯Po the observation
aver-age. Ps corresponds to MSE or PCR results and ¯Ps
corre-sponds to the average these combinations.
2.5.2 Square root of the mean squared error (RMSE)
where Poi is the observed data point and Psi, corresponds the combinations.
2.5.3 Pearson’s correlation coefficient (rp)
This measure indicates the degree of linear correla-tion between two variables. The values can be classified
(17) σobs= 1 N (Po− ¯Po)2 1/2 (18) σsim= 1 N (Ps− ¯Ps)2 1/2 (19) Ra= σsim σobs (20) RMSE = 1 N N i=1 (Psi−Poi)2
following the method proposed by Hopkins (2009), pre-sented in Table 2.
The statistical significance is calculated by the F-test, adopting 5 % significance level, where Poi is the observed data point, ¯Poi is the average of the observed data, Psi,
cor-responds the combinations and ¯Psi is the average value of
the series of simulated combinations.
2.5.4 Willmott’s index of agreement (d)
This index measures the level of agreement between esti-mated and observed values, ranging from 0 (no agreement) to 1 (perfect agreement). It is determined by:
where Poi is the observed data point, ¯Poi is the observed
data average, Psi, corresponds to the simulated data from the combinations and ¯Psi corresponds to the average value
of the series of simulated combinations.
3 Results
3.1 Climatology
Figure 2 shows the boxplot of monthly rainfall over the 30-year period from 1961 to 1990 in the three study regions. December to March is the period with highest rainfall in the AMZ region (Fig. 2a) and the maximum median value is 270 mm in March. April and October are transition months and the minimum precipitation occurs in August, with median around 100 mm. The meridional migration of Intertropical Convergence Zone (ITCZ) is the main large-scale meteorological system associated with
(21) rp= n i=1(Poi− ¯Po)(Psi− ¯Ps) n i=1(Psi− ¯Ps)2 ni=1(Poi− ¯Po)2 (22) d = 1 − n i=1(Psi−Poi)2 n i=1(Psi− ¯Poi) + (Poi− ¯Poi)2
Table 2 Classification of correlations according to the correlation coefficient following the method of Hopkins (2009)
Source: Hopkins (2009)
Correlation coefficient (rp) Correlation
0|— 0.01 Very low 0.1|— 0.3 Low 0.3|— 0.5 Moderate 0.5|— 0.7 High 0.7|— 0.9 Very high 0.9|— 1.0 Almost perfect
seasonal variability in AMZ. In addition, we observe outli-ers below 200 mm in January and around 220 mm in Feb-ruary, which are usually associated with interanual varia-bility of the Pacific (El Niño Southern Oscilation - ENSO) and Atlantic (interhemispheric sea surface temperature gradient) oceans.
In the semiarid NEB (Fig. 2b), the highest precipitation values are observed from December to April, with maxi-mum median of 180 mm in March. The median of low-est values does not reach 50 mm from June to September, while extreme wet months are observed during the rainy season, with total monthly rainfall above 300 mm. At same time, excessive precipitation can be associated with
anomalous Southern displacement of ITCZ, ENSO and Atlantic Interhemispheric SST gradient.
Figure 2c shows the monthly distribution for the LPB region, with a well-defined annual cycle of precipitation in which the rainy season is concentered during summer months and the dry season during winter. The rainy sea-son is affected by the South Atlantic Convergence Zone (SACZ) and transient systems, which are responsible for excessive amounts of rainfall. Outliers (extreme wet months) are also observed at the end of autumn and winter (dry months) and December (rainy month). Nevertheless, a Mesoscale Convective Complex (MCC) is frequently observed in this region, causing extreme rainfall events.
Fig. 2 Observed monthly precipitation (GPCC) for the three regions: AMZ (a), NEB (b) and the LPB (c), for the period 1961–1990. Units in mm/month
3.2 Regression model via principal component analysis First we analyzed the principal component (PC) of monthly data of GPCC and the rainfall simulated with six RCM (Eta, RegCM3, Promes, RCA, LMDZ and REMO) sepa-rated by past and future climate months, as shown in Tables 3 and 4. With analysis of the residuals, we con-firmed the homoscedasticity assumptions (i.e., average of residuals equal to zero and variance constant) for past and future climates and regions (not shown). Monthly QQ-plots (not shown) indicated normal distribution of the residu-als in all regions. Therefore, the data fit the multiple linear regression model (MLR) by PC.
The importance of a PC is assessed by its contribution, i.e., by the proportion of total variance explained by the component. In the AMZ region, PC1 + PC2 + PC3 explain 71 % of the variance in May and 86 % in November (Table 3). The lowest value coincides with the dry season (without much convective activity) and November (which has a higher value), consistent with rains or strong convec-tive activity in the Amazon region. The PC1 + PC2 + PC3 in the NEB region explain 73 % of the variance in 2 months (March and July) and 86 % in June. In March the rainfall might be associated with the Intertropical Convergence Zone (ITCZ) system, as stated by Moura and Shukla (1981) and Wang (2002). In turn, from June and July the main influence can be associated with the Easterly Wave Disturbances (EWD) (Mota, 1997; Coutinho and Fisch 2007). Unlike other regions, the LPB requires fewer princi-pal components to explain 70 % of the total data variability. This situation is associated with the higher variance (per-centage) that each PC represents in the total variance.
Second, we used the PCR to analyze the future climate (Table 4) because these models area able to estimate val-ues near the observed ones. For the future climate in the AMZ (Table 4), the cumulative proportion of PC1 + PC2 explains 70 % in February and October, rainy and transi-tion months, respectively, and explains 73 % in Septem-ber, which is end of the dry season. In the NEB region, PC1 + PC2 explain 71 % in June and 73 % in September. Likewise, in LPB, PC1 + PC2 explain around 73 % in June and August in the LPB.
3.3 Comparison of combined techniques
The comparison of the three techniques (PCR, MSE and AVE) in representing the monthly rainfall is shown in the boxplots of Figs. 3, 4, 5, 6, 7 and 8 in different regions and climates.
Figure 3 refers to the AMZ region for the past cli-mate. The three techniques manage to reduce the variance of residuals in the combined forecast compared to the observed data. Comparing the three techniques combined
with the GPCC shows that the MSE underestimated the rainfall observed in all months except October, November and December, where rainfall is overestimated. The PCR values are slightly smaller than the corresponding MSE ones and the PCR adequately simulates rainfall values near or equal to the observed ones.
Table 5 shows that the correlation in MSE is smaller than the PCR. In February, for example, the MSE has negative linear correlation of −0.35, considered moderate (Table 2), although this correlation is not statistically significant at the 5 % level (p value of 0.222). In the same month, there is a positive correlation of 0.62 in PCR that is statistically significant at the 5 % level (p value of 0.013). This means that for the month in question, the MSE does not show con-sistent or significant data at 95 % confidence by present-ing a p value of 0.222. In September, the MSE shows a positive correlation of the 0.24 while the PCR presents a positive correlation of the 0.55, considered low and moder-ate, respectively (Table 2). For both cases, the correlations are statically significantly at 5 % (Table 5). The forecast improvements in the PCR are notable in all months, mainly June to October, with lower rainfall values (Figs. 2a, 3).
Moreover, the three techniques capture extreme values (outliers) in different months (Fig. 3). The MSE captures dry extremes in March, October and December and PCR only in October (transition month). With respect to wet extremes, the MSE captures the months of March, April and August and PCR June and July (winter).
Although the AMZ and NEB are part of the tropical region, they have different responses to the techniques applied. In general, the response of the AMZ region con-verges to the observed values, presenting smaller bias compared to the NEB. The correlations for both combina-tion techniques (Table 5) are statistically significant at the 5 % level in these months with the exception of August (p = 0.059) for MSE. Furthermore, MSE continues present-ing a low correlation and PCR a moderate to strong correla-tion. As regards outliers, MSE captures more extreme wet events than PCR. Extreme wet events are verified in May, August, September, October and November for MSE and in January, October and December for PCR. Regarding the dry extremes, PCR captures more such events than MSE: these are found in January, March, August, September and December in PCR and only in August in MSE, thus dem-onstrating that extreme wet events are found more by MSE and more extreme dry events are captured better by PCR.
In the LPB region (Fig. 5) it can be seen that MSE under-estimates the observed rainfall (GPCC) in every month, while PCR has performance close to the observed values, since the medians (which measure the central tendency of a dataset) are close in most months. Both combination tech-niques have smaller variability in the precipitation values compared with the GPCC data. The correlations presented
in MSE are low to moderate, while they are moderate to strong in PCR (Table 5). All months show statistically sig-nificant values at 5 % for the two combination techniques, except in February in MSE (p value = 0.12).
Regarding outliers captured by the three combina-tion techniques in the LPB region (Fig. 5), it is possible to observe that MSE captures extreme wet events in the
months of April and November, while PCR does this in October. As also occurred in the NEB region, more extreme dry events are found by PCR, observed in March, April, June and July, versus only June and November in MSE.
For future climate (Figs. 6, 7, 8), we compared the three techniques with the principal component regression (PCR) model for past climate (PAST), as mentioned before. Note
Table 3 Proportion of variance and cumulative variance for the three study regions of each principal component (PC) with the GPCC data observed in the past climate (1961–1990)
Past climate
Proportion of variance Cumulative proportion
PC1 PC2 PC3 PC4 PC5 PC6 PC1 PC2 PC3 PC4 PC5 PC6 AMZ January 0.33 0.25 0.17 0.12 0.08 0.04 0.33 0.58 0.75 0.87 0.96 1.00 February 0.34 0.30 0.17 0.10 0.06 0.02 0.34 0.64 0.81 0.92 0.98 1.00 March 0.36 0.24 0.18 0.13 0.06 0.03 0.36 0.60 0.79 0.92 0.97 1.00 April 0.31 0.27 0.19 0.11 0.08 0.03 0.31 0.59 0.77 0.88 0.97 1.00 May 0.32 0.21 0.18 0.15 0.11 0.03 0.32 0.53 0.71 0.86 0.97 1.00 June 0.37 0.21 0.17 0.12 0.09 0.04 0.37 0.58 0.75 0.86 0.96 1.00 July 0.31 0.26 0.17 0.12 0.10 0.04 0.31 0.56 0.74 0.86 0.96 1.00 August 0.43 0.20 0.16 0.11 0.07 0.03 0.43 0.63 0.79 0.90 0.97 1.00 September 0.36 0.28 0.15 0.11 0.08 0.03 0.36 0.64 0.79 0.90 0.97 1.00 October 0.39 0.29 0.16 0.08 0.07 0.01 0.39 0.68 0.84 0.92 0.99 1.00 November 0.40 0.26 0.20 0.07 0.05 0.02 0.40 0.66 0.86 0.93 0.98 1.00 December 0.39 0.18 0.17 0.15 0.07 0.04 0.39 0.57 0.74 0.89 0.96 1.00 NEB January 0.33 0.26 0.18 0.11 0.07 0.04 0.33 0.59 0.78 0.89 0.96 1.00 February 0.41 0.23 0.16 0.12 0.05 0.03 0.41 0.64 0.80 0.93 0.97 1.00 March 0.33 0.23 0.17 0.14 0.09 0.05 0.33 0.56 0.73 0.87 0.95 1.00 April 0.34 0.29 0.15 0.09 0.08 0.05 0.34 0.62 0.78 0.87 0.95 1.00 May 0.42 0.22 0.16 0.10 0.06 0.03 0.42 0.65 0.81 0.91 0.97 1.00 June 0.41 0.26 0.18 0.07 0.05 0.02 0.41 0.68 0.86 0.93 0.98 1.00 July 0.33 0.20 0.19 0.11 0.09 0.07 0.33 0.53 0.73 0.84 0.93 1.00 August 0.33 0.28 0.19 0.13 0.04 0.03 0.33 0.61 0.80 0.93 0.97 1.00 September 0.37 0.21 0.19 0.13 0.06 0.05 0.37 0.57 0.76 0.90 0.95 1.00 October 0.37 0.27 0.15 0.13 0.06 0.03 0.37 0.64 0.79 0.92 0.97 1.00 November 0.35 0.23 0.19 0.13 0.07 0.03 0.35 0.56 0.77 0.90 0.97 1.00 December 0.36 0.26 0.16 0.11 0.07 0.04 0.36 0.62 0.78 0.89 0.96 1.00 LPB January 0.29 0.22 0.16 0.15 0.13 0.05 0.29 0.51 0.68 0.82 0.95 1.00 February 0.35 0.20 0.17 0.13 0.10 0.05 0.35 0.55 0.71 0.84 0.95 1.00 March 0.31 0.22 0.21 0.16 0.07 0.03 0.31 0.54 0.74 0.91 0.97 1.00 April 0.37 0.26 0.17 0.14 0.05 0.02 0.37 0.62 0.79 0.93 0.98 1.00 May 0.36 0.34 0.15 0.12 0.02 0.01 0.36 0.70 0.85 0.97 0.99 1.00 June 0.36 0.31 0.18 0.10 0.04 0.01 0.36 0.67 0.84 0.95 0.99 1.00 July 0.34 0.29 0.17 0.15 0.03 0.01 0.34 0.64 0.81 0.96 0.99 1.00 August 0.49 0.19 0.17 0.09 0.04 0.01 0.49 0.69 0.86 0.95 0.99 1.00 September 0.38 0.26 0.18 0.10 0.07 0.01 0.38 0.65 0.82 0.92 0.99 1.00 October 0.47 0.21 0.16 0.08 0.06 0.01 0.47 0.69 0.85 0.93 0.99 1.00 November 0.37 0.23 0.15 0.13 0.09 0.03 0.37 0.59 0.75 0.88 0.97 1.00 December 0.43 0.22 0.16 0.10 0.06 0.03 0.43 0.66 0.82 0.91 0.97 1.00
that in all regions for future climate, the MSE technique has difficulty in reducing variability residues of models as compared to the reference (PAST), while PCR reduces this variability and consequently the error, as also seen for the past climate.
In the AMZ region, (Fig. 6), MSE indicates the cli-mate will be drier than in the past in every month except
November and December (rainy season or strong convec-tive activity in the Amazon region), which will be wetter than in the past. MSE has greater variability in the amounts of precipitation than PCR, as can be clearly observed in the size of the box. In addition, few extreme wet and dry events are observed in this region. MSE captures wet extremes in October and November (and also two dry extremes in
Table 4 Proportion of variance and cumulative variance for the three study regions of each principal component (PC) with the regression model for future climate (2071–2100)
Future climate
Proportion of variance Cumulative proportion
PC1 PC2 PC3 PC4 PC5 PC6 PC1 PC2 PC3 PC4 PC5 PC6 AMZ January 0.29 0.27 0.17 0.12 0.09 0.05 0.29 0.56 0.73 0.85 0.95 1.00 February 0.40 0.31 0.15 0.08 0.04 0.03 0.40 0.70 0.85 0.93 0.97 1.00 March 0.41 027 0.17 0.08 0.04 0.03 0.41 0.68 0.85 0.92 0.97 1.00 April 0.38 0.26 0.15 0.11 0.06 0.05 0.38 0.64 0.78 0.89 0.95 1.00 May 0.40 0.22 0.15 0.10 0.08 0.06 0.40 0.61 0.76 0.87 0.94 1.00 June 0.34 0.28 0.16 0.11 0.06 0.04 0.34 0.63 0.78 0.90 0.96 1.00 July 0.39 0.24 016 0.10 0.07 0.04 0.39 0.63 0.79 0.89 0.95 1.00 August 0.40 0.22 0.18 0.11 0.06 0.04 0.40 0.62 0.80 0.91 0.96 1.00 September 0.44 0.29 0.14 0.05 0.04 0.03 0.44 0.73 0.87 0.92 0.97 1.00 October 0.39 0.31 0.16 0.07 0.05 0.02 0.39 0.70 0.86 0.93 0.98 1.00 November 0.32 0.27 0.16 0.14 0.08 0.04 0.32 0.59 0.75 0.89 0.96 1.00 December 0.33 0.21 0.17 0.15 0.09 0.05 0.33 0.53 0.70 0.86 0.95 1.00 NEB January 0.37 0.20 0.18 0.11 0.08 0.06 0.37 0.57 0.75 0.86 0.94 1.00 February 0.30 0.21 0.20 0.16 0.09 0.04 0.30 0.51 0.71 0.87 0.96 1.00 March 0.32 0.23 0.16 0.13 0.11 0.05 0.32 0.52 0.71 0.85 0.95 1.00 April 0.34 0.25 0.14 0.12 0.10 0.05 0.34 0.59 0.72 0.85 0.95 1.00 May 0.43 0.23 0.18 0.08 0.05 0.03 0.43 0.67 0.85 0.92 0.97 1.00 June 0.48 0.23 0.17 0.08 0.03 0.01 0.48 0.71 0.88 0.96 0.99 1.00 July 0.34 0.28 0.18 0.12 0.05 0.03 0.34 0.62 0.80 0.92 0.97 1.00 August 0.35 0.26 0.17 0.10 0.08 0.03 0.35 0.61 0.78 0.88 0.97 1.00 September 0.42 0.31 0.13 0.08 0.05 0.02 0.42 0.73 0.85 0.93 0.98 1.00 October 0.32 0.31 0.18 0.12 0.05 0.03 0.32 0.62 0.80 0.92 0.97 1.00 November 0.37 0.29 0.15 0.08 0.06 0.04 0.37 0.66 0.81 0.90 0.96 1.00 December 0.37 0.28 0.17 0.08 0.07 0.03 0.37 0.65 0.82 0.90 0.97 1.00 LPB January 0.31 0.23 0.17 0.15 0.10 0.04 0.31 0.54 0.71 0.86 0.96 1.00 February 0.31 0.25 0.21 0.12 0.06 0.04 0.31 0.57 0.78 0.90 0.96 1.00 March 0.37 0.27 0.14 0.09 0.07 0.06 0.37 0.65 0.79 0.87 0.94 1.00 April 0.41 0.20 0.16 0.15 0.05 0.03 0.41 0.61 0.77 0.92 0.97 1.00 May 0.37 0.32 0.15 0.12 0.03 0.02 0.37 0.68 0.83 0.95 0.98 1.00 June 0.46 0.27 0.15 0.09 0.02 0.01 0.46 0.73 0.88 0.97 0.99 1.00 July 0.41 0.28 0.18 0.10 0.03 0.01 0.41 0.68 0.86 0.96 0.99 1.00 August 0.50 0.23 0.17 0.06 0.03 0.01 0.50 0.73 0.90 0.96 0.99 1.00 September 0.37 0.31 0.15 0.10 0.04 0.03 0.37 0.67 0.83 0.93 0.97 1.00 October 0.39 0.20 0.16 0.10 0.09 0.05 0.39 0.59 0.75 0.85 0.95 1.00 November 0.32 0.30 0.21 0.07 0.06 0.04 0.32 0.62 0.83 0.90 0.96 1.00 December 0.34 0.31 0.16 0.11 0.05 0.03 0.34 0.65 0.81 0.92 0.97 1.00
November), while PCR captures wet extremes in January and February and extreme dry events in January.
For the NEB, MSE also indicates a drier future climate than in the past in all months except December, January and February, when MSE indicates a wetter climate (Fig. 7). In contrast, PCR continues to present values near the past, for example, in June (with lower precipitation values). As
regards variability of the data, this is higher in MSE than PCR. Regarding outliers, MSE shows humid extremes in February, March, April and September, while PCR shows wet extremes in September and dry ones in April, Septem-ber and DecemSeptem-ber.
Finally, in the LPB region (Fig. 8), MSE shows a drier climate compared to the past in nearly all months: only
Fig. 3 Comparison by boxplot of the combination of techniques (PCR, MSE and AVE) in estimating the monthly precipitation (mm) observed (GPCC) for the AMZ region in the past climate (1961–1990)
October and November do not show a marked difference. With respect to the variation of data, the PAST and PCR are quite similar and MSE has larger variation.
With respect to AVE technique, it was observed that dur-ing the months (Figs. 3, 4, 5) values were inconsistent in comparison with that observed. It is clear that the AVE is overestimating and/or underestimating the observed (past climate). In the future climate, the AVE in many instances,
differs from the behavior of other techniques (PCR and MSE). A clear example is the NEB region in the months December to March, the technique simulates a wetter future. In other months, simulates a more serious future with regards to be drier or more humid in the other regions. Thus, it is not a suitable technique proposed for this study. And so, only the techniques (PCR and MSE) will be evalu-ated statistically.
Fig. 5 As in Fig. 3, but for the LPB region
05 0 100 150 200 250 300 350 January
PAST MSE PCR AVE
0 50 100 150 200 250 300 350 February
PAST MSE PCR AVE
05 0 100 150 200 250 300 350 March
PAST MSE PCR AVE
0 50 100 150 200 250 300 350 April
PAST MSE PCR AVE
05 0 100 150 200 250 300 350 May
PAST MSE PCR AVE
05 0 100 150 200 250 300 350 June
PAST MSE PCR AVE
0 50 100 150 200 250 300 350 July
PAST MSE PCR AVE
0 50 100 150 200 250 300 350 August
PAST MSE PCR AVE
05 0 100 150 200 250 300 350 September
PAST MSE PCR AVE
0 50 100 150 200 250 300 350 October
PAST MSE PCR AVE
05 0 100 150 200 250 300 350 November
PAST MSE PCR AVE
05 0 100 150 200 250 300 350 December
PAST MSE PCR AVE
Fig. 6 Boxplot comparing the regression model via PC for past climate (PAST) and combination techniques (PCR, MSE and AVE) for future climate (2071–2100) in the AMZ region
3.4 Performance of the ensemble technique
3.4.1 RMSE evaluation
Figures 9, 10 and 11 show the error measures for the areas indicated in Fig. 1 from January to December. To simplify the discussion, the RMSE of the combinations through multiple regression by principle components is referred to as PCR and by convex combination as MSE. Overall,
reductions in RMSE for predictions with PCR (dashed curve) are particularly noteworthy in all regions.
In the AMZ region, (Fig. 9), the forecast corrected by PCR presents satisfactory results in the winter and spring, with monthly RMSE values below 40 mm/month. In the summer and autumn, these remain within the range of 40–60 mm/month. Both techniques have similar values for August, September and December. In general, in the summer months (December, January and February), PCR
05 0 100 150 200 250 300 350 January
PAST MSE PCR AVE
05 0 100 150 200 250 300 350 February
PAST MSE PCR AVE
0 50 100 150 200 250 300 350 March
PAST MSE PCR AVE
05 0 100 150 200 250 300 350 April
PAST MSE PCR AVE
0 50 100 150 200 250 300 350 May
PAST MSE PCR AVE
05 0 100 150 200 250 300 350 June
PAST MSE PCR AVE
0 50 100 150 200 250 300 350 July
PAST MSE PCR AVE
0 50 100 150 200 250 300 350 August
PAST MSE PCR AVE
0 50 100 150 200 250 300 350 September
PAST MSE PCR AVE
0 50 100 150 200 250 300 350 October
PAST MSE PCR AVE
05 0 100 150 200 250 300 350 November
PAST MSE PCR AVE
05 0 100 150 200 250 300 350 December
PAST MSE PCR AVE
Fig. 7 As in Fig. 6, but for the NEB region
0 50 10 0 15 0 20 02 50 30 03 50 January
PAST MSE PCR AVE
05 01 00 15 02 00 25 0 30 0 35 0 February
PAST MSE PCR AVE
0 50 10 01 50 20 0 25 0 30 03 50 March
PAST MSE PCR AVE
05 01 00 15 02 00 25 0 30 0 35 0 April
PAST MSE PCR AVE
05 01 00 15 02 00 25 03 00 35 0 May
PAST MSE PCR AVE
0 50 10 01 50 20 02 50 30 03 50 June
PAST MSE PCR AVE
05 01 00 15 0 20 0 25 03 00 35 0 July
PAST MSE PCR AVE
0 50 10 01 50 20 0 25 0 30 03 50 August
PAST MSE PCR AVE
05 01 00 15 0 20 0 25 0 30 03 50 September
PAST MSE PCR AVE
05 01 00 15 0 20 0 25 03 00 35 0 October
PAST MSE PCR AVE
05 01 00 15 02 00 25 03 00 35 0 November
PAST MSE PCR AVE
05 0 10 0 15 0 20 0 25 0 30 03 50 December
PAST MSE PCR AVE
presents slightly lower values than MSE, with some years reaching values of 50 mm/month, while MSE shows val-ues reaching 80 mm/month. The combination with PCR in some months and years produces slightly higher values than MSE.
In the NEB region (Fig. 10), PCR presents less disper-sion compared to MSE in virtually every month. In April, PCR presents a RMSE value greater than by MSE in the years 1968, 1974 and 1980 (Fig. 10). The curves of PCR and MSE in the winter months have lower dispersion, rang-ing from 0 to 22 mm/month. The highest errors of MSE are from December to March in most years, as noted in 1981, with a maximum around 189 mm/month in February. Also, both techniques show substantially higher RMSE values in late spring and summer.
For the LPB region (Fig. 11), which is considered a region with low to medium representation of climate, PCR in general provides a considerable improvement. However, in a few months PCR presents slightly larger errors than those found with MSE (Fig. 11), as can be observed in a few years.
Furthermore, in some years MSE presents peaks above of 90 mm in nearly all months, with a maximum value of 161 mm in May 1983 (Fig. 11). In turn, PCR shows much lower values, indicating less dispersion in this fore-cast, with peak in 1983, also in May, of around 102 mm. As with the other regions, PCR shows peaks above 60 mm in some years. The results for September and October are particularly good, with very small errors, ranging from 0 to 35 mm in PCR.
3.4.2 Skill indices
Table 6 shows the indices applied in this study to assess the accuracy of the combinations (MSE and PCR) according to Willmott’s index of agreement (d). We also carried out an additional test to determine the most realistic prediction within a set of simulations of a single case, based on the ratio between standard deviations (Table 6).
The highest agreement indices found among the three regions (Table 6) are observed in the AMZ and LPB via PCR, with values of 0.76 and 0.73 (January and February, respectively) for AMZ and 0.78 in January for LPB. These results corroborate the data dispersion results presented in Figs. 9 and 11, with smaller errors.
In the AMZ region, rainfall values in January and Feb-ruary range from 0 to 40 mm/month, while the dispersion is narrower in the LPB region, around 0–30 mm/month. It is clear that the best corrected forecast was produced by PCR, with indices ranging from 0.35 to 0.78 among the three regions (Table 6), while the prediction corrected by MSE shows values of 0.11–0.53 among the three regions (Table 6), with higher concordance found in the AMZ region in August (d = 0.53). Another interesting example is that in the LPB region, PCR reaches a maximum (d = 0.78) in January, very close to 1.0, demonstrating the quality and reliability of these results when comparing the corrected forecast with the observed data.
MSE has lower concordance level than PCR in the three regions studied in nearly all months, because MSE was penalized by the low correlation coefficient (Table 5),
Table 5 Correlation coefficient and p-value between the combination techniques and observed data (GPCC) in past climate for all regions
* No significant at 5 %
Correlation coefficient p value
GPCC × MSE GPCC × PCR GPCC × MSE GPCC × PCR
AMZ NEB LPB AMZ NEB LPB AMZ NEB LPB AMZ NEB LPB
Past climate January 0.00 −0.32 −0.36 0.64 0.55 0.68 0.001 0.001 <0.0005 0.018 0.008 0.042 February −0.35 −0.12 0.27 0.62 0.40 0.54 0.222* 0.009 0.12* 0.013 <0.0005 0.002 March 0.20 −0.14 −0.07 0.43 0.41 0.38 0.075* 0.008 0.005 <0.0005 <0.0005 0.0004 April −0.01 −0.25 −0.14 0.30 0.43 0.29 0.002 <0.0005 <0.0005 <0.0005 <0.0005 <0.0005 May 0.05 −0.18 −0.01 0.30 0.39 0.52 <0.0005 0.0022 <0.0005 <0.0005 <0.0005 0.0009 June 0.06 0.03 −0.12 0.54 0.33 0.49 0.075* 0.02171 <0.0005 0.0014 <0.0005 0.0002 July −0.05 0.17 −0.01 0.53 0.36 0.27 0.0001 <0.0005 <0.0005 0.001 <0.0005 <0.0005 August 0.23 0.09 −0.19 0.46 0.51 0.29 0.168* 0.059* 0.006 <0.0005 0.0004 <0.0005 September 0.24 −0.34 0.06 0.55 0.48 0.44 0.034 0.020 0.031 0.002 0.0002 <0.0005 October 0.01 0.15 0.28 0.53 0.40 0.38 0.412* 0.008 0.017 0.0010 <0.0005 <0.0005 November 0.16 0.02 −0.03 0.50 0.51 0.36 0.909* 0.022 <0.0005 0.0003 0.0005 <0.0005 December −0.01 0.07 −0.27 0.31 0.49 0.35 0.0005 <0.0005 <0.0005 <0.0005 0.0003 <0.0005
i.e., it did not perform well. However, the ratio of stand-ard deviations indicates greater accuracy of MSE than PCR. Regionally, higher accuracy of MSE is captured in the AMZ region, with values ranging from 0.45 to 0.98, followed by LPB with values from 0.32 to 0.74 and NEB with values between 0.23 and 0.70 (Table 6). One possi-ble explanation is that MSE is calculated by mean squared errors of the combined simulations.
4 Discussion
From the regression model via principal components (PC), the LPB region had the highest percentage of the total variance when compared with the other regions during the past climate, a situation that is associated with the greater variance (percentage) each PC represents in the total vari-ance. A possible explanation is the higher numbers of
atmospheric systems (cold fronts, cyclones, mesoscale con-vective complexes, etc.) due to the greater variability in this region. When comparing the past with the future climate, it was possible to observe a substantial increased and/or reduction in the explanatory variance for future climate in different months and regions, indicating that the increase and decrease of variance in future climate may be the explanation for the occurrence of extreme moist and/or dry events in different regions.
In comparing the three combination techniques, PCR was more reliable in representing the precipitation in differ-ent regions, because besides showing values nearer to the observed ones, the technique showed moderate to strong correlation at 5 % significance in almost every month in the different regions for past climate. This occurred because the in PCR the data are first subjected to PCA and the scores are used to perform the prediction, thereby making the technique more robust, very close to traditional multilinear
Fig. 9 Average values of RMSE for the AMZ combining the multiple regression by principal components (dashed curve. indicated by PCR) and convex combination (solid curve, indicated by MSE) for the months from January to December from 1961 to 1990 (units in mm/month)
regression (Sancevero et al. 2008), since this method can eliminate the effects of bias in the forecasts (Jeong and Kim 2009).
In relation to the performance of combination tech-niques, the average RMSE values showed improvement of PCR for regions with low or medium average tion of climate. In areas considered to have high representa-tion of climate, these small improvements may not be as satisfactory as in the areas of low or medium representation of climate. According to Misra (2006), the rain forecast between February and April in NEB depends on the SST anomaly over the Tropical Atlantic. The largest forecast-ing failures occur when SST anomalies over the Tropical North Atlantic are large and correlated with the anomaly patterns over the Eastern Pacific Ocean. SST anomalies over the Tropical Atlantic directly influence the position of the Intertropical Convergence Zone (ITCZ), which plays a role in precipitation patterns over the North and Northeast of Brazil (Goddard et al. 2001).
The statistical indices evaluated in the corrected fore-casts showed that the two combinations presented inverse behavior. PCR presented higher rates of agreement, with a correlation coefficient ranging from moderate to strong at 95 % confidence. However, the standard deviation ratios were relatively low. The inverse happened in MSE. For this reason, errors should be discriminated. Willmott (1982) mentioned that between measurements and simulations, random errors decrease the precision and systematic errors decrease the accuracy. For validation, according to Will-mott (1982) the RMSE is the best measure of models’ per-formance because it better analyzes the accuracy of mod-els. In this study, PCR was the best combination because it produced lower dispersions (errors) and therefore higher quality estimates.
Combinations of simulations have been performed by several authors (Christensen et al. 2010; Rozante et al. 2014; Da Silva and Silva 2014), who have emphasized that statistical techniques that combine several different
Fig. 11 As in Fig. 9, but for the LPB region
Table 6 Validation of combinations from the following statistical indices: Wilmott’s index of agreement (adim) and the ratio of standard deviations (adim) for past climate
Willmott concordance index (d) σMSE
σGPCC(adim)
σPCR
σGPCC(adim)
MSE × GPCC PCR × GPCC
AMZ NEB LPB AMZ NEB LPB AMZ NEB LPB AMZ NEB LPB
Past climate January 0.42 0.27 0.35 0.76 0.68 0.78 0.53 0.52 0.38 0.64 0.55 0.68 February 0.17 0.33 0.52 0.73 0.53 0.67 0.79 0.61 0.74 0.62 0.40 0.54 March 0.48 0.32 0.40 0.54 0.52 0.54 0.71 0.60 0.58 0.43 0.41 0.51 April 0.46 0.23 0.37 0.40 0.54 0.48 0.56 0.45 0.42 0.30 0.43 0.39 May 0.41 0.27 0.39 0.36 0.52 0.64 0.45 0.55 0.44 0.30 0.39 0.52 June 0.35 0.41 0.31 0.65 0.41 0.60 0.71 0.65 0.47 0.54 0.33 0.49 July 0.36 0.33 0.39 0.66 0.47 0.35 0.47 0.23 0.44 0.53 0.36 0.27 August 0.53 0.42 0.32 0.59 0.63 0.40 0.77 0.70 0.59 0.46 0.51 0.29 September 0.53 0.11 0.42 0.68 0.59 0.56 0.67 0.64 0.66 0.55 0.48 0.43 October 0.41 0.49 0.51 0.65 0.52 0.49 0.86 0.60 0.63 0.53 0.40 0.38 November 0.42 0.39 0.32 0.61 0.61 0.47 0.98 0.65 0.32 0.50 0.51 0.36 December 0.36 0.39 0.31 0.39 0.60 0.45 0.51 0.45 0.37 0.31 0.49 0.35
predictions provide significant improvements. Christensen et al. (2010) chose different metrics (aspects to reproduce large-scale circulation patterns, daily temperature distribu-tions and precipitation) to generate weights for RCM. The results showed that not all model quality aspects were cap-tured by weighting and the authors concluded that the use of model weights was sensitive to the particular process and showed different accuracies for the metrics chosen. Rozante et al. (2014) removed the biases in the MSE tech-nique to improve the prediction quality. They verified that the method applied was able to capture the rainfall patterns but tended to underestimate the heavy rainfall events and increase the areas with low rainfall.
Similar to this study, Da Silva and Silva (2014) com-pared two model combination techniques: the arithmetic mean and PCR. They found that PCR provided a more realistic forecast in terms of average daily rainfall. Com-paring the results of our PCR with that study, the variance (percentage) that each PC represents in total variance of the study of Da Silva and Silva was substantially higher in both regions (NEB and AMZ) compared to our study.
5 Conclusion
The aim of this study was to apply two robust combination techniques to the precipitation variable, simulated from six regional climate models (RCM) in three South Ameri-can regions: Amazon, Northeast Brazil and La Plata Basin, for the past (1961–1990) and future climate (2071–2100). The method applied in this study has rarely been applied to climate prediction, although many studies have quantified RCM sets with simple arithmetic averages.
A significant advantage of the combination methods was the ability to capture extreme events (outliers) for the study regions. In general, we observed that MSE captures more extreme wet events while PCR captures more extreme dry events in the three regions. The knowledge of these extreme events is of great importance, since the prediction of these events serves as an alert to civil defense agencies to take preventive measures regarding urban infrastructure, con-struction, transport and health.
In comparing the two combination techniques, PCR pro-duced a more accurate prediction than MSE in all regions and climates. Also, MSE in general, underestimated rainfall in the past climate and indicated drier regions in future cli-mate compared to the past clicli-mate.
The method used here can be applied with any number of simulations. However, further studies are still neces-sary, because mistakes and uncertainties in climate pre-diction will always exist due to various sources of errors present in the simulations, such as the initial conditions of models.
Acknowledgments We would like to thank the CLARIS-LPB project for providing the outputs of the models and also the Office to Improve University Personnel (CAPES) for funding.
References
Chakraborty A, Krishnamurti TN, Gnanaseelan C (2007) Prediction of the diurnal cycle using a multimodel superensemble. Part II: Clouds. Mon Weather Rev 135:4097–4116
Christensen JH, Kjellström E, Giorgi F, Lenderink G, Rummukainen M (2010) Weight assignment in regional climate models. Clim Res 44(2-3):179–194
Chou SC, Lan C-W (2012) Changes in the annual range of precipita-tion under global warming. J Clim 25(1):222–235
Coppola E, Giorgi F, Raffaele F, Fuentes-Franco R, Giuliani G, LLo-part-Pereira M, Torma C (2014) Present and future climatologies in the phase I CREMA experiment. Clim Change 125(1):23–38 Coutinho MM (1999) Previsão por conjuntos utilizando perturbações
baseadas em componentes principais. São José dos Campos. Dis-sertação (Mestrado em Meteorologia) - Instituto Nacional de Pesquisas Espaciais, p 136
Coutinho EDC, Fisch G (2007) Distúrbios ondulatórios de leste (DOLs) na região do Centro de Lançamento de Alcântara-MA. Revis Bras de Meteorol 22(2):193–203
Coutinho MDL, Lima KC, Silva CMS (2016) Regional climate simu-lations of the changes in the components of the moisture budget over South America. Int J Climatol 36(3):1170–1183
Domínguez M, Gaertner MA, De Rosnay P, Losada T (2010) A regional climate model simulation over West Africa:parameterization tests and analysis of land-surface fields. Clim Dyn 35(1):249–265
Da Rocha RP, Morales CA, Cuadra SV, Ambrizzi T (2009) Precipi-tation diurnal cycle and summer climatology assessment over South America: an evaluation of regional climate model ver-sion 3 simulations. J Geophys Res: Atmos. doi:10.1029/200 8JD010212
Da Rocha RP, Cuadra SV, Reboita MS, Krüger LF, Ambrizzi T, Krusche N (2012) Effects of RegCM3 parameterizations on sim-ulated rainy season over South America. Clim Res 52:253–265 Da Silva AG, Silva CMS (2014) Improving regional dynamic
down-scaling with multiple kinear regression model using components principal analysis: precipitation over Amazon and Northeast Bra-zil. Adv Meteorol 2014
Ferraz SET, Pedroso D (2013) Sensitivity of REGCM3 simulated precipitation over southern Brazil with different boundary condi-tions: ENSO case. Adv Meteorol 2013
Goddard L, Mason SJ, Zebiak SE, Ropelewski CF, Basher R, Cane MA (2001) Current approaches to seasonal-tointerannual climate predictions. Int J Climatol 21:1111–1152
Hopkins WG (2009) Correlation coefficient. Disponível em: http:// www.sportsci.org/resource/stats/correl.html. Acesso em: 20 fev. 2014
Jeong DI, Kim Y (2009) Combining single-value streamflow fore-casts—a review and guidelines for selecting techniques. J Hydrol 377(3):284–299
Kendall MG (1957) A course in multivariate analysis. Griffin, London Knutti R, Furrer R, Tebaldi C, Cermak J, Meehl GA (2010) Chal-lenges in combining projections from multiple climate models. J Clim 23:2739–2758. doi:10.1175/2009JCLI3361.1
Krishnamurti TN, Kishtawal CM, Larow TE, Bachiochi DR, Zhang Z (1999) Improved weather and seasonal climate forecasts from multi-model superensemble. Science 285(5433):1548–1550 Krishnamurti TN, Kishtawal CM, Zhang Z, LaRow T, Bachiochi DR,
forecasts for weather and seasonal climate. J Clim 13:4196–4216. doi:10.1175/1520-0442(2000)013<4196:MEFFWA>2.0.CO;2
Krishnamurti TN, Kishtawal CM, Zhang Z, LaRow TE, Bachiochi DR, Williford CE, Gadgil S, Surendran S (2000b) Improving tropical precipitation forecasts from a multianalysis superensem-ble. J Clim 13:4217–4227
Krishnamurti TN, Sanjay J, Mitra AK, Vijaya Kumar TSV (2004) Determination of forecast errors arising from different com-ponents of model physics and dynamics. Mon Weather Rev 132(11):2570–2594
Krishnamurti TN, Mishra AK, Chakraborty A, Rajeevan M (2009) Improving global model precipitation forecasts over India using downscaling and the FSU superensemble, Part I: 1–5-Day fore-casts. Mon Weather Rev 137:2713–2735
Kumar S, Tamura K, Jakobsen IB, Nei M (2001) MEGA2: molecu-lar evolutionary genetics analysis software. Bioinformatics 17(12):1244–1245
Lee JA, Kolczynski WC, Mccandless TC, Haupt SE (2012) An objec-tive methodology for configuring and down-selecting an nwp ensemble for low-level wind prediction. Mon Weather Rev 140(7):2270–2286
Lenartz F, Mourre B, Barth A, Beckers J-M, Vandenbulcke L, Rixen M (2010) Enhanced ocean temperature forecast skills through 3-D super-ensemble multi-model fusion. Geophys Res Lett 37(19). doi:10.1029/2010GL044591
Leung LR, Qian Y, Bian X, Washington WM, Han J, Roads JO (2004) Mid-century ensemble regional climate change scenarios for the western United States. Clim Change 62(1–3):75–113
Li L, Conil S (2003) Transient response to an atmospheric GCM to North Atlantic SST anomalies. J Clim 16:3993–3998
Majewski D (1991) The Europa-Modell of the Deutscher Wetterdi-enst. In: Seminar proceedings ECMWF, vol 2. pp 147–191 Marengo JA, Ambrizzi T, da Rocha RP, Alves LM, Cuadra SV,
Val-verde MC, Ferraz SET, Torres RR, Santos DC (2010) Future change of climate in South America in the late twenty-first cen-tury: intercomparison of scenarios from three regional climate models. Clim Dyn 35(6):1073–1097
Martens H (1992) Multivariate calibration. Wiley, New York
Meehl GA, Stocker TF, Collins WD, Friedlingstein P, Gaye AT, Gregory JM, Zhao ZC (2007) Global climate projections. Clim Change 3495:747–845
Menéndez CG, De Castro M, Boulanger JP, D’Onofrio A, Sanchez E, Sörensson AA, Teichmann C (2010) Downscaling extreme month-long anomalies in southern South America. Clim Change 98(3–4):379–403
Misra V (2006) Understanding the predictability of seasonal precipi-tation over northeast Brazil. Tellus 58a:307–319
Mota GV (1997) Estudo observacional de distúrbios ondulatórios de leste no nordeste brasileiro. São Paulo. 92 p. Dissertação (Mestrado em Meteorologia) – Instituto Astronômico e Geofísico–USP Moura AD, Shukla J (1981) On the dynamics of droughts in northeast
Brazil: observations, theory and numerical experiments with a general circulation model. J Atmos Sci 38(7):2653–2675 Mullen S, Buizza R (2001) Quantitative precipitation forecasts over
the United States by the ECMWF ensemble prediction system. Mon Weather Rev 129(4):638–663
Paiva EJ, Paiva AP, Ferreira JR, Balestrassi PP (2008) Otimização de múltiplas respostas baseada no Erro Quadrático Médio Multivar-iado. In: XXVIII Encontro Nacional de Engenharia de Produção. Rio de Janeiro
Pal et al (2007) Regional climate modeling for the developing world: The ICTP RegCM3 and RegCNET. Bull Am Meteorol Soc 88:1395–1409
Pesquero JF, Chou SC, Nobre CA, Marengo JA (2010) Climate down-scaling over South America for 1961–1970 using the Eta Model. Theoret Appl Climatol 99(1–2):75–93
Phillips DL, Dolph J, Marks D (1992) A comparison of geostatistical procedures for spatial analysis of precipitations in mountainous terrain. Agric For Meteorol 58(1):119–141
Richardson DS (2001) Measures of skill and value of ensemble pre-diction systems, theirinter relationship and the effect of ensem-ble size. Q J R Meteorol Soc 127:2473–2489. doi:10.1002/ qj.49712757715
Robertson AW, Lall U, Zebiak SE, Goddard L (2004) Improved com-bination of multiple atmospheric GCM ensembles for seasonal predition. Mon Weather Rev 132(12):2732–2744
Roggo Y, Chalus P, Maurer L, Lema-Martinez C, Edmond A, Jent N (2007) A review of near infrared spectroscopy and chemo-metrics in pharmaceutical technologies. J Pharm Biomed Anal 44(3):683–700
Rozante JR, Moreira DR, Godoy RCM, Fernandes AA (2014) Multi-model ensemble: technique and validation. Geosci Model Dev Discuss 7(3):2933–2959
Sadourny R, Le Van P, Hourdin F (1995) Discrétisation des equations de la dynamique dans le modèle LMDZ. Internal report, Labora-toire de Météorologie Dynamique du CNRS, Paris, p 4
Salathé EP, Steed R, Mass CF, Zahn P (2008) A high-resolution climate model for the U.S. Pacific Northwest: mesoscale feedbacks and local responses to climate change. J Clim 21(21):5708–5726
Samuelsson P, Kourzeneva E, Mironov D (2010) The impact of lakes on the European climate as simulated by a regional climate model. Boreal Environ Res 15(2):113–129
Samuelsson P, Jones C, Willén U, Ullerstig A, Gollvik S, Hansson U, Jansson C, Kjellström E, Nikulin G, Wyser K (2011) The Rossby centre regional climate model RCA3: model description and per-formance. Tellus 63(1):4–23
Sancevero SS, Remacre AZ, Vidal AC, Portugal RS (2008) Apli-cação de técnicas de estatística multivariada na definição da lito-logia a partir de perfis geofísicos de poços. Revis Bras Geosci 38(1):61–74
Sánchez E, Gaertner MA, Gallardo C, Padorno E, Arribas A, De Cas-tro M (2007) Impacts of a change in vegetation description on simulated European summer present-day and future climates. Clim dyn 29(2–3):319–332
Sánchez E, Solman S, Remedio ARC, Berbery H, Samuelsson P, Da Rocha RP, Jacob D (2015) Regional climate modelling in CLARIS-LPB: a concerted approach towards twentyfirst century projections of regional temperature and precipitation over South America. Clim Dyn 45(7–8):2193–2212
Schneider U, Fuchs T, Meyer-Christoffer A, Rudolf B (2008) Global precipitation analysis products of the GPCC. Global Precipitation Climatology Centre (GPCC), DWD, Internet Publikation, pp 1–12 Solman SA, Sánchez E, Samuelsson P, Da Rocha RP, Li L, Marengo
J, Jacob D (2013) Evaluation of an ensemble of regional climate model simulations over South America driven by the ERA-Interim reanalysis: model performance and uncertainties. Clim Dyn 41(5–6):1139–1157
Sörensson AA, Menéndez CG (2011) Summer soil-precipitation cou-pling in South America. Tellus 63(1):56–68
Stocker T, Qin D, Plattner GK, Tignor M, Midgley P (2010) Meeting report of the IPCC expert meeting on assessing and combining multi model climate projections. IPCC Working Group I Techni-cal Support Unit, Berlin
Sun L, Li H, Zebiak SE, Moncunill DF, Filho FD, Moura AD (2006) An operational dynamical downscaling prediction system for Nordeste Brazil and the 2002–04 real-time forecast evaluation. J Clim 19(10):1990–2007
Tebaldi C, Smith RW, Nychka D, Mearns LO (2005) Quantifying uncertainty in projections of regional climate change: a bayes-ian approach to the analysis of multimodel ensembles. J Clim 18(10):1524–1540
Van Lier Walqui M, Vukicevic T, Posselt DJ (2012) Quantification of cloud microphysical parameterization uncertainty using radar reflectivity. Mon Weather Rev 140(11):3442–3466
Wang C (2002) Atlantic climate variability and its associated atmos-pheric circulation cells. J Clim 15:1516–1536
Wilks DS (2006) Statistical methods in the atmospheric sciences, 2nd edn. Academic Press, San Diego
Willmott CJ (1982) Some comments on the evaluation of model per-formance. Bull Am Meteorol Soc 63(11):1309–1313
Zhu H, Thorpe A (2006) The predictability of extra-tropical cyclones: the influence of the initial condition and model uncertainties. J Atmos Sci 63(5):1483–1497
