Error Decomposition

114

ESTIMATION ET DÉCOMPOSITION DE L’INCERTITUDE À L’^{AIDE D}’UN ENSEMBLEM^ONTEC^ARLO

ET D’UN ENSEMBLE MULTI-MODÈLES

Table 4.4: Regression coefficients (averaged, inµgm⁻³) for the three different station types. Out- put data (regressand) is the temporal average of ozone peaks from multimodel ensemble. Only the average coefficients that are greater than twice their average standard deviation are reported.

In brackets, we show the spread among the stations of the coefficients, computed as the empirical standard deviation of all regression coefficients – and not as the mean of the deviationsσβk.

Field Name Background Rural Urban

O3 Boundary Conditions 42 (4.6) 45 (5) 43.8 (5)

NOx Emissions 21 (7) 13.2 (9.8) 15.3 (10.9)

Photolysis 17.9 (5.8) 17.8 (4.8) 18.8 (6.5) Biogenic emissions 7.8 (3.) 7.8 (2.2) 8.3 (3.2) Wind module -14 (3.5) -14.4 (5.5) -14 (6) Deposition -14 (4.4) -12.8 (3.6) -11.8 (4.2) NOx Boundary Conditions 3 (0.8) – 2.7 (0.9)

order to provide an ensemble of boundary conditions. The description of the uncertainty in the boundary conditions would then be much more accurate than the perturbation scheme we use in this work.

Now that we have analyzed the main sources of uncertainties due to the input fields and parameters, we investigate in the next section how the discrepancies between the observations and the simulations can be decomposed and what part is due to the shortcomings of the modeling.

Section 4.4 – Error Decomposition 115

Our objective is to estimate the variance of each error. First, we assume that the errorse_o,e_r ande_mhave zero mean and that they are mutually uncorrelated, e.g., E[H(X^t−X)(Y−Y^t)^T]=0.

IfH_iis theith row ofH, the covariance between two error componentsiand jofeis Cov(e)i j = E([(Yi−Y_i^t)+(Y_i^t−H_iX^t)+(HiX^t−H_iX)]

[(Yj−Y_j^t)+(Y_j^t−H_jX^t)+(HjX^t−H_jX)]^T)

= E[(Yi−Y_i^t)(Yj−Y_j^t)^T]

| {z }

measurement error variance + E[(Y_i^t−H_iX^t)(Y_j^t−H_jX^t)^T]

| {z }

representativeness error variance +H_i E[(X^t−X)(X^t−X)^T]

| {z }

modeling error variance

H^T_j (4.9)

4.4.1 Measure Error

In this study, we use ground stations from the European Airbase network. Ozone is measured by spectrometry, using its absorption in ultraviolet. A sample of ambient air is taken. A beam at wavelength 254 nm is emitted. Ozone molecules absorb a part of the radiation. A sensor turns the measured radiation into an electrical signal which is proportional with the sampling ozone con- centration. Airparif, the organization responsible for monitoring air quality in the Paris region, produces upper bounds on the measurement uncertainties, along with the measurements them- selves [Airparif,2007]. The uncertainty takes into account many error sources from the different stages of the measurement chain: air sampling, data capture, electronic device, calibration, . . .

Hourly ozone measurements and their uncertainty upper bounds, provided by Airparif for year 2009 and 30 monitoring stations, are clustered in a concentration intervals from [0,20]µgm⁻³ to [80,∞[µgm⁻³. The uncertainty and the relative uncertainty (i.e., the uncertainty divided by the concentration) are reported in table 4.5 and in figure 4.8. The uncertainty corresponds to the 95% confidence interval, so that, in case of Gaussian errors, it is equal to twice the stan- dard deviation. The uncertainty increases with the concentration, while the relative uncertainty decreases. The relative uncertainty can be higher than 50% when the measured ozone concen- tration is about 8µgm⁻³. For high concentrations, the relative uncertainty ranges from∼12% to

∼9% for ozone concentrations between∼80 and∼150µgm⁻³.

Table 4.5: For five concentration intervals, the table shows the average (µgm⁻³) of all measure- ments in the interval, the corresponding average of the uncertainty (µgm⁻³) and the correspond- ing relative uncertainty. The uncertainty corresponds to a 95% confidence interval; so if the error is Gaussian, it is twice the standard deviation.

Range (µgm⁻³) Av. Measure Uncertainty Rel. Unc.

0 – 20 8.8 6.8 0.77

20 – 40 30.4 7.4 0.24

40 – 60 49.7 8.1 0.16

60 – 80 68.5 8.9 0.13

≥80 97.9 10.4 0.11

116

ESTIMATION ET DÉCOMPOSITION DE L’INCERTITUDE À L’^{AIDE D}’UN ENSEMBLEM^ONTEC^ARLO

ET D’UN ENSEMBLE MULTI-MODÈLES

0 20 40 60 80 100 120 140 Measure

0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6

Relative Uncertainty

Figure 4.8: Relative uncertainty according to the observed ozone concentration inµgm⁻³. 4.4.2 Modeling and Representativeness Errors

In this section, we carry out two independent methods to estimate the variance of the repre- sentativeness error. We consider all ozone hourly observations (not only the ozone peaks) and, in the second method, the multimodel-ensemble mean.

The first method is solely based on the observations. We assume that the mean concentration in a grid cell can be approximated by the mean of the observed concentrations. This assumption is reasonable only if there are enough observation stations in one model grid cell, and if they are spread all over the grid cell. In grid cellk, we denoteJ_kthe set of the indexes of stations that are inside the grid cell. We approximate the mean concentration in the grid cell by

A_k= 1

|J_k| X

j∈J_k

Y_j, (4.10)

where|J_k|is the number of stations inside the grid cellk.

We select six grid cells that contain between 8 and 10 Airbase stations: two cells close to Marseille, one close to Paris, Barcelona, Valencia and London. All selected stations are urban, since in rural regions, the monitoring network is not dense enough to have so many stations in one grid cell.

We first compute A_k−Y_j for the eight grid cells of interest and for all corresponding station j∈J_k, which amounts to 300,000 discrepancies. This quantity measures how much observations can deviate from the approximate average in the cell. Figure 4.9shows the relative occurrence frequency ofA_k−Y_j. By definition ofA_k, the mean of the distribution is zero. The empirical stan- dard deviation, which provides an estimate of the standard deviation of the representativeness error, is equal to 12.9µgm⁻³.

The second method is the sometimes referred to as the observational method or the Hollingsworth- Lönnberg method [Hollingsworth et Lönnberg, 1986]. The variance of the modeling error and the sum of measurement and representativeness variances are estimated based on a variogram of the discrepancies Y−H X. The variogram plots the empirical covariance between all pairs (Yi−H_iX,Y_j−H_jX) against the distance between the locationsiand j. The first bar of the dia- gram, which corresponds to variances (because the distance is zero), is due to all three errorse_o, e_rande_m. If the observational errors are assumed to be uncorrelated, the height of the next bars is only due to the modeling errore_m. If one extrapolates from these bars to the origin, the differ- ence between the ordinate at the origin and the height of the first bar is due to the observational error (eo+e_r) — see figure4.10for an illustration.

116

Section 4.4 – Error Decomposition 117

60 40 20 0 20 40 60 Concentration

0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5

Probability

1e 2

Figure 4.9: Approximate relative frequency occurrence of the representativeness error. The em- pirical standard deviation is 12.9µgm⁻³.

0 10 20 30 40 50

Distance 0

50 100 150 200 250 300 350

Covariance

o2

+

Modeling error covariance

Figure 4.10: Illustration of the method byHollingsworth et Lönnberg[1986] to estimate observa- tional error variance with a variogram. This figure is inspired byBouttier et Courtier[1999].

118

ESTIMATION ET DÉCOMPOSITION DE L’INCERTITUDE À L’^{AIDE D}’UN ENSEMBLEM^ONTEC^ARLO

ET D’UN ENSEMBLE MULTI-MODÈLES

In our case, we consider all pairs (i,j) of urban stations, and all times at which observations are available. The error is computed using the mean of the calibrated multimodel sub-ensemble.

In total,∼190,000 covariances are computed. Figure4.11shows all covariances that decrease with the distance as the errors become uncorrelated.

Figure 4.11: Variogram of the error between observed and simulated ozone concentration. The covariance is inµg²m⁻⁶and the distance in degrees (latitude/longitude).

We collect all covariance values from points within a distance in ]0,0.5^◦]. The mean of these covariances is equal to σ²_m≃336µg²m⁻⁶. The mean of the variance (computed with all pairs with null distance) is σ² =489µg²m⁻⁶. Hence σ²_o+^σ2r ≃^σ2−^σ2m≃153µg²m⁻⁶. The mean of observed concentrations is about 43µgm⁻³. According to section4.4.1, it means that the variance of measurement errors is about 16µg²m⁻⁶. Consequently, the variance of the representativeness error can be estimated by 137µg²m⁻⁶, hence a standard deviation at 11.7µgm⁻³ which is a bit less than the estimation from the first method (12.9µgm⁻³).

Observational error should be independent of the model, provided the same observation op- erator is used, which is the case in our ensemble where all models have the same horizontal resolution. We carry out the method with seven models randomly selected from the multimodel ensemble. The estimated variance of the observational error varies between 146µg²m⁻⁶ and 156µg²m⁻⁶. The standard deviation of the representativeness error is then estimated between 11.7 and 12.5µgm⁻³.

Note that the measurement and representativeness errors are not entirely uncorrelated be- tween two points at close distance. For instance, the measurement errors can depend on the at- mospheric conditions, which are obviously correlated at short distance. Therefore the estimation of the modeling error variance is overestimated, and the representativeness errors is underesti- mated. According to this method, the standard deviation of the representativeness error is likely to be greater than 12.5µgm⁻³, which is consistent with the first method giving 12.9µgm⁻³.

The variance of the discrepancies between hourly ground-level ozone observations and the mean of the calibrated sub-ensemble is 489µg²m⁻⁶. Following (4.9), it can be decomposed in less than∼3.2% for measurement errors (σ²_o.16µg²m⁻⁶), in about 34% for representativeness error (σ²_r≃166µg²m⁻⁶) and in about 63% for modeling error (σ²_m≃489−16−166=307µg²m⁻⁶).