Accurately forecasting ozone levels that exceed specific thresholds is pivotal for mitigating adverse effects on both the environment and public health. However, predicting such ozone exceedances remains challenging due to the infrequent occurrence of high-concentration ozone data. This research, leveraging data from 57 German monitoring stations from 1999 to 2018, introduces an Enhanced Extreme Instance Augmentation Random Forest (EEIA-RF) approach that significantly improves the prediction of days when the maximum daily 8-hour average ozone concentrations exceed 120μg/m³. A pre-trained machine learning model is used to generate additional high-concentration data, which, combined with selectively reduced low-concentration data, forms a new dataset for training a refined model. This method achieved an improvement of at least 8% in the accuracy of predicting days with ozone exceedances across Germany. Our experiment underscores the approach's value in enhancing atmospheric modeling and supporting public health advisories and environmental policy-making related to ozone pollution.

@en

Dust storms pose significant risks to health and property, necessitating accurate forecasting for preventive measures. Despite advancements, dust models grapple with uncertainties arising from emission and transport processes. Data assimilation addresses these by integrating observations to rectify model error, enhancing forecast precision. The ensemble Kalman filter (EnKF) is a widely used assimilation algorithm that effectively optimize model states, particularly in terms of intensity adjustment. However, the EnKF's efficacy is challenged by position errors between modeled and observed dust features, especially under substantial position errors. This study introduces the valid time shifting ensemble Kalman filter (VTS-EnKF), which combines stochastic EnKF with a valid time shifting mechanism. By recruiting additional ensemble members from neighboring valid times, this method not only accommodates variations in dust load but also explicitly accounts for positional uncertainties. Consequently, the enlarged ensemble better represents both the intensity and positional errors, thereby optimizing the utilization of observational data. The proposed VTS-EnKF was evaluated against two severe dust storm cases from spring 2021, demonstrating that position errors notably deteriorated forecast performance in terms of root mean square error (RMSE) and normalized mean bias (NMB), impeding the EnKF's effective assimilation. Conversely, the VTS-EnKF improved both the analysis and forecast accuracy compared to the conventional EnKF. Additionally, to provide a more rigorous assessment of its performance, experiments were conducted using fewer ensemble members and different time intervals.@en

Measurements collected from ground monitoring stations have gained popularity as a valuable data source for evaluating numerical models and correcting model errors through data assimilation. The penalty quantified by simulation minus observations drives both model evaluation and assimilation. However, the penal forces are challenged by the existence of a spatial-scale disparity between model simulations and observations. Chemical transport models (CTMs) divide the atmosphere into grid cells, providing a structured way to simulate atmospheric processes. However, their spatial resolution often does not match the limited coverage of in situ measurements, especially for short-lived air pollutants. Within a broad grid cell, air pollutant concentrations can exhibit significant heterogeneity due to their rapid generation and dissipation. Ground observations with traditional methods (including “nearest search” and “grid mean”) are less representative when compared to model simulations. This study develops a new land-use-based representative (LUBR) observational operator to generate spatially representative gridded observations for model evaluation. It incorporates high-resolution urban–rural land use data to address intra-grid variability. The LUBR operator has been validated to consistently provide insights that align with satellite Ozone Monitoring Instrument (OMI) measurements. It is an effective solution to accurately quantify these spatial-scale mismatches and further resolve them via assimilation. Model evaluations with 2015–2017 NO2 measurements in the study area demonstrate that biases and errors differed substantially when the LUBR and other operators were used, respectively. The results highlight the importance of considering fine-scale urban–rural differences when comparing models and observations, especially for short-lived pollutants like NO2.@en

Meteorological fields calculated by numerical weather prediction (NWP) models drive offline chemical transport models (CTMs) to solve the transport, chemical reactions, and atmospheric interaction over the geographical domain of interest. HARMONIE (HIRLAM ALADIN Research on Mesoscale Operational NWP in Euromed) is a state-of-The-Art non-hydrostatic NWP community model used at several European weather agencies to forecast weather at the local and/or regional scale. In this work, the HARMONIE WINS50 (cycle 43 cy43) reanalysis dataset at a resolution of 0.025°ĝ€¯×ĝ€¯0.025° covering an area surrounding the North Sea for the years 2019-2021 was coupled offline to the LOTOS-EUROS (LOng-Term Ozone Simulation-EURopean Operational Smog model, v2.2.002) CTM. The impact of using either meteorological fields from HARMONIE or from ECMWF on LOTOS-EUROS simulations of NO2 has been evaluated against ground-level observations and TROPOMI tropospheric NO2 vertical columns. Furthermore, the difference between crucial meteorological input parameters such as the boundary layer height and the vertical diffusion coefficient between the hydrostatic ECMWF and non-hydrostatic HARMONIE data has been studied, and the vertical profiles of temperature, humidity, and wind are evaluated against meteorological observations at Cabauw in The Netherlands. The results of these first evaluations of the LOTOS-EUROS model performance in both configurations are used to investigate current uncertainties in air quality forecasting in relation to driving meteorological parameters and to assess the potential for improvements in forecasting pollution episodes at high resolutions based on the HARMONIE NWP model.

@en

Statistical methods, particularly machine learning models, have gained significant popularity in air quality predictions. These prediction models are commonly trained using the historical measurement datasets independently collected at the environmental monitoring stations and their operational forecasts in advance using inputs of the real-time ambient pollutant observations. Therefore, these high-quality machine learning models only provide site-available predictions and cannot solely be used as the operational forecast. In contrast, deterministic chemical transport models (CTMs), which simulate the full life cycles of air pollutants, provide predictions that are continuous in the 3D field. Despite their benefits, CTM predictions are typically biased, particularly on a fine scale, owing to the complex error sources due to the emission, transport, and removal of pollutants. In this study, we proposed a fusion of site-available machine learning prediction, which is from our regional feature selection-based machine learning model (RFSML v1.0), and a CTM prediction. Compared to the normal pure machine learning model, the fusion system provides a gridded prediction with relatively high accuracy. The prediction fusion was conducted using the Bayesian-theory-based ensemble Kalman filter (EnKF). Background error covariance was an essential part in the assimilation process. Ensemble CTM predictions driven by the perturbed emission inventories were initially used for representing their spatial covariance statistics, which could resolve the main part of the CTM error. In addition, a covariance inflation algorithm was designed to amplify the ensemble perturbations to account for other model errors next to the uncertainty in emission inputs. Model evaluation tests were conducted based on independent measurements. Our EnKF-based prediction fusion presented superior performance compared to the pure CTM. Moreover, covariance inflation further enhanced the fused prediction, particularly in cases of severe underestimation.@en

The chemical transport model LOTOS-EUROS uses a volatility basis set (VBS) approach to represent the formation of secondary organic aerosol (SOA) in the atmosphere. Inclusion of the VBS approximately doubles the dimensionality of LOTOS-EUROS and slows computation of the advection operator by a factor of two. This complexity limits SOA representation in operational forecasts. We develop a mass-conserving dimensionality reduction method based on matrix factorization to find latent patterns in the VBS tracers that correspond to a smaller set of superspecies. Tracers are reversibly compressed to superspecies before transport, and the superspecies are subsequently decompressed to tracers for process-based SOA modeling. This physically interpretable data-driven method conserves the total concentration and phase of the tracers throughout the process. The superspecies approach is implemented in LOTOS-EUROS and found to accelerate the advection operator by a factor of 1.5–1.8. Concentrations remain numerically stable over model simulation times of 2 weeks, including simulations at higher spatial resolutions than the data-driven models were trained on. The reversible compression of VBS tracers enables detailed, process-based SOA representation in LOTOS-EUROS operational forecasts in a computationally efficient manner. Beyond this case study, the physically consistent data-driven approach developed in this work enforces conservation laws that are essential to other Earth system modeling applications, and generalizes to other processes where computational benefit can be gained from a two-way mapping between detailed process variables and their representation in a reduced-dimensional space.

@en

Chemical transport models (CTM) are crucial for simulating the distribution of air pollutants, such as particulate matter, and evaluating their impact on the environment and human health. However, these models rely heavily on accurate emission inventory and meteorological inputs, usually obtained from reanalyzed weather data, such as the European Centre for Medium-Range Weather Forecasts (ECMWF). These inputs do not accurately reflect the complex topography and micro-scale meteorology in tropical regions where air pollution can pose a severe public health threat. We propose coupling the LOTOS–EUROS CTM model and the weather research and forecasting (WRF) model to improve LOTOS–EUROS representation. Using WRF as a meteorological driver provides high-resolution inputs for accurate pollutant simulation. We compared LOTOS–EUROS results when WRF and ECMWF provided the meteorological inputs during low and high pollutant concentration periods. The findings indicate that the WRF–LOTOS–EUROS coupling offers a more precise representation of the meteorology and pollutant dispersion than the default input of ECMWF. The simulations also capture the spatio-temporal variability of pollutant concentration and emphasize the importance of accounting for micro-scale meteorology and topography in air pollution modelling.@en

Super dust storms re-occurred over East Asia in 2021 spring and casted great health damages and property losses. It is essential to achieve an accurate dust forecast to reduce the damage for early warning. The forecasting system fundamentally relies on a numerical model which can forecast the full evolution of dust storms. However, large uncertainties exist in model forecasts. Meanwhile, various near-real-time observations are available that contain valuable dust information. A dust storm forecasting system is here developed through coupling a chemical transport model, LOTOS-EUROS, and Localized EnKF (LEnKF) assimilation approach. The assimilations are carried out via an interface of our self-designed assimilation toolbox, PyFilter v1.0. Ground-based PM₁₀ measurements from air quality monitoring network are assimilated. Sequential assimilation tests are carried out over the 2021 spring super dust storms. The results show that the assimilation-based forecasting system produces a promising dust forecast than model-only forecast, and the improvements is also validated through comparing to the independent MODIS aerosol optical depth (AOD). Superior performance is obtained when LEnKF is implemented, as the localization helps EnKF in resolving the PM₁₀ measurements that have a large spatial variability with limited ensemble members. In addition, sensitivity experiments are conducted to exploit the distance-dependent localization for the LEnKF. Considering both cases, the optimal choice of the distance is tested to be around 500 km: the larger distance is less effective in removing the spurious correction, while the smaller one easily falls into the local optimum and the model would become divergent rapidly.

@en

Last spring, super dust storms reappeared in East Asia after being absent for one and a half decades. The event caused enormous losses in both Mongolia and China. Accurate simulation of such super sandstorms is valuable for the quantification of health damage, aviation risks, and profound impacts on the Earth system, but also to reveal the climatic driving force and the process of desertification. However, accurate simulation of dust life cycles is challenging, mainly due to imperfect knowledge of emissions. In this study, the emissions that lead to the 2021 spring dust storms are estimated through assimilation of MODIS AOD and ground-based PM10 concentration data simultaneously. With this, the dust concentrations during these super storms could be reproduced and validated with concentration observations. The multi-observation assimilation is also compared against emission inversion that assimilates AOD or PM10 concentration measurements alone, and the added values are analyzed. The emission inversion results reveal that wind-blown dust emissions originated from both China and Mongolia during spring 2021. Specifically, 19.9×106 and 37.5×106ĝ€¯t of particles were released in the Chinese and Mongolian Gobi, respectively, during these severe dust events. By source apportionment it was revealed that the Mongolian Gobi poses more severe threats to the densely populated regions of the Fenwei Plain (FWP) and the North China Plain (NCP) located in northern China than does the Chinese Gobi. It was estimated that 63ĝ€¯% of the dust deposited in FWP was due to transnational transport from Mongolia. For NCP, the long-distance transport dust from Mongolia contributes about 69ĝ€¯% to the dust deposition.

@en

This work proposes a robust and non-Gaussian version of the shrinkage-based knowledge-aided EnKF implementation called Ensemble Time Local H_∞ Filter Knowledge-Aided (EnTLHF-KA). The EnTLHF-KA requires a target covariance matrix to integrate previously obtained information and knowledge directly into the data assimilation (DA). The proposed method is based on the robust H_∞ filter and on its ensemble time-local version the EnTLHF, using an adaptive inflation factor depending on the shrinkage covariance estimated matrix. This implies a theoretical and solid background to construct robust filters from the well-known covariance inflation technique. The proposed technique is implemented in a synthetic assimilation experiment, and in an air quality application using the LOTOS-EUROS model over the Aburrá Valley to evaluate its potential for non-linear and non-Gaussian large systems. In the spatial distribution of the PM_2.5 concentrations along the valley, the method outperforms the well-known Local Ensemble Transform Kalman Filter (LETKF), and the non-robust knowledge-aided Ensemble Kalman filter (EnKF-KA). In contrast to the other simulations, the ability to issue warnings for high concentration events is also increased. Finally, the simulation using EnTLHF-KA has lower error values than using EnKF-KA, indicating the advantages of robust approaches in high uncertainty systems.@en

With the explosive growth of atmospheric data, machine learning models have achieved great success in air pollution forecasting because of their higher computational efficiency than the traditional chemical transport models. However, in previous studies, new prediction algorithms have only been tested at stations or in a small region; a large-scale air quality forecasting model remains lacking to date. Huge dimensionality also means that redundant input data may lead to increased complexity and therefore the over-fitting of machine learning models. Feature selection is a key topic in machine learning development, but it has not yet been explored in atmosphere-related applications. In this work, a regional feature selection-based machine learning (RFSML) system was developed, which is capable of predicting air quality in the short term with high accuracy at the national scale. Ensemble-Shapley additive global importance analysis is combined with the RFSML system to extract significant regional features and eliminate redundant variables at an affordable computational expense. The significance of the regional features is also explained physically. Compared with a standard machine learning system fed with relative features, the RFSML system driven by the selected key features results in superior interpretability, less training time, and more accurate predictions. This study also provides insights into the difference in interpretability among machine learning models (i.e., random forest, gradient boosting, and multi-layer perceptron models).

@en

In this paper, we propose an efficient and practical implementation of the ensemble Kalman filter via shrinkage covariance matrix estimation. Our filter implementation combines information brought by an ensemble of model realizations, and that based on our prior knowledge about the dynamical system of interest. We perform the combination of both sources of information via optimal shrinkage factors. The method exploits the rank-deficiency of ensemble covariance matrices to provide an efficient and practical implementation of the analysis step in EnKF based formulations. Localization and inflation aspects are discussed, as well. Experimental tests are performed to assess the accuracy of our proposed filter implementation by employing an Advection Diffusion Model and an Atmospheric General Circulation Model. The experimental results reveal that the use of our proposed filter implementation can mitigate the impact of sampling noise, and even more, it can avoid the impact of spurious correlations during assimilation steps.

@en

When calibrating simulations of dust clouds, both the intensity and the position are important. Intensity errors arise mainly from uncertain emission and sedimentation strengths, while position errors are attributed either to imperfect emission timing or to uncertainties in the transport. Though many studies have been conducted on the calibration or correction of dust simulations, most of these focus on intensity solely and leave the position errors mainly unchanged. In this paper, a grid-distorted data assimilation, which consists of an image-morphing method and an ensemble-based variational assimilation, is designed for realigning a simulated dust plume to correct the position error. This newly developed grid-distorted data assimilation has been applied to a dust storm event in May 2017 over East Asia. Results have been compared for three configurations: a traditional assimilation configuration that focuses solely on intensity correction, a grid-distorted data assimilation that focuses on position correction only and the hybrid assimilation that combines these two. For the evaluated case, the position misfit in the simulations is shown to be dominant in the results. The traditional emission inversion only slightly improves the dust simulation, while the grid-distorted data assimilation effectively improves the dust simulation and forecasting. The hybrid assimilation that corrects both position and intensity of the dust load provides the best initial condition for forecasting of dust concentrations.@en

Tropospheric ozone is a secondary pollutant which can affect human health and plant growth. In this paper, we investigated transferred convolutional neural network long short-term memory (TL-CNN-LSTM) model to predict ozone concentration. Hourly CNN-LSTM model is used to extract features and predict ozone for next hour, which is superior to commonly used models in previous studies. In the daily ozone prediction model, prediction over a large time-scale requires more data, however, only limited data are available, which causes the CNN-LSTM model to fail to accurately predict. Network-based transfer learning methods based on hourly models can obtain information from smaller temporal resolution. It can reduce prediction errors and shorten run time for model training. However, for extreme cases where the amount of data is severely insufficient, transfer learning based on smaller time scale cannot improve model prediction accuracy.

@en

In this work, we present the development of a 4D-Ensemble-Variational (4DEnVar) data assimilation technique to estimate NOx top-down emissions using the regional chemical transport model LOTOS-EUROS with the NO2 observations from the TROPOspheric Monitoring Instrument (TROPOMI). The assimilation was performed for a domain in the northwest of South America centered over Colombia, and includes regions in Panama, Venezuela and Ecuador. In the 4DEnVar approach, the implementation of the linearized and adjoint model are avoided by generating an ensemble of model simulations and by using this ensemble to approximate the nonlinear model and observation operator. Emission correction parameters’ locations were defined for positions where the model simulations showed significant discrepancies with the satellite observations. Using the 4DEnVar data assimilation method, optimal emission parameters for the LOTOS-EUROS model were estimated, allowing for corrections in areas where ground observations are unavailable and the region’s emission inventories do not correctly reflect the current emissions activities. The analyzed 4DEnVar concentrations were compared with the ground measurements of one local air quality monitoring network and the data retrieved by the satellite instrument Ozone Monitoring Instrument (OMI). The assimilation had a low impact on NO2 surface concentrations reducing the Mean Fractional Bias from 0.45 to 0.32, primordially enhancing the spatial and temporal variations in the simulated NO2 fields.@en

The use of low air quality networks has been increasing in recent years to study urban pollution dynamics. Here we show the evaluation of the operational Aburrá Valley’s low-cost network against the official monitoring network. The results show that the PM_2.5 low-cost measurements are very close to those observed by the official network. Additionally, the low-cost allows a higher spatial representation of the concentrations across the valley. We integrate low-cost observations with the chemical transport model Long Term Ozone Simulation-European Operational Smog (LOTOS-EUROS) using data assimilation. Two different configurations of the low-cost network were assimilated: using the whole low-cost network (255 sensors), and a high-quality selection using just the sensors with a correlation factor greater than 0.8 with respect to the official network (115 sensors). The official stations were also assimilated to compare the more dense low-cost network’s impact on the model performance. Both simulations assimilating the low-cost model outperform the model without assimilation and assimilating the official network. The capability to issue warnings for pollution events is also improved by assimilating the low-cost network with respect to the other simulations. Finally, the simulation using the high-quality configuration has lower error values than using the complete low-cost network, showing that it is essential to consider the quality and location and not just the total number of sensors. Our results suggest that with the current advance in low-cost sensors, it is possible to improve model performance with low-cost network data assimilation.

@en

Air quality warning and forecasting systems are usually based on numerical chemical transport models (CTMs). Those dynamic models perform predictions by simulating the life cycles of the atmospheric components, including emission, transport and removal. However, the accuracy of these CTMs are still limited because of many imperfections, e.g., uncertainties in the input sources such as emission inventories, wind fields, boundary conditions, as well as insufficient knowledge about the atmospheric dynamics themselves. All these will mislead the CTM prediction constantly, or in a systematic way. In this paper, an approach based on machine learning is applied to predict model bias in the CTM. It is then combined with the CTM for formulating a hybrid forecast system. To our knowledge, it is the first time that machine learning methods are used in this way. The hybrid system is tested on the fine particular matter (PM_2.5) prediction in Shanghai, China. The results showed that machine learning can be an effective tool to improve the accuracy of CTM prediction. In case of short term PM_2.5 forecast (forecast length less than 12 h), statistical metrics of the root mean square error, mean absolute error, mean absolute percentage error as well as the air quality rank predicted accuracy all show the forecast skill is remarkably improved; while for long term prediction, improvement is not ensured.

@en

Emission inversion using data assimilation fundamentally relies on having the correct assumptions about the emission background error covariance. A perfect covariance accounts for the uncertainty based on prior knowledge and is able to explain differences between model simulations and observations. In practice, emission uncertainties are constructed empirically; hence, a partially unrepresentative covariance is unavoidable. Concerning its complex parameterization, dust emissions are a typical example where the uncertainty could be induced from many underlying inputs, e.g., information on soil composition and moisture, land cover and erosive wind velocity, and these can hardly be taken into account together. This paper describes how an adjoint model can be used to detect errors in the emission uncertainty assumptions. This adjoint-based sensitivity method could serve as a supplement of a data assimilation inverse modeling system to trace back the error sources in case large observation-minus-simulation residues remain after assimilation based on empirical background covariance.

The method follows an application of a data assimilation emission inversion for an extreme severe dust storm over East Asia <span classCombining double low line"cit"idCombining double low line"xref_paren.1">(<a hrefCombining double low line"#bib1.bibx31">Jin et al.</a>, <a hrefCombining double low line"#bib1.bibx31">2019</a><a hrefCombining double low line"#bib1.bibx31">b</a>)</span>. The assimilation system successfully resolved observation-minus-simulation errors using satellite AOD observations in most of the dust-affected regions. However, a large underestimation of dust in Northeast China remained despite the fact that the assimilated measurements indicated severe dust plumes there. An adjoint implementation of our dust simulation model is then used to detect the most likely source region for these unresolved dust loads. The backward modeling points to the Horqin desert as the source region, which was indicated as a non-source region by the existing emission scheme. The reference emission and uncertainty are then reconstructed over the Horqin desert by assuming higher surface erodibility. After the emission reconstruction, the emission inversion is performed again, and the posterior dust simulations and reality are now in much closer harmony. Based on our results, it is advised that emission sources in dust transport models include the Horqin desert as a more active source region.

@en

A data assimilation system for the LOTOS-EUROS chemical transport model has been implemented to improve the simulation and forecast of PM₁₀ and PM_2.5 in a densely populated urban valley of the tropical Andes. The Aburrá Valley in Colombia was used as a case study, given data availability and current environmental issues related to population expansion. The data assimilation system is an Ensemble Kalman filter with covariance localization based on specification of uncertainties in the emissions. Observations assimilated were obtained from a surface network for the period March–April of 2016, a period of one of the worst air quality crisis in recent history of the region. In a first series of experiments, the spatial length scale of the covariance localization and the temporal length scale of the stochastic model for the emission uncertainty were calibrated to optimize the assimilation system. The calibrated system was then used in a series of assimilation experiments, where simulation of particulate matter concentrations was strongly improved during the assimilation period, which also improved the ability to accurately forecast PM₁₀ and PM_2.5 concentrations over a period of several days.

@en

Aerosol optical depths (AODs) from the new Himawari-8 satellite instrument have been assimilated in a dust simulation model over East Asia. This advanced geostationary instrument is capable of monitoring the East Asian dust storms which usually have great spatial and temporal variability. The quality of the data has been verified through a comparison with AErosol RObotic NETwork AODs. This study focuses on extreme dust events only when dust aerosols are dominant; promising results are obtained in AOD assimilation experiments during a case in May 2017. The dust emission fields that drive the simulation model are strongly improved by the inverse modeling, and consequently, the simulated dust concentrations are in better agreements with the observed AOD as well as ground-based observations of PM ₁₀ . However, some satellite AODs show significant inconsistence with the simulations and the PM ₁₀ and AErosol RObotic NETwork observations, which might arise from retrieval errors over a partially clouded scene. The data assimilation procedure therefore includes a screening method to exclude these observations in order to avoid unrealistic results. A dust mask screening method is designed, which is based on selecting only those observations where the deterministic model produces a substantial amount of dust. This screen algorithm is tested to give more accurate result compared to the traditional method based on background covariance in the case study. Note that our screen method would exclude valuable information in case the model is not able to simulate the dust plume shape correctly; hence, applications in related studies require inspections of simulations and observations by user.

@en