The Experts below are selected from a list of 90402 Experts worldwide ranked by ideXlab platform
Lars Nerger - One of the best experts on this subject based on the ideXlab platform.
-
Efficient ensemble Data Assimilation for coupled models with the Parallel Data Assimilation Framework: example of AWI-CM (AWI-CM-PDAF 1.0)
Geoscientific Model Development, 2020Co-Authors: Lars Nerger, Qi TangAbstract:Abstract. Data Assimilation integrates information from observational measurements with numerical models. When used with coupled models of Earth system compartments, e.g., the atmosphere and the ocean, consistent joint states can be estimated. A common approach for Data Assimilation is ensemble-based methods which utilize an ensemble of state realizations to estimate the state and its uncertainty. These methods are far more costly to compute than a single coupled model because of the required integration of the ensemble. However, with uncoupled models, the ensemble methods also have been shown to exhibit a particularly good scaling behavior. This study discusses an approach to augment a coupled model with Data Assimilation functionality provided by the Parallel Data Assimilation Framework (PDAF). Using only minimal changes in the codes of the different compartment models, a particularly efficient Data Assimilation system is generated that utilizes parallelization and in-memory Data transfers between the models and the Data Assimilation functions and hence avoids most of the file reading and writing, as well as model restarts during the Data Assimilation process. This study explains the required modifications to the programs with the example of the coupled atmosphere–sea-ice–ocean model AWI-CM (AWI Climate Model). Using the case of the Assimilation of oceanic observations shows that the Data Assimilation leads only to small overheads in computing time of about 15 % compared to the model without Data Assimilation and a very good parallel scalability. The model-agnostic structure of the Assimilation software ensures a separation of concerns in which the development of Data Assimilation methods can be separated from the model application.
-
Efficient ensemble Data Assimilation for coupled models with theParallel Data Assimilation Framework: Example of AWI-CM
2019Co-Authors: Lars Nerger, Qi TangAbstract:Abstract. Data Assimilation integrates information from observational measurements with numerical models. When used with coupled models of Earth system compartments, e.g. the atmosphere and the ocean, consistent joint states can be estimated. A common approach for Data Assimilation are ensemble-based methods which use an ensemble of state realizations to estimate the state and its uncertainty. These methods are far more costly to compute than a single coupled model because of the required integration of the ensemble. However, with uncoupled models, the methods also have been shown to exhibit a particularly good scaling behavior. This study discusses an approach to augment a coupled model with Data Assimilation functionality provided by the Parallel Data Assimilation Framework (PDAF). Using only minimal changes in the codes of the different compartment models, a particularly efficient Data Assimilation system is generated that utilizes parallelization and in-memory Data transfers between the models and the Data Assimilation functions and hence avoids most of the filter reading and writing and also model restarts during the Data Assimilation process. The study explains the required modifications of the programs on the example of the coupled atmosphere-sea ice-ocean model AWI-CM. Using the case of the Assimilation of oceanic observations shows that the Data Assimilation leads only small overheads in computing time of about 15 % compared to the model without Data Assimilation and a very good parallel scalability. The model-agnostic structure of the Assimilation software ensures a separation of concerns in that the development of Data Assimilation methods and be separated from the model application.
-
Building an Efficient Ensemble Data Assimilation System for Coupled Models with the Parallel Data Assimilation Framework
2019Co-Authors: Lars Nerger, Qi Tang, Dmitry SidorenkoAbstract:We discuss how to build an ensemble Data Assimilation system using a direct connection between a coupled model system and the ensemble Data Assimilation software PDAF (Parallel Data Assimilation Framework, http://pdaf.awi.de). The direct connection results in a Data Assimilation program with high flexibility, efficiency, and parallel scalability. For this we augment the source code of the coupled model by Data Assimilation routines and hence create an online-coupled assimilative model. This first modifies the coupled model to be able to simulate an ensemble. Using a combination of in-memory access and parallel communication with the Message Passing Interface (MPI) standard we can further add the analysis step of ensemble-based filter methods, which compute the Assimilation of observations, without the need to stop and restart the whole coupled model system. Instead, the analysis step is performed in between time steps and is independent of the actual model coupler that couples the different model compartments. This strategy to build the Assimilation system allows us to perform both weakly coupled (in-compartment) and strongly coupled (cross-compartment) Assimilation. The Assimilation frequency can be kept flexible, so that the Assimilation of observations from different compartments can be performed at different intervals. Further, the reading and writing of disk files is minimized. The resulting assimilative model can be run in the same way as the regular coupled model, but with additional parameters controlling the Assimilation and with a higher number of processors to simulate the ensemble. Using the example of the coupled climate model AWI-CM that contains the FESOM model for the ocean and sea ice and ECHAM6 for the atmosphere, both coupled through the OASIS-MCT coupler, we discuss the features of the online Assimilation coupling strategy and the performance of the resulting assimilative model.
-
Tutorial: Ensemble Data Assimilation with the Parallel Data Assimilation Framework
2018Co-Authors: Lars NergerAbstract:Ensemble Data Assimilation (EnDA) is used to combine numerical models and observations in a quantitative way. EnDA allows us to join the information from model and observations, e.g. for a better estimate of the system state in all variables represented by the model, include those which are not observed. Further, one can improve the model formulation through the estimation of model parameters. An ensemble of model state realizations is used to estimate the uncertainty of the model state and correlations between different variables. To simplify the implementation of EnDA with numerical models, the open-source Parallel Data Assimilation Framework (PDAF, http://pdaf.awi.de) has been developed. PDAF provides support for the ensemble simulations and optimized filter algorithms so that one can implement the EnDA with very small changes to a model code. This tutorial will first provide an overview of possibilities and components of EnDA. Subsequently, the example of combining the ocean general circulation model MITgcm with PDAF will be used to discuss the required implementation steps for adding EnDA to a model. The tutorial should be useful for scientists to get an overview of the EnDA methodology and to learn how ensemble Data Assimilation can be added to a numerical model.
-
Ensemble Data Assimilation with the parallel Data Assimilation framework PDAF
2017Co-Authors: Lars NergerAbstract:The Parallel Data Assimilation Framework (PDAF) is a unified framework for ensemble Data Assimilation. PDAF has been developed to simplify the implementation of scalable ensemble Data Assimilation systems with existing high-dimensional numerical models. It provides support for the parallelization of the ensemble integration and fully implemented and parallelized ensemble Kalman and nonlinear filters. PDAF encapsulates the filter algorithms so that model and Data Assimilation developments can be conducted separately. I will review the structure and features of PDAF and discuss its use in different applications of ocean-biogeochemical and coupled atmosphere-ocean models.
D. Fonteyn - One of the best experts on this subject based on the ideXlab platform.
-
Data Assimilation of stratospheric constituents: a review
Atmospheric Chemistry and Physics, 2007Co-Authors: William Lahoz, Q. Errera, R. Swinbank, D. FonteynAbstract:The Data Assimilation of stratospheric constituents is reviewed. Several Data Assimilation methods are introduced, with particular consideration to their application to stratospheric constituent measurements. Differences from meteorological Data Assimilation are outlined. Historically, two approaches have been used to carry out constituent Assimilation. One approach has carried constituent Assimilation out as part of a Numerical Weather Prediction system; the other has carried it out in a standalone chemical model, often with a more sophisticated representation of chemical processes. Whereas the aim of the Numerical Weather Prediction approach has been to improve weather forecasts, the aims of the chemical model approach have included providing chemical forecasts and analyses of chemical constituents. A range of constituent Assimilation systems developed in these two areas is presented and strengths and weaknesses discussed. The use of stratospheric constituent Data Assimilation to evaluate models, observations and analyses, and to provide analyses of constituents, monitor ozone, and make ozone forecasts is discussed. Finally, the current state of affairs is assessed, future directions are discussed, and potential key drivers identified.
-
Data Assimilation of stratospheric constituents: a review
2007Co-Authors: William Lahoz, Q. Errera, R. Swinbank, D. FonteynAbstract:Abstract. The Data Assimilation of stratospheric constituents is reviewed. The Data Assimilation method is introduced, with particular consideration to its application to stratospheric constituent measurements. Differences from meteorological Data Assimilation are outlined. Historically, two approaches have been used to carry out constituent Assimilation. One approach has carried constituent Assimilation out as part of a numerical weather prediction system; the other has carried it out in a standalone chemical model, often with a more sophisticated representation of chemical processes. Whereas the aim of the numerical weather prediction approach has been to improve weather forecasts, the aims of the chemical model approach have included providing chemical forecasts and analyses of chemical constituents. A range of constituent Assimilation systems developed in these two areas is presented and strengths and weaknesses discussed. The use of stratospheric constituent Data Assimilation to evaluate models, observations and analyses, and to provide analyses of constituents, monitor ozone, and make ozone forecasts is discussed. Finally, the current state of affairs is assessed, future directions are discussed, and potential key drivers identified.
Paul R Houser - One of the best experts on this subject based on the ideXlab platform.
-
Soil Moisture Data Assimilation
Data Assimilation for Atmospheric Oceanic and Hydrologic Applications (Vol. III), 2016Co-Authors: Viviana Maggioni, Paul R HouserAbstract:Soil moisture plays an important role in the global to regional water and energy cycle, as it controls the partitioning of water and radiation into runoff, evaporation and infiltration at the land-atmosphere interface. Soil moisture information can be obtained through in situ observations, land surface models and remote-sensing retrievals. This chapter reviews the capability of land Data Assimilation systems to merge observations (either in situ or remotely sensed) with the spatially and temporally complete information from land surface models, in order to provide an improved dynamic representation of surface and root-zone soil moisture . Among the different Data Assimilation techniques, the ensemble Kalman Filter and variational methods are becoming the methods of choice for large-scale soil moisture Data Assimilation . The improvement in soil moisture estimation via Data Assimilation largely depends on the quality of the land surface model meteorological forcings. Since precipitation is the major driver for soil moisture, uncertainty in precipitation affects the efficiency of assimilating soil moisture observations most profoundly. Data Assimilation systems have also been demonstrated to be extremely valuable for downscaling coarser resolution satellite brightness temperature observations in order to produce higher resolution soil moisture estimates. Skill metrics for evaluating the improvement in soil moisture estimates via land Data Assimilation is also maturing. Besides biases and root mean square errors, common metrics to evaluate Data Assimilation are the anomaly correlation coefficient, the exceedance and uncertainty ratios and rank histograms.
-
the global land Data Assimilation system
Bulletin of the American Meteorological Society, 2004Co-Authors: Mathew Rodell, Paul R Houser, U Jambor, Jon Gottschalck, Kenneth E Mitchell, C J Meng, Kristi R Arsenault, B Cosgrove, J Radakovich, Michael G BosilovichAbstract:A Global Land Data Assimilation System (GLDAS) has been developed. Its purpose is to ingest satellite- and ground-based observational Data products, using advanced land surface modeling and Data Assimilation techniques, in order to generate optimal fields of land surface states and fluxes. GLDAS is unique in that it is an uncoupled land surface modeling system that drives multiple models, integrates a huge quantity of observation-based Data, runs globally at high resolution (0.25°), and produces results in near–real time (typically within 48 h of the present). GLDAS is also a test bed for innovative modeling and Assimilation capabilities. A vegetation-based “tiling” approach is used to simulate subgrid-scale variability, with a 1-km global vegetation Dataset as its basis. Soil and elevation parameters are based on high-resolution global Datasets. Observation-based precipitation and downward radiation and output fields from the best available global coupled atmospheric Data Assimilation systems are employe...
-
Advances in Land Data Assimilation Systems
2001Co-Authors: Paul R HouserAbstract:Assimilation of remotely sensed land surface observations into regional to global scale numerical models have the potential to significantly advance our ability, to assess, understand, and predict surface water, energy, and carbon cycles. This session seeks to assess the state-of-the-art in Data Assimilation methods for integrating land surface remote sensing and modeling, with a focus on practical applications and techniques. Assimilated land surface variables of interest include (but are not limited to, soil moisture, surface temperature, snowpack, streamflow, vegetation dynamics, and carbon storage. Contributions describing the development of practical land surface Data Assimilation methods, multivariate land surface Data Assimilation strategies, evaluation of the required accuracy and resolution of remote sensing observations, the effects of scale, process complexity, and uncertainty on Data Assimilation, and the optimal treatment of model and observation errors are encouraged.
Avelino Avellano - One of the best experts on this subject based on the ideXlab platform.
-
the Data Assimilation research testbed a community facility
Bulletin of the American Meteorological Society, 2009Co-Authors: Jeffrey L Anderson, Timothy J Hoar, Nancy Collins, Ryan D Torn, Kevin Raeder, Avelino AvellanoAbstract:The Data Assimilation Research Testbed (DART) is an open-source community facility for Data Assimilation education, research, and development. DART's ensemble Data Assimilation algorithms, careful ...
-
the Data Assimilation research testbed a community facility
Bulletin of the American Meteorological Society, 2009Co-Authors: Jeffrey L Anderson, Timothy J Hoar, Nancy Collins, Ryan D Torn, Kevin Raeder, Avelino AvellanoAbstract:Abstract The Data Assimilation Research Testbed (DART) is an open-source community facility for Data Assimilation education, research, and development. DART's ensemble Data Assimilation algorithms, careful software engineering, and diagnostic tools allow atmospheric scientists, oceanographers, hydrologists, chemists, and other geophysicists to build state-of-the-art Data Assimilation systems with unprecedented ease. For global numerical weather prediction, DART produces ensemble-mean analyses comparable to analyses from major centers while also providing initial conditions for ensemble predictions. In addition, DART supports more novel Assimilation applications like parameter estimation, sensitivity analysis, observing system design, and smoothing. Implementing basic systems for large models requires only a few person-weeks; comprehensive systems have been built in a few months. Incorporating new observation types is also straightforward, requiring only a forward operator mapping between a model's state a...
Jens Schröter - One of the best experts on this subject based on the ideXlab platform.
-
Scalable sequential Data Assimilation with the Parallel Data Assimilation Framework PDAF
2013Co-Authors: Lars Nerger, Jens Schröter, Wolfgang HillerAbstract:Data Assimilation applications with high-dimensional numerical models exhibit extreme requirements on computational resources. Good scalability of the Assimilation system is necessary to make these applications feasible. Sequential Data Assimilation methods based on ensemble forecasts, like ensemble-based Kalman filters, provide such good scalability, because the forecast of each ensemble member can be performed independently. However, this parallelism has to be combined with the parallelization of both the numerical model and the Data Assimilation algorithm. In order to simplify the implementation of scalable Data Assimilation systems based on existing numerical models, the Parallel Data Assimilation Framework PDAF [http://pdaf.awi.de] has been developed. PDAF is suitable for educational use with toy models but also for high-dimensional applications and operational use. PDAF is distributed as open source software. PDAF provides a framework for implementing a Data Assimilation system with parallel ensemble forecasts and parallel numerical models. For maximum efficiency, a single Assimilation program can be built that includes both the model and the analysis step. A well-defined interface connects PDAF to the model as well as to the observations. To compute the analysis, PDAF provides several optimized parallel filter algorithms and smoothers. Included are ensemble filters like the Local Ensemble Transform Kalman Filter (LETKF) and the Error Subspace Transform Kalman Filter (ESTKF). We discuss the philosophy behind PDAF as well as features and scalability of Data Assimilation systems based on PDAF on the example of Data Assimilation with the finite element ocean model FEOM.
-
The Parallel Data Assimilation Framework PDAF - a flexible softwareframework for ensemble Data Assimilation
2012Co-Authors: Lars Nerger, Wolfgang Hiller, Jens SchröterAbstract:Ensemble filter algorithms can be implemented in a generic way such that they can be applied with various models with only a minimum amount of recoding. This is possible due to the fact that ensemble filters can operate on abstract state vectors and require only limited information about the numerical model and the observational Data used for a Data Assimilation application. To build an Assimilation system, the analysis step of a filter algorithm needs to be connected to the numerical model. Furthermore, ensemble integrations have to be enabled. The Parallel Data Assimilation Framework PDAF has been developed to provide these features: It is a generic framework that allows to extend a numerical model with a filter to build an ensemble Data Assimilation system with minimal changes to the model code. PDAF also provides a selection of common ensemble Kalman filter algorithms. As the computational cost of ensemble Data Assimilation is a multiple of that of a pure forward model, the framework and the filter algorithms are parallelized and support parallelized models. Thus, Data Assimilation with high-dimensional numerical models is feasible. PDAF is coded in Fortran and available as free software (http://pdaf.awi.de). We discuss the features of PDAF and the parallel computing performance of Data Assimilation systems based on PDAF on the example of Data Assimilation with the finite element ocean model FEOM.
-
Scalable sequential Data Assimilation with the Parallel Data Assimilation Framework PDAF
2011Co-Authors: Lars Nerger, Wolfgang Hiller, Jens SchröterAbstract:Data Assimilation applications with large-scale numerical models exhibit extreme requirements on computational resources. Good scalability of the Assimilation system is necessary to make these applications feasible. Sequential Data Assimilation methods based on ensemble forecasts, like ensemble-based Kalman filters, provide such good scalability, because the forecast of each ensemble member can be performed independently. However, this parallelism has to be combined with the parallelization of both the numerical model and the Data Assimilation algorithm. In order to simplify the implementation of scalable Data Assimilation systems based on existing numerical models, the Parallel Data Assimilation Framework PDAF (http://pdaf.awi.de) has been developed. PDAF provides support for implementing a Data Assimilation system with parallel ensemble forecasts and parallel numerical models. Further, it includes several optimized parallel filter algorithms, like the Ensemble Transform Kalman Filter. We will discuss the philosophy behind PDAF as well as features and scalability of Data Assimilation systems based on PDAF on the example of Data Assimilation with the finite element ocean model FEOM.
-
Scalable sequential Data Assimilation with the Parallel Data Assimilation Framework PDAF
2010Co-Authors: Lars Nerger, Wolfgang Hiller, Jens SchröterAbstract:Data Assimilation applications with high-dimensional numerical modelsshow extreme requirements on computational resources. Thus, goodscalability of the Assimilation system is necessary to make theseapplications feasible. Sequential Data Assimilation methods based onensemble forecasts, like ensemble-based Kalman filters, provide suchgood scalability, because the forecast of each ensemble member can beperformed independently. However, this parallelism has to be combinedwith the parallelization of both the numerical model and the Data Assimilation algorithm. In order to simplify the implementation ofscalable Data Assimilation systems based on existing numerical models,the Parallel Data Assimilation Framework PDAF has been developed. Itprovides support for parallel ensemble forecasts and parallelnumerical models. Further, it includes several optimized parallel filteralgorithms, like the ensemble transform Kalman filter. We will discussthe features and scalability of Data Assimilation systems based onPDAF on the example of Data Assimilation with the finite element oceanmodel FEOM.
-
Towards Operational Data Assimilation in the North and Baltic Seas with the Parallel Data Assimilation Framework
2010Co-Authors: Lars Nerger, Wolfgang Hiller, Jens Schröter, Svetlana Loza, Frank JanssenAbstract:Within the GMES-related project DeMarine Environment, the operational circulation model of the German Maritime and Hydrographic Agency (BSH) is extended into a Data Assimilation system. The aim of the Data Assimilation is to improve the forecast of sea surface height, temperatura, currents and salinity in the North and Baltic Seas. For the Data Assimilation component, the Parallel Data Assimilation Framework (PDAF, http://pdaf.awi.de) is coupled to the operational circulation model. PDAF provides the Assimilation environment as well as fully implemented and optimized filter algorithms. We discuss technical aspects of the Data Assimilation system used with the BSH operational circulation model.
