The Experts below are selected from a list of 360 Experts worldwide ranked by ideXlab platform
Elias D Ninoruiz - One of the best experts on this subject based on the ideXlab platform.
-
a maximum likelihood ensemble filter via a modified Cholesky Decomposition for non gaussian data assimilation
Sensors, 2020Co-Authors: Elias D Ninoruiz, Alfonso Mancillaherrera, Santiago Lopezrestrepo, Olga QuinteromontoyaAbstract:This paper proposes an efficient and practical implementation of the Maximum Likelihood Ensemble Filter via a Modified Cholesky Decomposition (MLEF-MC). The method works as follows: via an ensemble of model realizations, a well-conditioned and full-rank square-root approximation of the background error covariance matrix is obtained. This square-root approximation serves as a control space onto which analysis increments can be computed. These are calculated via Line-Search (LS) optimization. We theoretically prove the convergence of the MLEF-MC. Experimental simulations were performed using an Atmospheric General Circulation Model (AT-GCM) and a highly nonlinear observation operator. The results reveal that the proposed method can obtain posterior error estimates within reasonable accuracies in terms of l - 2 error norms. Furthermore, our analysis estimates are similar to those of the MLEF with large ensemble sizes and full observational networks.
-
an adjoint free four dimensional variational data assimilation method via a modified Cholesky Decomposition and an iterative woodbury matrix formula
Nonlinear Dynamics, 2020Co-Authors: Elias D Ninoruiz, Luis G Guzmanreyes, Rolando BeltranarrietaAbstract:In this paper, we propose an efficient and practical implementation of a four-dimensional variational ensemble Kalman filter (4D-EnKF) via a modified Cholesky Decomposition. The main scope of our method is to avoid the intrinsic needed of adjoint models in the four-dimensional context. As it is well known, in practice, adjoint models can be labor-intensive to develop and computationally expensive to run. We avoid the use of adjoint models by taking snapshots of an ensemble of model realizations at observation times. Then, we employ a modified Cholesky Decomposition on those ensembles to build control spaces, which in turn are employed to estimate analysis increments and to mitigate the impact of sampling noise. We discuss a matrix-free implementation of our 4D-EnKF formulation via the Woodbury matrix identity. Experimental tests are performed by using the Lorenz 96 model and an atmospheric general circulation model. The numerical results reveal that the accuracy of our proposed filter implementation outperforms those of traditional 4D-EnKF formulations in terms of L-2 error norms and root-mean-square error values.
-
an ensemble kalman filter implementation based on modified Cholesky Decomposition for inverse covariance matrix estimation
SIAM Journal on Scientific Computing, 2018Co-Authors: Elias D Ninoruiz, Adrian Sandu, Xinwei DengAbstract:This paper develops an efficient implementation of the ensemble Kalman filter based on a modified Cholesky Decomposition for inverse covariance matrix estimation. This implementation is named EnKF-MC. Background errors corresponding to distant model components with respect to some radius of influence are assumed to be conditionally independent. This allows one to obtain sparse estimators of the inverse background error covariance matrix. The computational effort of the proposed method is discussed and different formulations based on various matrix identities are provided. Furthermore, an asymptotic proof of convergence with regard to the ensemble size is presented. In order to assess the performance and the accuracy of the proposed method, experiments are performed making use of the atmospheric general circulation model SPEEDY. The results are compared against those obtained using the local ensemble transform Kalman filter (LETKF). Tests are performed for dense observations (100% and 50% of the model comp...
-
a matrix free posterior ensemble kalman filter implementation based on a modified Cholesky Decomposition
Atmosphere, 2017Co-Authors: Elias D NinoruizAbstract:In this paper, a matrix-free posterior ensemble Kalman filter implementation based on a modified Cholesky Decomposition is proposed. The method works as follows: the precision matrix of the background error distribution is estimated based on a modified Cholesky Decomposition. The resulting estimator can be expressed in terms of Cholesky factors which can be updated based on a series of rank-one matrices in order to approximate the precision matrix of the analysis distribution. By using this matrix, the posterior ensemble can be built by either sampling from the posterior distribution or using synthetic observations. Furthermore, the computational effort of the proposed method is linear with regard to the model dimension and the number of observed components from the model domain. Experimental tests are performed making use of the Lorenz-96 model. The results reveal that, the accuracy of the proposed implementation in terms of root-mean-square-error is similar, and in some cases better, to that of a well-known ensemble Kalman filter (EnKF) implementation: the local ensemble transform Kalman filter. In addition, the results are comparable to those obtained by the EnKF with large ensemble sizes.
-
a posterior ensemble kalman filter based on a modified Cholesky Decomposition
International Conference on Conceptual Structures, 2017Co-Authors: Elias D Ninoruiz, Alfonso Mancilla, Juan C CalabriaAbstract:Abstract In this paper, we propose a posterior ensemble Kalman filter (EnKF) based on a modified Cholesky Decomposition. The main idea behind our approach is to estimate the moments of the analysis distribution based on an ensemble of model realizations. The method proceeds as follows: initially, an estimate of the precision background error covariance matrix is computed via a modified Cholesky Decomposition and then, based on rank-one updates, the Cholesky factors of the inverse background error covariance matrix are updated in order to obtain an estimate of the inverse analysis covariance matrix. The special structure of the Cholesky factors can be exploited in order to obtain a matrix-free implementation of the EnKF. Once the analysis covariance matrix is estimated, the posterior mode of the distribution can be approximated and samples about it are taken in order to build the posterior ensemble. Experimental tests are performed making use of the Lorenz 96 model in order to assess the accuracy of the proposed implementation. The results reveal that, the accuracy of the proposed implementation is similar to that of the well-known local ensemble transform Kalman filter and even more, the use of our estimator reduces the impact of sampling errors during the assimilation of observations.
Xinwei Deng - One of the best experts on this subject based on the ideXlab platform.
-
an improved modified Cholesky Decomposition approach for precision matrix estimation
Journal of Statistical Computation and Simulation, 2020Co-Authors: Xiaoning Kang, Xinwei DengAbstract:The modified Cholesky Decomposition is commonly used for precision matrix estimation given a specified order of random variables. However, the order of variables is often not available or cannot be...
-
a parallel implementation of the ensemble kalman filter based on modified Cholesky Decomposition
Journal of Computational Science, 2019Co-Authors: Elias D Nino, Adrian Sandu, Xinwei DengAbstract:Abstract This paper discusses an efficient parallel implementation of the ensemble Kalman filter based on the modified Cholesky Decomposition. The proposed implementation starts with decomposing the domain into sub-domains. In each sub-domain a sparse estimation of the inverse background error covariance matrix is computed via a modified Cholesky Decomposition; the estimates are computed concurrently on separate processors. The sparsity of this estimator is dictated by the conditional independence of model components for some radius of influence. Then, the assimilation step is carried out in parallel without the need of inter-processor communication. Once the local analysis states are computed, the analysis sub-domains are mapped back onto the global domain to obtain the analysis ensemble. Computational experiments are performed using the Atmospheric General Circulation Model (SPEEDY) with the T-63 resolution on the Blueridge cluster at Virginia Tech. The number of processors used in the experiments ranges from 96 to 2048. The proposed implementation outperforms in terms of accuracy the well-known local ensemble transform Kalman filter (LETKF) for all the model variables. The computational time of the proposed implementation is similar to that of the parallel LETKF method (where no covariance estimation is performed). Finally, for the largest number of processors, the proposed parallel implementation is 400 times faster than the serial version of the proposed method.
-
an ensemble kalman filter implementation based on modified Cholesky Decomposition for inverse covariance matrix estimation
SIAM Journal on Scientific Computing, 2018Co-Authors: Elias D Ninoruiz, Adrian Sandu, Xinwei DengAbstract:This paper develops an efficient implementation of the ensemble Kalman filter based on a modified Cholesky Decomposition for inverse covariance matrix estimation. This implementation is named EnKF-MC. Background errors corresponding to distant model components with respect to some radius of influence are assumed to be conditionally independent. This allows one to obtain sparse estimators of the inverse background error covariance matrix. The computational effort of the proposed method is discussed and different formulations based on various matrix identities are provided. Furthermore, an asymptotic proof of convergence with regard to the ensemble size is presented. In order to assess the performance and the accuracy of the proposed method, experiments are performed making use of the atmospheric general circulation model SPEEDY. The results are compared against those obtained using the local ensemble transform Kalman filter (LETKF). Tests are performed for dense observations (100% and 50% of the model comp...
-
An Improved Modified Cholesky Decomposition Method for Precision Matrix Estimation
arXiv: Machine Learning, 2017Co-Authors: Xiaoning Kang, Xinwei DengAbstract:The modified Cholesky Decomposition is commonly used for precision matrix estimation given a specified order of random variables. However, the order of variables is often not available or cannot be pre-determined. In this work, we propose to address the variable order issue in the modified Cholesky Decomposition for sparse precision matrix estimation. The key idea is to effectively combine a set of estimates obtained from multiple permutations of variable orders, and to efficiently encourage the sparse structure for the resultant estimate by the thresholding technique on the ensemble Cholesky factor matrix. The consistent property of the proposed estimate is established under some weak regularity conditions. Simulation studies are conducted to evaluate the performance of the proposed method in comparison with several existing approaches. The proposed method is also applied into linear discriminant analysis of real data for classification.
-
an improved modified Cholesky Decomposition method for inverse covariance matrix estimation
arXiv: Machine Learning, 2017Co-Authors: Xiaoning Kang, Xinwei DengAbstract:The modified Cholesky Decomposition is commonly used for inverse covariance matrix estimation given a specified order of random variables. However, the order of variables is often not available or cannot be pre-determined. Hence, we propose a novel estimator to address the variable order issue in the modified Cholesky Decomposition to estimate the sparse inverse covariance matrix. The key idea is to effectively combine a set of estimates obtained from multiple permutations of variable orders, and to efficiently encourage the sparse structure for the resultant estimate by the use of thresholding technique on the combined Cholesky factor matrix. The consistent property of the proposed estimate is established under some weak regularity conditions. Simulation studies show the superior performance of the proposed method in comparison with several existing approaches. We also apply the proposed method into the linear discriminant analysis for analyzing real-data examples for classification.
Juan C Calabria - One of the best experts on this subject based on the ideXlab platform.
-
a posterior ensemble kalman filter based on a modified Cholesky Decomposition
International Conference on Conceptual Structures, 2017Co-Authors: Elias D Ninoruiz, Alfonso Mancilla, Juan C CalabriaAbstract:Abstract In this paper, we propose a posterior ensemble Kalman filter (EnKF) based on a modified Cholesky Decomposition. The main idea behind our approach is to estimate the moments of the analysis distribution based on an ensemble of model realizations. The method proceeds as follows: initially, an estimate of the precision background error covariance matrix is computed via a modified Cholesky Decomposition and then, based on rank-one updates, the Cholesky factors of the inverse background error covariance matrix are updated in order to obtain an estimate of the inverse analysis covariance matrix. The special structure of the Cholesky factors can be exploited in order to obtain a matrix-free implementation of the EnKF. Once the analysis covariance matrix is estimated, the posterior mode of the distribution can be approximated and samples about it are taken in order to build the posterior ensemble. Experimental tests are performed making use of the Lorenz 96 model in order to assess the accuracy of the proposed implementation. The results reveal that, the accuracy of the proposed implementation is similar to that of the well-known local ensemble transform Kalman filter and even more, the use of our estimator reduces the impact of sampling errors during the assimilation of observations.
Elias D Nino - One of the best experts on this subject based on the ideXlab platform.
-
a parallel implementation of the ensemble kalman filter based on modified Cholesky Decomposition
Journal of Computational Science, 2019Co-Authors: Elias D Nino, Adrian Sandu, Xinwei DengAbstract:Abstract This paper discusses an efficient parallel implementation of the ensemble Kalman filter based on the modified Cholesky Decomposition. The proposed implementation starts with decomposing the domain into sub-domains. In each sub-domain a sparse estimation of the inverse background error covariance matrix is computed via a modified Cholesky Decomposition; the estimates are computed concurrently on separate processors. The sparsity of this estimator is dictated by the conditional independence of model components for some radius of influence. Then, the assimilation step is carried out in parallel without the need of inter-processor communication. Once the local analysis states are computed, the analysis sub-domains are mapped back onto the global domain to obtain the analysis ensemble. Computational experiments are performed using the Atmospheric General Circulation Model (SPEEDY) with the T-63 resolution on the Blueridge cluster at Virginia Tech. The number of processors used in the experiments ranges from 96 to 2048. The proposed implementation outperforms in terms of accuracy the well-known local ensemble transform Kalman filter (LETKF) for all the model variables. The computational time of the proposed implementation is similar to that of the parallel LETKF method (where no covariance estimation is performed). Finally, for the largest number of processors, the proposed parallel implementation is 400 times faster than the serial version of the proposed method.
David C Sherrill - One of the best experts on this subject based on the ideXlab platform.
-
accuracy and efficiency of coupled cluster theory using density fitting Cholesky Decomposition frozen natural orbitals and a t1 transformed hamiltonian
Journal of Chemical Theory and Computation, 2013Co-Authors: Eugene A Deprince, David C SherrillAbstract:We present an algorithm for coupled-cluster through perturbative triples [CCSD(T)] based on a t1-dressed Hamiltonian and the use of density fitting (DF) or Cholesky Decomposition (CD) approximations for the construction and contraction of all electron repulsion integrals (ERIs). An efficient implementation of this algorithm is then used to explore whether preoptimized density fitting basis sets [specifically, the (aug-)cc-pVXZ-RI series designed for DF-MP2 computations] are suitable for DF-CCSD(T) computations and how they compare to the CD representation of the integrals. The code is also used to systematically explore the accuracy and efficiency of DF/CD combined with frozen natural orbitals (FNOs) to reduce computational costs. The mean absolute errors due to DF/CD in the CCSD(T)/aug-cc-pVDZ interaction energies of 11 van der Waals dimers are only 0.001 kcal mol(-1) for the preoptimized RI basis set and only 0.002 and 0.001 kcal mol(-1) for CD with cutoffs of 10(-4) and 10(-5), respectively. The very similar performance of the aug-cc-pVDZ-RI auxiliary set is a bit surprising considering that the numbers of CD vectors using these thresholds are, on average, 28% and 73% larger than the dimension of the RI set. When FNOs are coupled with DF/CD, the DF/CD error is roughly an order of magnitude less than the FNO truncation error (at a conservative FNO occupation cutoff of 10(-5)). Utilizing t1-dressed three-index integrals, which remove the explicit dependence of the doubles residual equations on the t1-amplitudes, results in a moderate performance acceleration for the CCSD portion of the algorithm. Moreover, the t1-dressing results in a simpler code which will be more amenable to parallelization. Utilizing both CD and FNO techniques, we observe a speedup of four times for the evaluation of the three-body contribution to the interaction energy for the benzene trimer described by an aug-cc-pVDZ basis set; the error incurred by the CD and FNO approximations in the three-body contribution is only 0.002 kcal mol(-1).
-
density fitting and Cholesky Decomposition approximations in symmetry adapted perturbation theory implementation and application to probe the nature of π π interactions in linear acenes
Journal of Chemical Physics, 2010Co-Authors: Edward G Hohenstein, David C SherrillAbstract:Density fitting (DF) approximations have been used to increase the efficiency of several quantum mechanical methods. In this work, we apply DF and a related approach, Cholesky Decomposition (CD), to wave function-based symmetry-adapted perturbation theory (SAPT). We also test the one-center approximation to the Cholesky Decomposition. The DF and CD approximations lead to a dramatic improvement in the overall computational cost of SAPT, while introducing negligible errors. For typical target accuracies, the Cholesky basis needed is noticeably smaller than the DF basis (although the cost of constructing the Cholesky vectors is slightly greater than that of constructing the three-index DF integrals). The SAPT program developed in this work is applied to the interactions between acenes previously studied by Grimme [Angew. Chem., Int. Ed. 47, 3430 (2008)], expanding the cases studied by adding the pentacene dimer. The SAPT Decomposition of the acene interactions provides a more realistic picture of the interac...