The Experts below are selected from a list of 6231 Experts worldwide ranked by ideXlab platform
Jeroen Tromp - One of the best experts on this subject based on the ideXlab platform.
-
Preconditioned BFGS-based Uncertainty Quantification in elastic Full Waveform Inversion
arXiv: Computational Physics, 2020Co-Authors: Qiancheng Liu, Stephen Beller, W. Lei, Daniel Peter, Jeroen TrompAbstract:Full Waveform Inversion (FWI) plays a vital role in reconstructing geophysical structures. The Uncertainty Quantification regarding the inversion results is equally important but has been missing out in most of the current geophysical inversions. Mathematically, uncertainty quantification is involved with the inverse Hessian (or the posterior covariance matrix), which is prohibitive in computation and storage for practical geophysical FWI problems. L-BFGS populates as the most efficient Gauss-Newton method; however, in this study, we empower it with the new possibility of accessing the inverse Hessian for uncertainty quantification in FWI. To facilitate the inverse-Hessian retrieval, we put together BFGS (essentially, full-history L-BFGS) with randomized singular value decomposition towards a low-rank approximation of the Hessian inverse. That the rank number equals the number of iterations makes this solution efficient and memory-affordable even for large-scale inversions. Also, based on the adjoint method, we formulate different diagonal Hessian initials as preconditioners and compare their performances in elastic FWI. We highlight our methods with the elastic Marmousi benchmark, demonstrating the applicability of preconditioned BFGS in large-scale FWI and uncertainty quantification.
Yunong Zhang - One of the best experts on this subject based on the ideXlab platform.
-
discrete time zhang neural network for online time varying nonlinear optimization with application to manipulator motion generation
IEEE Transactions on Neural Networks, 2015Co-Authors: Long Jin, Yunong ZhangAbstract:In this brief, a discrete-time Zhang neural network (DTZNN) model is first proposed, developed, and investigated for online time-varying nonlinear optimization (OTVNO). Then, Newton iteration is shown to be derived from the proposed DTZNN model. In addition, to eliminate the explicit matrix-inversion operation, the quasi-Newton Broyden–Fletcher–Goldfarb–Shanno (BFGS) method is introduced, which can effectively approximate the inverse of Hessian matrix. A DTZNN-BFGS model is thus proposed and investigated for OTVNO, which is the combination of the DTZNN model and the quasi-Newton BFGS method. In addition, theoretical analyses show that, with step-size $h=1$ and/or with zero initial error, the maximal residual error of the DTZNN model has an $O(\tau ^{2})$ pattern, whereas the maximal residual error of the Newton iteration has an $O(\tau )$ pattern, with $\tau $ denoting the sampling gap. Besides, when $h\neq 1$ and $h\in (0,2)$ , the maximal steady-state residual error of the DTZNN model has an $O(\tau ^{2})$ pattern. Finally, an illustrative numerical experiment and an application example to manipulator motion generation are provided and analyzed to substantiate the efficacy of the proposed DTZNN and DTZNN-BFGS models for OTVNO.
-
link between and comparison and combination of zhang neural network and quasi newton BFGS method for time varying quadratic minimization
IEEE Transactions on Systems Man and Cybernetics, 2013Co-Authors: Yunong Zhang, Huicheng ZhengAbstract:Since 2001, a novel type of recurrent neural network called Zhang neural network (ZNN) has been proposed, investigated, and exploited for solving online time-varying problems in a variety of scientific and engineering fields. In this paper, three discrete-time ZNN models are first proposed to solve the problem of time-varying quadratic minimization (TVQM). Such discrete-time ZNN models exploit methodologically the time derivatives of time-varying coefficients and the inverse of the time-varying coefficient matrix. To eliminate explicit matrix-inversion operation, the quasi-Newton BFGS method is introduced, which approximates effectively the inverse of the Hessian matrix; thus, three discrete-time ZNN models combined with the quasi-Newton BFGS method (named ZNN-BFGS) are proposed and investigated for TVQM. In addition, according to the criterion of whether the time-derivative information of time-varying coefficients is explicitly known/used or not, these proposed discrete-time models are classified into three categories: 1) models with time-derivative information known (i.e., ZNN-K and ZNN-BFGS-K models), 2) models with time-derivative information unknown (i.e., ZNN-U and ZNN-BFGS-U models), and 3) simplified models without using time-derivative information (i.e., ZNN-S and ZNN-BFGS-S models). The well-known gradient-based neural network is also developed to handle TVQM for comparison with the proposed ZNN and ZNN-BFGS models. Illustrative examples are provided and analyzed to substantiate the efficacy of these proposed models for TVQM.
-
random scaling of quasi newton BFGS method to improve the o n 2 operation approximation of covariance matrix inverse in gaussian process
International Symposium on Intelligent Control, 2007Co-Authors: Yunong Zhang, William Leithead, Douglas J. LeithAbstract:Gaussian process (GP) is a Bayesian nonparametric regression model, showing good performance in various applications. Similar to other computational models, Gaussian process frequently encounters the matrix-inverse problem during its model-tuning procedure. The matrix inversion is generally of O(N3) operations where N is the matrix dimension. We proposed using the O(N2)-operation quasi-Newton BFGS method to approximate/replace the exact inverse of covariance matrix in the GP context. As inspired during a paper revision, in this paper we show that by using the random-scaling technique, the accuracy and effectiveness of such a BFGS matrix-inverse approximation could be further improved. These random-scaling BFGS techniques could be widely generalized to other machine-learning systems which rely on explicit matrix-inverse.
-
Random Scaling of Quasi-Newton BFGS Method to Improve the O(N 2 )-operation Approximation of Covariance-matrix Inverse in
2007Co-Authors: Yunong Zhang, William Leithead, Douglas J. LeithAbstract:Gaussian process (GP) is a Bayesian nonparamet- ric regression model, showing good performance in various applications. Similar to other computational models, Gaussian process frequently encounters the matrix-inverse problem dur- ing its model-tuning procedure. The matrix inversion is gen- erally of O(N 3 ) operations where N is the matrix dimension. We proposed using the O(N 2 )-operation quasi-Newton BFGS method to approximate/replace the exact inverse of covariance matrix in the GP context. As inspired during a paper revision, in this paper we show that by using the random-scaling technique, the accuracy and effectiveness of such a BFGS matrix-inverse approximation could be further improved. These random- scaling BFGS techniques could be widely generalized to other machine-learning systems which rely on explicit matrix-inverse.
Gonglin Yuan - One of the best experts on this subject based on the ideXlab platform.
-
a limited memory BFGS algorithm based on a trust region quadratic model for large scale nonlinear equations
PLOS ONE, 2015Co-Authors: Yong Li, Gonglin YuanAbstract:In this paper, a trust-region algorithm is proposed for large-scale nonlinear equations, where the limited-memory BFGS (L-M-BFGS) update matrix is used in the trust-region subproblem to improve the effectiveness of the algorithm for large-scale problems. The global convergence of the presented method is established under suitable conditions. The numerical results of the test problems show that the method is competitive with the norm method.
-
limited memory BFGS method with backtracking for symmetric nonlinear equations
Mathematical and Computer Modelling, 2011Co-Authors: Gonglin Yuan, Zengxin WeiAbstract:A limited memory BFGS algorithm is presented for solving symmetric nonlinear equations. The global convergence of the given method is established under mild conditions. Numerical results show that the limited memory BFGS method is more interesting than the normal BFGS method for the given problems.
-
the superlinear convergence of a modified BFGS type method for unconstrained optimization
Computational Optimization and Applications, 2004Co-Authors: Zengxin Wei, Gonglin Yuan, Zhigang LianAbstract:The BFGS method is the most effective of the quasi-Newton methods for solving unconstrained optimization problems. Wei, Li, and Qi [16] have proposed some modified BFGS methods based on the new quasi-Newton equation Bk+1sk e y*k, where y*k is the sum of yk and Aksk, and Ak is some matrix. The average performance of Algorithm 4.3 in [16] is better than that of the BFGS method, but its superlinear convergence is still open. This article proves the superlinear convergence of Algorithm 4.3 under some suitable conditions.
Qiancheng Liu - One of the best experts on this subject based on the ideXlab platform.
-
Preconditioned BFGS-based Uncertainty Quantification in elastic Full Waveform Inversion
arXiv: Computational Physics, 2020Co-Authors: Qiancheng Liu, Stephen Beller, W. Lei, Daniel Peter, Jeroen TrompAbstract:Full Waveform Inversion (FWI) plays a vital role in reconstructing geophysical structures. The Uncertainty Quantification regarding the inversion results is equally important but has been missing out in most of the current geophysical inversions. Mathematically, uncertainty quantification is involved with the inverse Hessian (or the posterior covariance matrix), which is prohibitive in computation and storage for practical geophysical FWI problems. L-BFGS populates as the most efficient Gauss-Newton method; however, in this study, we empower it with the new possibility of accessing the inverse Hessian for uncertainty quantification in FWI. To facilitate the inverse-Hessian retrieval, we put together BFGS (essentially, full-history L-BFGS) with randomized singular value decomposition towards a low-rank approximation of the Hessian inverse. That the rank number equals the number of iterations makes this solution efficient and memory-affordable even for large-scale inversions. Also, based on the adjoint method, we formulate different diagonal Hessian initials as preconditioners and compare their performances in elastic FWI. We highlight our methods with the elastic Marmousi benchmark, demonstrating the applicability of preconditioned BFGS in large-scale FWI and uncertainty quantification.
Stephen Beller - One of the best experts on this subject based on the ideXlab platform.
-
Preconditioned BFGS-based Uncertainty Quantification in elastic Full Waveform Inversion
arXiv: Computational Physics, 2020Co-Authors: Qiancheng Liu, Stephen Beller, W. Lei, Daniel Peter, Jeroen TrompAbstract:Full Waveform Inversion (FWI) plays a vital role in reconstructing geophysical structures. The Uncertainty Quantification regarding the inversion results is equally important but has been missing out in most of the current geophysical inversions. Mathematically, uncertainty quantification is involved with the inverse Hessian (or the posterior covariance matrix), which is prohibitive in computation and storage for practical geophysical FWI problems. L-BFGS populates as the most efficient Gauss-Newton method; however, in this study, we empower it with the new possibility of accessing the inverse Hessian for uncertainty quantification in FWI. To facilitate the inverse-Hessian retrieval, we put together BFGS (essentially, full-history L-BFGS) with randomized singular value decomposition towards a low-rank approximation of the Hessian inverse. That the rank number equals the number of iterations makes this solution efficient and memory-affordable even for large-scale inversions. Also, based on the adjoint method, we formulate different diagonal Hessian initials as preconditioners and compare their performances in elastic FWI. We highlight our methods with the elastic Marmousi benchmark, demonstrating the applicability of preconditioned BFGS in large-scale FWI and uncertainty quantification.