The Experts below are selected from a list of 360 Experts worldwide ranked by ideXlab platform

Wei Zhang - One of the best experts on this subject based on the ideXlab platform.

  • analytical convergence regions of accelerated gradient descent in nonconvex optimization under Regularity Condition
    Automatica, 2020
    Co-Authors: Huaqing Xiong, Yuejie Chi, Wei Zhang
    Abstract:

    Abstract There is a growing interest in using robust control theory to analyze and design optimization and machine learning algorithms. This paper studies a class of nonconvex optimization problems whose cost functions satisfy the so-called Regularity Condition (RC). Empirical studies show that accelerated gradient descent (AGD) algorithms (e.g. Nesterov’s acceleration and Heavy-ball) with proper initializations often work well in practice. However, the convergence of such AGD algorithms is largely unknown in the literature. The main contribution of this paper is the analytical characterization of the convergence regions of AGD under RC via robust control tools. Since such optimization problems arise frequently in many applications such as phase retrieval, training of neural networks and matrix sensing, our result shows promise of robust control theory in these areas.

  • analytical convergence regions of accelerated first order methods in nonconvex optimization under Regularity Condition
    arXiv: Optimization and Control, 2018
    Co-Authors: Huaqing Xiong, Yuejie Chi, Wei Zhang
    Abstract:

    Gradient descent (GD) converges linearly to the global optimum for even nonconvex problems when the loss function satisfies certain benign geometric properties that are strictly weaker than strong convexity. One important property studied in the literature is the so-called Regularity Condition (RC). The RC property has been proven valid for many problems such as deep linear neural networks, shallow neural networks with nonlinear activations, phase retrieval, to name a few. Moreover, accelerated first-order methods (e.g. Nesterov's accelerated gradient and Heavy-ball) achieve great empirical success when the parameters are tuned properly but lack theoretical understandings in the nonconvex setting. In this paper, we use tools from robust control to analytically characterize the region of hyperparameters that ensure linear convergence of accelerated first-order methods under RC. Our results apply to all functions satisfying RC and therefore are more general than results derived for specific problem instances. We derive a set of Linear Matrix Inequalities (LMIs), based on which analytical regions of convergence are obtained by exploiting the Kalman-Yakubovich-Popov (KYP) lemma. Our work provides deeper understandings on the convergence behavior of accelerated first-order methods in nonconvex optimization.

Satoru Takahashi - One of the best experts on this subject based on the ideXlab platform.

  • coordination failure in repeated games with private monitoring
    Journal of Economic Theory, 2013
    Co-Authors: Takuo Sugaya, Satoru Takahashi
    Abstract:

    Abstract Players coordinate continuation play in repeated games with public monitoring. We investigate the robustness of such equilibrium behavior with respect to ex-ante small private-monitoring perturbations. We show that with full support of public signals, no perfect public equilibrium is robust if it induces a “regular” 2 × 2 coordination game in the continuation play. This Regularity Condition is violated in all belief-free equilibria. Indeed, with an individual full rank Condition, every interior belief-free equilibrium is robust. We also analyze block belief-free equilibria and point out that the notion of robustness is sensitive to whether we allow for uninterpretable signals.

  • coordination failure in repeated games with private monitoring
    Research Papers in Economics, 2011
    Co-Authors: Takuo Sugaya, Satoru Takahashi
    Abstract:

    Players coordinate continuation play in repeated games with public monitoring. This paper asks the robustness of such equilibrium play with respect to privatemonitoring perturbations that are ex-ante close to the public-monitoring structure. We show that, in two-player games with full support of public signals, no perfect public equilibrium is robust to private-monitoring perturbations under a Regularity Condition. This non-robustness result does not apply to belief-free equilibria, which violate the Regularity Condition. Indeed, we show that, in two-player games with an individual rank Condition on public signals, every interior belief-free equilibrium is robust to private-monitoring perturbations. We also argue by means of an example that the non-robustness result is sensitive to the assumption that every private signal must be interpreted as some public signal with probability 1, and not with probability close to 1.

Miguel A Arcones - One of the best experts on this subject based on the ideXlab platform.

Huaqing Xiong - One of the best experts on this subject based on the ideXlab platform.

  • analytical convergence regions of accelerated gradient descent in nonconvex optimization under Regularity Condition
    Automatica, 2020
    Co-Authors: Huaqing Xiong, Yuejie Chi, Wei Zhang
    Abstract:

    Abstract There is a growing interest in using robust control theory to analyze and design optimization and machine learning algorithms. This paper studies a class of nonconvex optimization problems whose cost functions satisfy the so-called Regularity Condition (RC). Empirical studies show that accelerated gradient descent (AGD) algorithms (e.g. Nesterov’s acceleration and Heavy-ball) with proper initializations often work well in practice. However, the convergence of such AGD algorithms is largely unknown in the literature. The main contribution of this paper is the analytical characterization of the convergence regions of AGD under RC via robust control tools. Since such optimization problems arise frequently in many applications such as phase retrieval, training of neural networks and matrix sensing, our result shows promise of robust control theory in these areas.

  • analytical convergence regions of accelerated first order methods in nonconvex optimization under Regularity Condition
    arXiv: Optimization and Control, 2018
    Co-Authors: Huaqing Xiong, Yuejie Chi, Wei Zhang
    Abstract:

    Gradient descent (GD) converges linearly to the global optimum for even nonconvex problems when the loss function satisfies certain benign geometric properties that are strictly weaker than strong convexity. One important property studied in the literature is the so-called Regularity Condition (RC). The RC property has been proven valid for many problems such as deep linear neural networks, shallow neural networks with nonlinear activations, phase retrieval, to name a few. Moreover, accelerated first-order methods (e.g. Nesterov's accelerated gradient and Heavy-ball) achieve great empirical success when the parameters are tuned properly but lack theoretical understandings in the nonconvex setting. In this paper, we use tools from robust control to analytically characterize the region of hyperparameters that ensure linear convergence of accelerated first-order methods under RC. Our results apply to all functions satisfying RC and therefore are more general than results derived for specific problem instances. We derive a set of Linear Matrix Inequalities (LMIs), based on which analytical regions of convergence are obtained by exploiting the Kalman-Yakubovich-Popov (KYP) lemma. Our work provides deeper understandings on the convergence behavior of accelerated first-order methods in nonconvex optimization.

Takuo Sugaya - One of the best experts on this subject based on the ideXlab platform.

  • coordination failure in repeated games with private monitoring
    Journal of Economic Theory, 2013
    Co-Authors: Takuo Sugaya, Satoru Takahashi
    Abstract:

    Abstract Players coordinate continuation play in repeated games with public monitoring. We investigate the robustness of such equilibrium behavior with respect to ex-ante small private-monitoring perturbations. We show that with full support of public signals, no perfect public equilibrium is robust if it induces a “regular” 2 × 2 coordination game in the continuation play. This Regularity Condition is violated in all belief-free equilibria. Indeed, with an individual full rank Condition, every interior belief-free equilibrium is robust. We also analyze block belief-free equilibria and point out that the notion of robustness is sensitive to whether we allow for uninterpretable signals.

  • coordination failure in repeated games with private monitoring
    Research Papers in Economics, 2011
    Co-Authors: Takuo Sugaya, Satoru Takahashi
    Abstract:

    Players coordinate continuation play in repeated games with public monitoring. This paper asks the robustness of such equilibrium play with respect to privatemonitoring perturbations that are ex-ante close to the public-monitoring structure. We show that, in two-player games with full support of public signals, no perfect public equilibrium is robust to private-monitoring perturbations under a Regularity Condition. This non-robustness result does not apply to belief-free equilibria, which violate the Regularity Condition. Indeed, we show that, in two-player games with an individual rank Condition on public signals, every interior belief-free equilibrium is robust to private-monitoring perturbations. We also argue by means of an example that the non-robustness result is sensitive to the assumption that every private signal must be interpreted as some public signal with probability 1, and not with probability close to 1.