The Experts below are selected from a list of 291 Experts worldwide ranked by ideXlab platform
Alec Koppel - One of the best experts on this subject based on the ideXlab platform.
-
Nonparametric Compositional Stochastic Optimization for Risk-Sensitive Kernel Learning
IEEE Transactions on Signal Processing, 2021Co-Authors: Amrit Singh Bedi, Alec Koppel, Ketan Rajawat, Panchajanya SanyalAbstract:In this work, we address optimization problems where the objective function is a nonlinear function of an expected value, i.e., compositional stochastic programs. We consider the case where the decision variable is not vector-valued but instead belongs to a Reproducing Kernel Hilbert Space (RKHS), motivated by risk-aware formulations of supervised learning. We develop the first memory-efficient stochastic algorithm for this setting, which we call Compositional Online Learning with Kernels (COLK). COLK, at its core a two time-scale stochastic approximation method, addresses the facts that (i) compositions of expected value problems cannot be addressed by stochastic gradient method due to the presence of an inner expectation; and (ii) the RKHS-induced parameterization has complexity which is proportional to the Iteration Index which is mitigated through greedily constructed subspace projections. We provide, for the first time, a non-asymptotic tradeoff between the complexity of a function parameterization and its required convergence accuracy for both strongly convex and non-convex objectives under constant step-sizes. Experiments with risk-sensitive supervised learning demonstrate that COLK consistently converges and performs reliably even when data is full of outliers, and thus marks a step towards overfitting. Specifically, we observe a favorable tradeoff between model complexity, consistent convergence, and statistical accuracy for data associated with heavy-tailed distributions.
-
Projected Stochastic Primal-Dual Method for Constrained Online Learning With Kernels
IEEE Transactions on Signal Processing, 2019Co-Authors: Alec Koppel, Kaiqing Zhang, Hao Zhu, Tamer BasarAbstract:We consider the problem of stochastic optimization with nonlinear constraints, where the decision variable is not vector-valued but instead a function belonging to a reproducing Kernel Hilbert Space (RKHS). Currently, there exist solutions to only special cases of this problem. To solve this constrained problem with kernels, we first generalize the Representer Theorem to a class of saddle-point problems defined over RKHS. Then, we develop a primal-dual method which that executes alternating projected primal/dual stochastic gradient descent/ascent on the dual-augmented Lagrangian of the problem. The primal projection sets are low-dimensional subspaces of the ambient function space, which are greedily constructed using matching pursuit. By tuning the projection-induced error to the algorithm step-size, we are able to establish mean convergence in both primal objective sub-optimality and constraint violation, to respective ${\mathcal O}(\sqrt{T})$ and ${\mathcal O}(T^{3/4})$ neighborhoods. Here, $T$ is the final Iteration Index and the constant step-size is chosen as $1/\sqrt{T}$ with $1/T$ approximation budget. Finally, we demonstrate experimentally the effectiveness of the proposed method for risk-aware supervised learning.
-
Nonparametric Compositional Stochastic Optimization for Risk-Sensitive Kernel Learning
arXiv: Optimization and Control, 2019Co-Authors: Amrit Singh Bedi, Alec Koppel, Ketan Rajawat, Panchajanya SanyalAbstract:In this work, we address optimization problems where the objective function is a nonlinear function of an expected value, i.e., compositional stochastic {strongly convex programs}. We consider the case where the decision variable is not vector-valued but instead belongs to a reproducing Kernel Hilbert Space (RKHS), motivated by risk-aware formulations of supervised learning and Markov Decision Processes defined over continuous spaces. We develop the first memory-efficient stochastic algorithm for this setting, which we call Compositional Online Learning with Kernels (COLK). COLK, at its core a two-time-scale stochastic approximation method, addresses the fact that (i) compositions of expected value problems cannot be addressed by classical stochastic gradient due to the presence of the inner expectation; and (ii) the RKHS-induced parameterization has complexity which is proportional to the Iteration Index which is mitigated through greedily constructed subspace projections. We establish almost sure convergence of COLK with attenuating step-sizes, and linear convergence in mean to a neighborhood with constant step-sizes, as well as the fact that its complexity is at-worst finite. The experiments with robust formulations of supervised learning demonstrate that COLK reliably converges, attains consistent performance across training runs, and thus overcomes overfitting.
-
Nonparametric Compositional Stochastic Optimization
arXiv: Optimization and Control, 2019Co-Authors: Amrit Singh Bedi, Alec Koppel, Ketan RajawatAbstract:In this work, we address optimization problems where the objective function is a nonlinear function of an expected value, i.e., compositional stochastic {strongly convex programs}. We consider the case where the decision variable is not vector-valued but instead belongs to a reproducing Kernel Hilbert Space (RKHS), motivated by risk-aware formulations of supervised learning and Markov Decision Processes defined over continuous spaces. We develop the first memory-efficient stochastic algorithm for this setting, which we call Compositional Online Learning with Kernels (COLK). COLK, at its core a two-time-scale stochastic approximation method, addresses the fact that (i) compositions of expected value problems cannot be addressed by classical stochastic gradient due to the presence of the inner expectation; and (ii) the RKHS-induced parameterization has complexity which is proportional to the Iteration Index which is mitigated through greedily constructed subspace projections. We establish almost sure convergence of COLK with attenuating step-sizes, and linear convergence in mean to a neighborhood with constant step-sizes, as well as the fact that its complexity is at-worst finite. The experiments with robust formulations of supervised learning demonstrate that COLK reliably converges, attains consistent performance across training runs, and thus overcomes overfitting.
-
ACC - Controlling the Bias-Variance Tradeoff via Coherent Risk for Robust Learning with Kernels
2019 American Control Conference (ACC), 2019Co-Authors: Alec Koppel, Amrit Singh Bedi, Ketan RajawatAbstract:In supervised learning, we learn a statistical model by minimizing a measure of fitness averaged over data. Doing so, however, ignores the variance, i.e., the gap between the optimal within a hypothesized function class and the Bayes Risk. We propose to account for both the bias and variance by modifying training to incorporate coherent risk which quantifies the uncertainty of a given decision. We develop the first online solution to this problem when estimators belong to a reproducing kernel Hilbert space (RKHS), which we call Compositional Online Learning with Kernels (COLK). COLK addresses the fact that (i) minimizing risk functions requires handling objectives which are compositions of expected value problems by generalizing the two time-scale stochastic quasi-gradient method to RKHSs; and (ii) the RKHS-induced parameterization has complexity which is proportional to the Iteration Index which is mitigated through greedily constructed subspace projections. We establish linear convergence in mean to a neighborhood with constant stepsizes, as well as the fact that its complexity is at-worst finite. Experiments on synthetic and benchmark data demonstrate that COLK exhibits consistent performance across training runs, estimates that are both low bias and low variance, and thus marks a step towards overcoming overfitting.
Raduemil Precup - One of the best experts on this subject based on the ideXlab platform.
-
Adaptive Charged System Search Approach to Path Planning for Multiple Mobile Robots
IFAC-PapersOnLine, 2015Co-Authors: Raduemil Precup, Emil M Petriu, Mircea-bogdan Radae, Emil-ioan Voisan, Florin DraganAbstract:Abstract This paper suggests the application of adaptive Charged System Search (CSS) algorithms to the optimal path planning (PP) of multiple mobile robots. An off-line adaptive CSS-based PP approach is proposed and applied to holonomic wheeled platforms in static environments. The adaptive CSS algorithms solve the optimisation problems that aim the minimisation of objective functions (o.f.s) specific to PP and expressed as the weighted sum of four functions that target separate PP objectives. A penalty term is added in certain situations in the first step of the PP approach. The specific features of the adaptive CSS algorithms are the adaptation of the acceleration, velocity, and separation distance parameters to the Iteration Index, and the substitution of the worst charged particles’ fitness function values and positions with the best performing particle data. The fitness function in the adaptive CSS algorithms corresponds to the o.f., and the search space and agents (charged particles) in the adaptive CSS algorithms correspond to the solution space and to the mobile robots, respectively. A case study and experiments are included validate the new adaptive CSS-based PP approach and to compare it with non- adaptive CSS-, Particle Swarm Optimization- and Gravitational Search Algorithm-based PP approaches.
-
novel adaptive charged system search algorithm for optimal tuning of fuzzy controllers
Expert Systems With Applications, 2014Co-Authors: Raduemil Precup, Emil M Petriu, Stefan Preitl, Raducodru David, Mirceabogdan RdacAbstract:This paper proposes a novel Adaptive Charged System Search (ACSS) algorithm for the optimal tuning of Takagi-Sugeno proportional-integral fuzzy controllers (T-S PI-FCs). The five stages of this algorithm, namely the engagement, exploration, explanation, elaboration and evaluation, involve the adaptation of the acceleration, velocity, and separation distance parameters to the Iteration Index, and the substitution of the worst charged particles' fitness function values and positions with the best performing particle data. The ACSS algorithm solves the optimization problems aiming to minimize the objective functions expressed as the sum of absolute control error plus squared output sensitivity function, resulting in optimal fuzzy control systems with reduced parametric sensitivity. The ACSS-based tuning of T-S PI-FCs is applied to second-order servo systems with an integral component. The ACSS algorithm is validated by an experimental case study dealing with the optimal tuning of a T-S PI-FC for the position control of a nonlinear servo system.
-
fuzzy logic based adaptive gravitational search algorithm for optimal tuning of fuzzy controlled servo systems
Iet Control Theory and Applications, 2013Co-Authors: Raduemil Precup, Raducodrut David, Emil M Petriu, Stefan Preitl, Mirceabogdan RadacAbstract:This study proposes an adaptive gravitational search algorithm (AGSA) which carries out adaptation of depreciation law of the gravitational constant and of a parameter in the weighted sum of all forces exerted from the other agents to the Iteration Index. The adaptation is ensured by a simple single input–two output (SITO) fuzzy block in the algorithm's structure. SITO fuzzy block operates in the Iteration domain, the Iteration Index is the input variable and the gravitational constant and the parameter in the weighted sum of all forces are the output variables. AGSA's convergence is guaranteed by a theorem derived from Popov's hyperstability analysis results. AGSA is embedded in an original design and tuning method for Takagi-Sugeno proportional-integral fuzzy controllers (T-S PI-FCs) dedicated to servo systems modelled by second-order models with an integral component and variable parameters. AGSA solves a minimisation-type optimisation problem based on an objective function which depends on the sensitivity function with respect to process gain variations, therefore a reduced process gain sensitivity is offered. AGSA is validated by a case study that optimally tunes a T-S PI-FC for position control of a laboratory servo system.Representative experimental results are presented.
-
Gravitational Search Algorithms in Fuzzy Control Systems Tuning
IFAC Proceedings Volumes, 2011Co-Authors: Raduemil Precup, Emil M Petriu, Stefan Preitl, Radu-codruţ David, Mirceabogdan RadacAbstract:Abstract This paper suggests the use of Gravitational Search Algorithms (GSAs) in fuzzy control systems tuning. New GSAs are first offered on the basis of the modification of the depreciation equation of the gravitational constant with the Iteration Index and of an additional constraint regarding system's overshoot. The GSAs are next used in solving the optimization problems which minimize the discrete-time objective functions defined as the weighted sum of the squared control error and of the squared output sensitivity functions. The sensitivity functions are derived from the sensitivity models defined with respect to the parametric variations of the controlled plant such that to aim the parametric sensitivity reduction. The presentation focuses the representative case of Takagi-Sugeno PI-fuzzy controllers (PI-FCs) that controls a class of servo systems characterized by second-order linearized models with integral component. Discussions concerning the tuning of the PI-FC parameters in a case study are included.
Alejandro Ribeiro - One of the best experts on this subject based on the ideXlab platform.
-
Nonparametric Stochastic Compositional Gradient Descent for Q-Learning in Continuous Markov Decision Problems
arXiv: Learning, 2018Co-Authors: Alec Koppel, Ethan Stump, Ekaterina Tolstaya, Alejandro RibeiroAbstract:We consider Markov Decision Problems defined over continuous state and action spaces, where an autonomous agent seeks to learn a map from its states to actions so as to maximize its long-term discounted accumulation of rewards. We address this problem by considering Bellman's optimality equation defined over action-value functions, which we reformulate into a nested non-convex stochastic optimization problem defined over a Reproducing Kernel Hilbert Space (RKHS). We develop a functional generalization of stochastic quasi-gradient method to solve it, which, owing to the structure of the RKHS, admits a parameterization in terms of scalar weights and past state-action pairs which grows proportionately with the algorithm Iteration Index. To ameliorate this complexity explosion, we apply Kernel Orthogonal Matching Pursuit to the sequence of kernel weights and dictionaries, which yields a controllable error in the descent direction of the underlying optimization method. We prove that the resulting algorithm, called KQ-Learning, converges with probability 1 to a stationary point of this problem, yielding a fixed point of the Bellman optimality operator under the hypothesis that it belongs to the RKHS. Under constant learning rates, we further obtain convergence to a small Bellman error that depends on the chosen learning rates. Numerical evaluation on the Continuous Mountain Car and Inverted Pendulum tasks yields convergent parsimonious learned action-value functions, policies that are competitive with the state of the art, and exhibit reliable, reproducible learning behavior.
-
ACC - Nonparametric Stochastic Compositional Gradient Descent for Q-Learning in Continuous Markov Decision Problems
2018 Annual American Control Conference (ACC), 2018Co-Authors: Ekaterina Tolstaya, Alec Koppel, Ethan Stump, Alejandro RibeiroAbstract:We consider Markov Decision Problems defined over continuous state and action spaces, where an autonomous agent seeks to learn a map from its states to actions so as to maximize its long-term discounted accumulation of rewards. We address this problem by considering Bellman's optimality equation defined over action-value functions, which we reformulate into a nested non-convex stochastic optimization problem defined over a Reproducing Kernel Hilbert Space (RKHS). We develop a functional generalization of stochastic quasi-gradient method to solve it, which, owing to the structure of the RKHS, admits a parameterization in terms of scalar weights and past state-action pairs which grows proportionately with the algorithm Iteration Index. To ameliorate this complexity explosion, we apply Kernel Orthogonal Matching Pursuit to the sequence of kernel weights and dictionaries, which yields a controllable error in the descent direction of the underlying optimization method. We prove that the resulting algorithm, called KQ Learning, converges with probability 1 to a stationary point of this problem, yielding a fixed point of the Bellman optimality operator under the hypothesis that it belongs to the RKHS. Numerical evaluation on the continuous Mountain Car task yields convergent parsimonious learned action-value functions and policies that are competitive with the state of the art.
-
ICASSP - Parsimonious Online Learning with Kernels via sparse projections in function space
2017 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP), 2017Co-Authors: Alec Koppel, Garrett Warnell, Ethan Stump, Alejandro RibeiroAbstract:We consider stochastic nonparametric regression problems in a reproducing kernel Hilbert space (RKHS), an extension of expected risk minimization to nonlinear function estimation. Popular perception is that kernel methods are inapplicable to online settings, since the generalization of stochastic methods to kernelized function spaces require memory storage that is cubic in the Iteration Index (“the curse of kernelization”). We alleviate this intractability in two ways: (1) we consider the use of functional stochastic gradient method (FSGD) which operates on a subset of training examples at each step; and (2), we extract parsimonious approximations of the resulting stochastic sequence via a greedy sparse subspace projection scheme based on kernel orthogonal matching pursuit (KOMP). We establish that this method converges almost surely in both diminishing and constant algorithm step-size regimes for a specific selection of sparse approximation budget. The method is evaluated on a kernel multi-class support vector machine problem, where data samples are generated from class-dependent Gaussian mixture models.
-
GlobalSIP - Decentralized online optimization with heterogeneous data sources
2016 IEEE Global Conference on Signal and Information Processing (GlobalSIP), 2016Co-Authors: Alec Koppel, Brian M. Sadler, Alejandro RibeiroAbstract:We consider stochastic optimization problems in decentralized settings, where a network of agents aims to learn decision variables which are optimal in terms of a global objective which depends on possibly heterogeneous streaming observations received at each node. Consensus optimization techniques implicitly operate on the hypothesis that each node aims to learn a common parameter vector, which is inappropriate for this context. Motivated by this observation, we formulate a problem where each agent minimizes a global objective while enforcing network proximity constraints that may encode correlation structures among the observations at each node. To solve this problem, we propose a decentralized stochastic saddle point algorithm inspired by Arrow and Hurwicz. We establish that under a constant step-size regime the time-average suboptimality and constraint violation are contained in a neighborhood whose radius vanishes with the Iteration Index. Further, the time-average primal vectors converge to the optimal objective while satisfying the network proximity constraints. We apply this method to an online source localization problem and show it outperforms consensus-based schemes.
Tamer Basar - One of the best experts on this subject based on the ideXlab platform.
-
Projected Stochastic Primal-Dual Method for Constrained Online Learning With Kernels
IEEE Transactions on Signal Processing, 2019Co-Authors: Alec Koppel, Kaiqing Zhang, Hao Zhu, Tamer BasarAbstract:We consider the problem of stochastic optimization with nonlinear constraints, where the decision variable is not vector-valued but instead a function belonging to a reproducing Kernel Hilbert Space (RKHS). Currently, there exist solutions to only special cases of this problem. To solve this constrained problem with kernels, we first generalize the Representer Theorem to a class of saddle-point problems defined over RKHS. Then, we develop a primal-dual method which that executes alternating projected primal/dual stochastic gradient descent/ascent on the dual-augmented Lagrangian of the problem. The primal projection sets are low-dimensional subspaces of the ambient function space, which are greedily constructed using matching pursuit. By tuning the projection-induced error to the algorithm step-size, we are able to establish mean convergence in both primal objective sub-optimality and constraint violation, to respective ${\mathcal O}(\sqrt{T})$ and ${\mathcal O}(T^{3/4})$ neighborhoods. Here, $T$ is the final Iteration Index and the constant step-size is chosen as $1/\sqrt{T}$ with $1/T$ approximation budget. Finally, we demonstrate experimentally the effectiveness of the proposed method for risk-aware supervised learning.
Emil M Petriu - One of the best experts on this subject based on the ideXlab platform.
-
Adaptive Charged System Search Approach to Path Planning for Multiple Mobile Robots
IFAC-PapersOnLine, 2015Co-Authors: Raduemil Precup, Emil M Petriu, Mircea-bogdan Radae, Emil-ioan Voisan, Florin DraganAbstract:Abstract This paper suggests the application of adaptive Charged System Search (CSS) algorithms to the optimal path planning (PP) of multiple mobile robots. An off-line adaptive CSS-based PP approach is proposed and applied to holonomic wheeled platforms in static environments. The adaptive CSS algorithms solve the optimisation problems that aim the minimisation of objective functions (o.f.s) specific to PP and expressed as the weighted sum of four functions that target separate PP objectives. A penalty term is added in certain situations in the first step of the PP approach. The specific features of the adaptive CSS algorithms are the adaptation of the acceleration, velocity, and separation distance parameters to the Iteration Index, and the substitution of the worst charged particles’ fitness function values and positions with the best performing particle data. The fitness function in the adaptive CSS algorithms corresponds to the o.f., and the search space and agents (charged particles) in the adaptive CSS algorithms correspond to the solution space and to the mobile robots, respectively. A case study and experiments are included validate the new adaptive CSS-based PP approach and to compare it with non- adaptive CSS-, Particle Swarm Optimization- and Gravitational Search Algorithm-based PP approaches.
-
novel adaptive charged system search algorithm for optimal tuning of fuzzy controllers
Expert Systems With Applications, 2014Co-Authors: Raduemil Precup, Emil M Petriu, Stefan Preitl, Raducodru David, Mirceabogdan RdacAbstract:This paper proposes a novel Adaptive Charged System Search (ACSS) algorithm for the optimal tuning of Takagi-Sugeno proportional-integral fuzzy controllers (T-S PI-FCs). The five stages of this algorithm, namely the engagement, exploration, explanation, elaboration and evaluation, involve the adaptation of the acceleration, velocity, and separation distance parameters to the Iteration Index, and the substitution of the worst charged particles' fitness function values and positions with the best performing particle data. The ACSS algorithm solves the optimization problems aiming to minimize the objective functions expressed as the sum of absolute control error plus squared output sensitivity function, resulting in optimal fuzzy control systems with reduced parametric sensitivity. The ACSS-based tuning of T-S PI-FCs is applied to second-order servo systems with an integral component. The ACSS algorithm is validated by an experimental case study dealing with the optimal tuning of a T-S PI-FC for the position control of a nonlinear servo system.
-
fuzzy logic based adaptive gravitational search algorithm for optimal tuning of fuzzy controlled servo systems
Iet Control Theory and Applications, 2013Co-Authors: Raduemil Precup, Raducodrut David, Emil M Petriu, Stefan Preitl, Mirceabogdan RadacAbstract:This study proposes an adaptive gravitational search algorithm (AGSA) which carries out adaptation of depreciation law of the gravitational constant and of a parameter in the weighted sum of all forces exerted from the other agents to the Iteration Index. The adaptation is ensured by a simple single input–two output (SITO) fuzzy block in the algorithm's structure. SITO fuzzy block operates in the Iteration domain, the Iteration Index is the input variable and the gravitational constant and the parameter in the weighted sum of all forces are the output variables. AGSA's convergence is guaranteed by a theorem derived from Popov's hyperstability analysis results. AGSA is embedded in an original design and tuning method for Takagi-Sugeno proportional-integral fuzzy controllers (T-S PI-FCs) dedicated to servo systems modelled by second-order models with an integral component and variable parameters. AGSA solves a minimisation-type optimisation problem based on an objective function which depends on the sensitivity function with respect to process gain variations, therefore a reduced process gain sensitivity is offered. AGSA is validated by a case study that optimally tunes a T-S PI-FC for position control of a laboratory servo system.Representative experimental results are presented.
-
Gravitational Search Algorithms in Fuzzy Control Systems Tuning
IFAC Proceedings Volumes, 2011Co-Authors: Raduemil Precup, Emil M Petriu, Stefan Preitl, Radu-codruţ David, Mirceabogdan RadacAbstract:Abstract This paper suggests the use of Gravitational Search Algorithms (GSAs) in fuzzy control systems tuning. New GSAs are first offered on the basis of the modification of the depreciation equation of the gravitational constant with the Iteration Index and of an additional constraint regarding system's overshoot. The GSAs are next used in solving the optimization problems which minimize the discrete-time objective functions defined as the weighted sum of the squared control error and of the squared output sensitivity functions. The sensitivity functions are derived from the sensitivity models defined with respect to the parametric variations of the controlled plant such that to aim the parametric sensitivity reduction. The presentation focuses the representative case of Takagi-Sugeno PI-fuzzy controllers (PI-FCs) that controls a class of servo systems characterized by second-order linearized models with integral component. Discussions concerning the tuning of the PI-FC parameters in a case study are included.
