The Experts below are selected from a list of 23046 Experts worldwide ranked by ideXlab platform
M. Vidyasagar - One of the best experts on this subject based on the ideXlab platform.
-
randomized algorithms for robust controller synthesis using Statistical Learning Theory
Automatica, 2001Co-Authors: M. VidyasagarAbstract:By now it is known that several problems in the robustness analysis and synthesis of control systems are NP-complete or NP-hard. These negative results force us to modify our notion of ''solving'' a given problem. An approach that is recently gaining popularity is that of using randomized algorithms, which can be used to solve a problem approximately, most of the time. We begin with the premise that many problems in robustness analysis and synthesis can be formulated as the minimization of an objective function with respect to the controller parameters. It is argued that, in order to assess the performance of a controller as the plant varies over a prespecified family, it is better to use the average performance of the controller as the objective function to be minimized, rather than its worst-case performance, as the worst-case objective function usually leads to rather conservative designs. Then it is shown that a property from Statistical Learning Theory known as uniform convergence of empirical means (UCEM) plays an important role in allowing us to construct efficient randomized algorithms for a wide variety of controller synthesis problems. In particular, whenever the UCEM property holds, there exists an efficient (i.e., polynomial-time) randomized algorithm. Using very recent results in Statistical Learning Theory, it is shown that the UCEM property holds in any problem in which the satisfaction of a performance constraint can be expressed in terms of a finite number of polynomial inequalities. In particular, several problems such as robust stabilization and weighted H"2/H"~-norm minimization are amenable to the randomized approach.
-
Randomized Algorithms for Robust Controller Synthesis Using Statistical Learning Theory: A Tutorial Overview
European Journal of Control, 2001Co-Authors: M. VidyasagarAbstract:By now it is well known that several problems in the robustness analysis and synthesis of control systems are NP-complete or NP-hard. These negative results force us to modify our notion of “solving”a given problem. If we cannot solve a problem exactly because it is NP-hard, then we must settle for solving it approximately. If we cannot solve all instances of a problem, we must settle for solving “almost all” instances of a problem. An approach that is recently gaining popularity is that of using randomized algorithms. The notion of a randomized algorithm as defined here is somewhat different from that in the computer science literature, and enlarges the class of problems that can be efficiently solved. We begin with the premise that many problems in robustness analysis and synthesis can be formulated as the minimization of an objective function with respect to the controller parameters. It is argued that, in order to assess the performance of a controller as the plant varies over a prespecified family, it is better to use the average performance of the controller as the objective function to be minimized, rather than its worst-case performance, as the worstcase objective function usually leads to rather conservative designs. Then it is shown that a property from Statistical Learning Theory known as uniform convergence of empirical means (UCEM) plays an important role in allowing us to construct efficient randomized algorithms for a wide variety of controller synthesis problems. In particular, whenever the UCEM property holds, there exists an efficient (i.e., polynomial-time) randomized algorithm. Using very recent results in Statistical Learning Theory, it is shown that the UCEM property holds in several problems such as robust stabilization and weighted H 2 /H ∞ -norm minimization. Hence it is possible to solve such problems efficiently using randomized algorithms.
-
Statistical Learning Theory and Randomized Algorithms for Control
IEEE Control Systems, 1998Co-Authors: M. VidyasagarAbstract:The topic of the present article is the use of randomized algorithms to solve some problems in control system designs that are perceived to be “difficult”. A brief introduction is given to the notions of computational complexity that are pertinent to the present discussion, and then some problems in control system analysis and synthesis that are difficult in a complexity-theoretic sense are described. Some of the elements of Statistical Learning Theory, which forms the basis of the randomized approach, are briefly described. Finally, these two sets of ideas are brought together to show that it is possible to construct efficient randomized algorithms for each of the difficult problems discussed by using the ideas of Statistical Learning Theory. A real-life design example of synthesizing a first-order controller for the longitudinal stabilization of an unstable fighter aircraft is then presented to show that the randomized approach can be quite successful in tackling a practical problem
E F Camacho - One of the best experts on this subject based on the ideXlab platform.
-
revisiting Statistical Learning Theory for uncertain feasibility and optimization problems
Conference on Decision and Control, 2007Co-Authors: T Alamo, R Tempo, E F CamachoAbstract:In this paper, we study two general semi-infinite programming problems by means of Statistical Learning Theory. The sample size results obtained with this approach are generally considered to be very conservative by the control community. The main contribution of this paper is to demonstrate that this is not necessarily the case. Using as a starting point one-side results from Statistical Learning Theory, we obtain bounds on the number of required samples that are manageable for "reasonable" values of confidence delta and accuracy isin. In particular, we provide sample size bounds growing with 1/isin ln 1/isin instead of the usual 1/isin2 ln 1/isin2 dependence.
-
CDC - Revisiting Statistical Learning Theory for uncertain feasibility and optimization problems
2007 46th IEEE Conference on Decision and Control, 2007Co-Authors: T Alamo, R Tempo, E F CamachoAbstract:In this paper, we study two general semi-infinite programming problems by means of Statistical Learning Theory. The sample size results obtained with this approach are generally considered to be very conservative by the control community. The main contribution of this paper is to demonstrate that this is not necessarily the case. Using as a starting point one-side results from Statistical Learning Theory, we obtain bounds on the number of required samples that are manageable for "reasonable" values of confidence delta and accuracy isin. In particular, we provide sample size bounds growing with 1/isin ln 1/isin instead of the usual 1/isin2 ln 1/isin2 dependence.
Chaouki T Abdallah - One of the best experts on this subject based on the ideXlab platform.
-
Statistical Learning Theory to evaluate the performance of game theoretic power control algorithms for wireless data in arbitrary channels
Wireless Communications and Networking Conference, 2003Co-Authors: Mohammad Hayajneh, Chaouki T AbdallahAbstract:In this paper we use Statistical Learning Theory to evaluate the performance of game theoretic power control algorithms for wireless data in arbitrary channels, i.e., no presumed channel model is required. To show the validity of Statistical Learning Theory in this context, we studied a flat fading channel, and more specifically, we simulated the case of Rayleigh flat fading channel. With the help of a relatively small number of training samples, the results suggest the learnability of the utility function classes defined by changing the user power (adjusted parameter) for each user's utility function.
-
WCNC - Statistical Learning Theory to evaluate the performance of game theoretic power control algorithms for wireless data in arbitrary channels
2003 IEEE Wireless Communications and Networking 2003. WCNC 2003., 1Co-Authors: Mohammad Hayajneh, Chaouki T AbdallahAbstract:In this paper we use Statistical Learning Theory to evaluate the performance of game theoretic power control algorithms for wireless data in arbitrary channels, i.e., no presumed channel model is required. To show the validity of Statistical Learning Theory in this context, we studied a flat fading channel, and more specifically, we simulated the case of Rayleigh flat fading channel. With the help of a relatively small number of training samples, the results suggest the learnability of the utility function classes defined by changing the user power (adjusted parameter) for each user's utility function.
Gilbert Harman - One of the best experts on this subject based on the ideXlab platform.
-
an elementary introduction to Statistical Learning Theory
2011Co-Authors: Sanjeev R Kulkarni, Gilbert HarmanAbstract:A thought-provoking look at Statistical Learning Theory and its role in understanding human Learning and inductive reasoningA joint endeavor from leading researchers in the fields of philosophy and electrical engineering, An Elementary Introduction to Statistical Learning Theory is a comprehensive and accessible primer on the rapidly evolving fields of Statistical pattern recognition and Statistical Learning Theory. Explaining these areas at a level and in a way that is not often found in other books on the topic, the authors present the basic Theory behind contemporary machine Learning and uniquely utilize its foundations as a framework for philosophical thinking about inductive inference.Promoting the fundamental goal of Statistical Learning, knowing what is achievable and what is not, this book demonstrates the value of a systematic methodology when used along with the needed techniques for evaluating the performance of a Learning system. First, an introduction to machine Learning is presented that includes brief discussions of applications such as image recognition, speech recognition, medical diagnostics, and Statistical arbitrage. To enhance accessibility, two chapters on relevant aspects of probability Theory are provided. Subsequent chapters feature coverage of topics such as the pattern recognition problem, optimal Bayes decision rule, the nearest neighbor rule, kernel rules, neural networks, support vector machines, and boosting.Appendices throughout the book explore the relationship between the discussed material and related topics from mathematics, philosophy, psychology, and statistics, drawing insightful connections between problems in these areas and Statistical Learning Theory. All chapters conclude with a summary section, a set of practice questions, and a reference sections that supplies historical notes and additional resources for further study.An Elementary Introduction to Statistical Learning Theory is an excellent book for courses on Statistical Learning Theory, pattern recognition, and machine Learning at the upper-undergraduate and graduatelevels. It also serves as an introductory reference for researchers and practitioners in the fields of engineering, computer science, philosophy, and cognitive science that would like to further their knowledge of the topic.
-
Statistical Learning Theory: a tutorial
Wiley Interdisciplinary Reviews: Computational Statistics, 2011Co-Authors: Sanjeev R Kulkarni, Gilbert HarmanAbstract:In this article, we provide a tutorial overview of some aspects of Statistical Learning Theory, which also goes by other names such as Statistical pattern recognition, nonparametric classification and estimation, and supervised Learning. We focus on the problem of two-class pattern classification for various reasons. This problem is rich enough to capture many of the interesting aspects that are present in the cases of more than two classes and in the problem of estimation, and many of the results can be extended to these cases. Focusing on two-class pattern classification simplifies our discussion, and yet it is directly applicable to a wide range of practical settings. We begin with a description of the two-class pattern recognition problem. We then discuss various classical and state-of-the-art approaches to this problem, with a focus on fundamental formulations, algorithms, and theoretical results. In particular, we describe nearest neighbor methods, kernel methods, multilayer perceptrons, Vapnik-Chervonenkis Theory, support vector machines, and boosting. WIREs Comp Stat 2011 3 543-556 DOI: 10.1002/wics.179
-
Statistical Learning Theory: at utorial
2011Co-Authors: Sanjeev R Kulkarni, Gilbert HarmanAbstract:In this article, we provide a tutorial overview of some aspects of Statistical Learning Theory, which also goes by other names such as Statistical pattern recognition, nonparametric classification and estimation, and supervised Learning. We focus on the problem of two-class pattern classification for various reasons. This problem is rich enough to capture many of the interesting aspects that are present in the cases of more than two classes and in the problem of estimation, and many of the results can be extended to these cases. Focusing on two-class pattern classification simplifies our discussion, and yet it is directly applicable to a wide range of practical settings. We begin with a description of the two-class pattern recognition problem. We then discuss various classical and state-of-the-art approaches to this problem, with a focus on fundamental formulations, algorithms, and theoretical results. In particular, we describe nearest neighbor methods, kernel methods, multilayer perceptrons, Vapnik‐Chervonenkis Theory, support vector machines,
-
Statistical Learning Theory as a Framework for the Philosophy of Induction
Philosophy of Statistics, 2011Co-Authors: Gilbert Harman, Sanjeev R KulkarniAbstract:Publisher Summary Statistical Learning Theory is the basic Theory behind contemporary machine Learning and pattern recognition. It suggests that the Theory provides an excellent framework for the philosophy of induction. There are various paradigmatic approaches to specifying the problem of induction. It assumes one has an initial known subjective probability distribution satisfying certain more or less weak conditions along with a method for updating one's probabilities, e.g. by conditionalization, and proves theorems about the results of such a method. Statistical Learning Theory represents another paradigm which assumes there is an unknown objective probability distribution that characterizes the data and the new cases about which inferences are to be made, the goal being to do as well as possible in characterizing the new cases in terms of that unknown objective probability distribution. The basic Theory attempts to specify what can be proved about various methods for using data to reach conclusions about new cases.
-
Statistical Learning Theory and Induction
2010Co-Authors: Gilbert HarmanAbstract:Statistical Learning Theory (SLT) is a mathematical Theory of a certain type of inductive reasoning involving Learning from examples. SLT makes relatively minimal assumptions about an assumed background probability distribution responsible for connections between features of examples and their correct classification, the probability that particular examples will occur, etc. The Theory seeks to describe various Learning methods and say how well they can be expected to do at producing rules with minimum expected error on new cases.
T Alamo - One of the best experts on this subject based on the ideXlab platform.
-
Statistical Learning Theory a pack based strategy for uncertain feasibility and optimization problems
Lecture Notes in Control and Information Sciences, 2008Co-Authors: T Alamo, Roberto Tempo, F EduardoAbstract:In this paper, a new powerful technique, denoted as pack-based strategy is introduced in the context of Statistical Learning Theory. This strategy allows us to derive bounds on the number of required samples that are manageable for “reasonable” values of probabilistic confidence and accuracy. Using this technique for feasibility and optimization problems involving Boolean expressions consisting of polynomials, we prove that the number of required samples grows with the accuracy parameter ∈ as 1/∈ ln 1/∈. This is a significant improvement when compared to the existing bounds which depend on 1/∈2 ln 1/∈2. We also apply this strategy to convex optimization problems. In this case, we show that the required sample size is inversely proportional to the accuracy for fixed confidence.
-
revisiting Statistical Learning Theory for uncertain feasibility and optimization problems
Conference on Decision and Control, 2007Co-Authors: T Alamo, R Tempo, E F CamachoAbstract:In this paper, we study two general semi-infinite programming problems by means of Statistical Learning Theory. The sample size results obtained with this approach are generally considered to be very conservative by the control community. The main contribution of this paper is to demonstrate that this is not necessarily the case. Using as a starting point one-side results from Statistical Learning Theory, we obtain bounds on the number of required samples that are manageable for "reasonable" values of confidence delta and accuracy isin. In particular, we provide sample size bounds growing with 1/isin ln 1/isin instead of the usual 1/isin2 ln 1/isin2 dependence.
-
CDC - Revisiting Statistical Learning Theory for uncertain feasibility and optimization problems
2007 46th IEEE Conference on Decision and Control, 2007Co-Authors: T Alamo, R Tempo, E F CamachoAbstract:In this paper, we study two general semi-infinite programming problems by means of Statistical Learning Theory. The sample size results obtained with this approach are generally considered to be very conservative by the control community. The main contribution of this paper is to demonstrate that this is not necessarily the case. Using as a starting point one-side results from Statistical Learning Theory, we obtain bounds on the number of required samples that are manageable for "reasonable" values of confidence delta and accuracy isin. In particular, we provide sample size bounds growing with 1/isin ln 1/isin instead of the usual 1/isin2 ln 1/isin2 dependence.