The Experts below are selected from a list of 15342 Experts worldwide ranked by ideXlab platform
John Dunagan - One of the best experts on this subject based on the ideXlab platform.
-
a simple polynomial time rescaling Algorithm for solving linear programs
Mathematical Programming, 2008Co-Authors: John Dunagan, Santosh Vempala Aff NAbstract:The Perceptron Algorithm, developed mainly in the machine learning literature, is a simple greedy method for finding a feasible solution to a linear program (alternatively, for learning a threshold function). In spite of its exponential worst-case complexity, it is often quite useful, in part due to its noise-tolerance and also its overall simplicity. In this paper, we show that a randomized version of the Perceptron Algorithm along with periodic rescaling runs in polynomial-time. The resulting Algorithm for linear programming has an elementary description and analysis.
-
STOC - A simple polynomial-time rescaling Algorithm for solving linear programs
Proceedings of the thirty-sixth annual ACM symposium on Theory of computing - STOC '04, 2004Co-Authors: John Dunagan, Santosh VempalaAbstract:The Perceptron Algorithm, developed mainly in the machine learning literature, is a simple greedy method for finding a feasible solution to a linear program (alternatively, for learning a threshold function. ). In spite of its exponential worst-case complexity, it is often quite useful, in part due to its noise-tolerance and also its overall simplicity. In this paper, we show that a randomized version of the Perceptron Algorithm with periodic rescaling runs in polynomial-time. The resulting Algorithm for linear programming has an elementary description and analysis.
-
smoothed analysis of the Perceptron Algorithm for linear programming
Symposium on Discrete Algorithms, 2002Co-Authors: Avrim Blum, John DunaganAbstract:The smoothed complexity [1] of an Algorithm is the expected running time of the Algorithm on an arbitrary instance under a random perturbation. It was shown recently that the simplex Algorithm has polynomial smoothed complexity. We show that a simple greedy Algorithm for linear programming, the Perceptron Algorithm, also has polynomial smoothed complexity, in a high probability sense; that is, the running time is polynomial with high probability over the random perturbation.
-
SODA - Smoothed analysis of the Perceptron Algorithm for linear programming
2002Co-Authors: Avrim Blum, John DunaganAbstract:The smoothed complexity [1] of an Algorithm is the expected running time of the Algorithm on an arbitrary instance under a random perturbation. It was shown recently that the simplex Algorithm has polynomial smoothed complexity. We show that a simple greedy Algorithm for linear programming, the Perceptron Algorithm, also has polynomial smoothed complexity, in a high probability sense; that is, the running time is polynomial with high probability over the random perturbation.
Murat Saraclar - One of the best experts on this subject based on the ideXlab platform.
-
CICLing - Morphological Disambiguation of Turkish Text with Perceptron Algorithm
Computational Linguistics and Intelligent Text Processing, 2007Co-Authors: Hasim Sak, Tunga Güngör, Murat SaraclarAbstract:This paper describes the application of the Perceptron Algorithm to the morphological disambiguation of Turkish text. Turkish has a productive derivational morphology. Due to the ambiguity caused by complex morphology, a word may have multiple morphological parses, each with a different stem or sequence of morphemes. The methodology employed is based on ranking with Perceptron Algorithm which has been successful in some NLP tasks in English. We use a baseline statistical trigram-based model of a previous work to enumerate an n-best list of candidate morphological parse sequences for each sentence. We then apply the Perceptron Algorithm to rerank the n-best list using a set of 23 features. The Perceptron trained to do morphological disambiguation improves the accuracy of the baseline model from 93.61% to 96.80%. When we train the Perceptron as a POS tagger, the accuracy is 98.27%. Turkish morphological disambiguation and POS tagging results that we obtained is the best reported so far.
-
Discriminative n-gram language modeling
Computer Speech & Language, 2007Co-Authors: Brian Roark, Murat Saraclar, Michael CollinsAbstract:This paper describes discriminative language modeling for a large vocabulary speech recognition task. We contrast two parameter estimation methods: the Perceptron Algorithm, and a method based on maximizing the regularized conditional log-likelihood. The models are encoded as deterministic weighted finite state automata, and are applied by intersecting the automata with word-lattices that are the output from a baseline recognizer. The Perceptron Algorithm has the benefit of automatically selecting a relatively small feature set in just a couple of passes over the training data. We describe a method based on regularized likelihood that makes use of the feature set given by the Perceptron Algorithm, and initialization with the Perceptron's weights; this method gives an additional 0.5% reduction in word error rate (WER) over training with the Perceptron alone. The final system achieves a 1.8% absolute reduction in WER for a baseline first-pass recognition system (from 39.2% to 37.4%), and a 0.9% absolute reduction in WER for a multi-pass recognition system (from 28.9% to 28.0%).
-
discriminative language modeling with conditional random fields and the Perceptron Algorithm
Meeting of the Association for Computational Linguistics, 2004Co-Authors: Brian Roark, Michael Collins, Murat Saraclar, Mark JohnsonAbstract:This paper describes discriminative language modeling for a large vocabulary speech recognition task. We contrast two parameter estimation methods: the Perceptron Algorithm, and a method based on conditional random fields (CRFs). The models are encoded as deterministic weighted finite state automata, and are applied by intersecting the automata with word-lattices that are the output from a baseline recognizer. The Perceptron Algorithm has the benefit of automatically selecting a relatively small feature set in just a couple of passes over the training data. However, using the feature set output from the Perceptron Algorithm (initialized with their weights), CRF training provides an additional 0.5% reduction in word error rate, for a total 1.8% absolute reduction from the baseline of 39.2%.
-
corrective language modeling for large vocabulary asr with the Perceptron Algorithm
International Conference on Acoustics Speech and Signal Processing, 2004Co-Authors: Brian Roark, Murat Saraclar, Michael CollinsAbstract:This paper investigates error-corrective language modeling using the Perceptron Algorithm on word lattices. The resulting model is encoded as a weighted finite-state automaton, and is used by intersecting the model with word lattices, making it simple and inexpensive to apply during decoding. We present results for various training scenarios for the Switchboard task, including using n-gram features of different orders, and performing n-best extraction versus using full word lattices. We demonstrate the importance of making the training conditions as close as possible to testing conditions. The best approach yields a 1.3 percent improvement in first pass accuracy, which translates to 0.5 percent improvement after other rescoring passes.
-
language model adaptation with map estimation and the Perceptron Algorithm
North American Chapter of the Association for Computational Linguistics, 2004Co-Authors: Michiel Bacchiani, Brian Roark, Murat SaraclarAbstract:In this paper, we contrast two language model adaptation approaches: MAP estimation and the Perceptron Algorithm. Used in isolation, we show that MAP estimation outperforms the latter approach, for reasons which argue for combining the two approaches. When combined, the resulting system provides a 0.7 percent absolute reduction in word error rate over MAP estimation alone. In addition, we demonstrate that, in a multi-pass recognition scenario, it is better to use the Perceptron Algorithm on early pass word lattices, since the improved error rate improves acoustic model adaptation.
Santosh Vempala - One of the best experts on this subject based on the ideXlab platform.
-
An Efficient Rescaled Perceptron Algorithm for Conic Systems
Mathematics of Operations Research, 2009Co-Authors: Alexandre Belloni, Robert M. Freund, Santosh VempalaAbstract:The classical Perceptron Algorithm is an elementary row-action/relaxation Algorithm for solving a homogeneous linear inequality system Ax > 0. A natural condition measure associated with this Algorithm is the Euclidean width τ of the cone of feasible solutions, and the iteration complexity of the Perceptron Algorithm is bounded by 1/τ2 [see Rosenblatt, F. 1962. Principles of Neurodynamics. Spartan Books, Washington, DC]. Dunagan and Vempala [Dunagan, J., S. Vempala. 2007. A simple polynomial-time rescaling Algorithm for solving linear programs. Math. Programming114(1) 101--114] have developed a rescaled version of the Perceptron Algorithm with an improved complexity of O(n ln (1/τ)) iterations (with high probability), which is theoretically efficient in τ and, in particular, is polynomial time in the bit-length model. We explore extensions of the concepts of these Perceptron methods to the general homogeneous conic system Ax ∈ intK, where K is a regular convex cone. We provide a conic extension of the rescaled Perceptron Algorithm based on the notion of a deep-separation oracle of a cone, which essentially computes a certificate of strong separation. We show that the rescaled Perceptron Algorithm is theoretically efficient if an efficient deep-separation oracle is available for the feasible region. Furthermore, when K is the cross-product of basic cones that are either half-spaces or second-order cones, then a deep-separation oracle is available and, hence, the rescaled Perceptron Algorithm is theoretically efficient. When the basic cones of K include semidefinite cones, then a probabilistic deep-separation oracle for K can be constructed that also yields a theoretically efficient version of the rescaled Perceptron Algorithm.
-
an efficient re scaled Perceptron Algorithm for conic systems
Conference on Learning Theory, 2007Co-Authors: Alexandre Belloni, Robert M. Freund, Santosh VempalaAbstract:The classical Perceptron Algorithm is an elementary Algorithm for solving a homogeneous linear inequality system Ax > 0, with many important applications in learning theory (e.g., [11,8]). A natural condition measure associated with this Algorithm is the Euclidean width τ of the cone of feasible solutions, and the iteration complexity of the Perceptron Algorithm is bounded by 1/τ2. Dunagan and Vempala [5] have developed a re-scaled version of the Perceptron Algorithm with an improved complexity of O(n ln(1/τ)) iterations (with high probability), which is theoretically efficient in τ, and in particular is polynomial-time in the bit-length model. We explore extensions of the concepts of these Perceptron methods to the general homogeneous conic system Ax ∈ int K where K is a regular convex cone. We provide a conic extension of the re-scaled Perceptron Algorithm based on the notion of a deep-separation oracle of a cone, which essentially computes a certificate of strong separation. We give a general condition under which the re-scaled Perceptron Algorithm is theoretically efficient, i.e., polynomial-time; this includes the cases when K is the cross-product of half-spaces, second-order cones, and the positive semi-definite cone.
-
STOC - A simple polynomial-time rescaling Algorithm for solving linear programs
Proceedings of the thirty-sixth annual ACM symposium on Theory of computing - STOC '04, 2004Co-Authors: John Dunagan, Santosh VempalaAbstract:The Perceptron Algorithm, developed mainly in the machine learning literature, is a simple greedy method for finding a feasible solution to a linear program (alternatively, for learning a threshold function. ). In spite of its exponential worst-case complexity, it is often quite useful, in part due to its noise-tolerance and also its overall simplicity. In this paper, we show that a randomized version of the Perceptron Algorithm with periodic rescaling runs in polynomial-time. The resulting Algorithm for linear programming has an elementary description and analysis.
-
COLT - An efficient re-scaled Perceptron Algorithm for conic systems
Learning Theory, 1Co-Authors: Alexandre Belloni, Robert M. Freund, Santosh VempalaAbstract:The classical Perceptron Algorithm is an elementary Algorithm for solving a homogeneous linear inequality system Ax > 0, with many important applications in learning theory (e.g., [11,8]). A natural condition measure associated with this Algorithm is the Euclidean width τ of the cone of feasible solutions, and the iteration complexity of the Perceptron Algorithm is bounded by 1/τ2. Dunagan and Vempala [5] have developed a re-scaled version of the Perceptron Algorithm with an improved complexity of O(n ln(1/τ)) iterations (with high probability), which is theoretically efficient in τ, and in particular is polynomial-time in the bit-length model. We explore extensions of the concepts of these Perceptron methods to the general homogeneous conic system Ax ∈ int K where K is a regular convex cone. We provide a conic extension of the re-scaled Perceptron Algorithm based on the notion of a deep-separation oracle of a cone, which essentially computes a certificate of strong separation. We give a general condition under which the re-scaled Perceptron Algorithm is theoretically efficient, i.e., polynomial-time; this includes the cases when K is the cross-product of half-spaces, second-order cones, and the positive semi-definite cone.
C Collins - One of the best experts on this subject based on the ideXlab platform.
-
trigger based language modeling using a loss sensitive Perceptron Algorithm
International Conference on Acoustics Speech and Signal Processing, 2007Co-Authors: N Singhmiller, C CollinsAbstract:Discriminative language models using n-gram features have been shown to be effective in reducing speech recognition word error rates. In this paper we describe a method for incorporating discourse-level triggers into a discriminative language model. Triggers are features identifying re-occurrence of words within a conversation. We introduce triggers that are specific to particular unigrams and bigrams, as well as "back off" trigger features that allow generalizations to be made across different unigrams. We train our model using a new loss-sensitive variant of the Perceptron Algorithm that makes effective use of information from multiple hypotheses in an n-best list. We train and test on the switchboard data set and show a 0.5 absolute reduction in WER over a baseline discriminative model which uses n-gram features alone, and a 1.5 absolute reduction in WER over the baseline recognizer.
-
ICASSP (4) - Trigger-Based Language Modeling using a Loss-Sensitive Perceptron Algorithm
2007 IEEE International Conference on Acoustics Speech and Signal Processing - ICASSP '07, 2007Co-Authors: N. Singh-miller, C CollinsAbstract:Discriminative language models using n-gram features have been shown to be effective in reducing speech recognition word error rates. In this paper we describe a method for incorporating discourse-level triggers into a discriminative language model. Triggers are features identifying re-occurrence of words within a conversation. We introduce triggers that are specific to particular unigrams and bigrams, as well as "back off" trigger features that allow generalizations to be made across different unigrams. We train our model using a new loss-sensitive variant of the Perceptron Algorithm that makes effective use of information from multiple hypotheses in an n-best list. We train and test on the switchboard data set and show a 0.5 absolute reduction in WER over a baseline discriminative model which uses n-gram features alone, and a 1.5 absolute reduction in WER over the baseline recognizer.
Michael Collins - One of the best experts on this subject based on the ideXlab platform.
-
Discriminative n-gram language modeling
Computer Speech & Language, 2007Co-Authors: Brian Roark, Murat Saraclar, Michael CollinsAbstract:This paper describes discriminative language modeling for a large vocabulary speech recognition task. We contrast two parameter estimation methods: the Perceptron Algorithm, and a method based on maximizing the regularized conditional log-likelihood. The models are encoded as deterministic weighted finite state automata, and are applied by intersecting the automata with word-lattices that are the output from a baseline recognizer. The Perceptron Algorithm has the benefit of automatically selecting a relatively small feature set in just a couple of passes over the training data. We describe a method based on regularized likelihood that makes use of the feature set given by the Perceptron Algorithm, and initialization with the Perceptron's weights; this method gives an additional 0.5% reduction in word error rate (WER) over training with the Perceptron alone. The final system achieves a 1.8% absolute reduction in WER for a baseline first-pass recognition system (from 39.2% to 37.4%), and a 0.9% absolute reduction in WER for a multi-pass recognition system (from 28.9% to 28.0%).
-
discriminative language modeling with conditional random fields and the Perceptron Algorithm
Meeting of the Association for Computational Linguistics, 2004Co-Authors: Brian Roark, Michael Collins, Murat Saraclar, Mark JohnsonAbstract:This paper describes discriminative language modeling for a large vocabulary speech recognition task. We contrast two parameter estimation methods: the Perceptron Algorithm, and a method based on conditional random fields (CRFs). The models are encoded as deterministic weighted finite state automata, and are applied by intersecting the automata with word-lattices that are the output from a baseline recognizer. The Perceptron Algorithm has the benefit of automatically selecting a relatively small feature set in just a couple of passes over the training data. However, using the feature set output from the Perceptron Algorithm (initialized with their weights), CRF training provides an additional 0.5% reduction in word error rate, for a total 1.8% absolute reduction from the baseline of 39.2%.
-
corrective language modeling for large vocabulary asr with the Perceptron Algorithm
International Conference on Acoustics Speech and Signal Processing, 2004Co-Authors: Brian Roark, Murat Saraclar, Michael CollinsAbstract:This paper investigates error-corrective language modeling using the Perceptron Algorithm on word lattices. The resulting model is encoded as a weighted finite-state automaton, and is used by intersecting the model with word lattices, making it simple and inexpensive to apply during decoding. We present results for various training scenarios for the Switchboard task, including using n-gram features of different orders, and performing n-best extraction versus using full word lattices. We demonstrate the importance of making the training conditions as close as possible to testing conditions. The best approach yields a 1.3 percent improvement in first pass accuracy, which translates to 0.5 percent improvement after other rescoring passes.
-
Incremental parsing with the Perceptron Algorithm
Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics - ACL '04, 2004Co-Authors: Michael Collins, Brian RoarkAbstract:This paper describes an incremental parsing approach where parameters are estimated using a variant of the Perceptron Algorithm. A beam-search Algorithm is used during both training and decoding phases of the method. The Perceptron approach was implemented with the same feature set as that of an existing generative model (Roark, 2001a), and experimental results show that it gives competitive performance to the generative model on parsing the Penn treebank. We demonstrate that training a Perceptron model to combine with the generative model during search provides a 2.1 percent F-measure improvement over the generative model alone, to 88.8 percent.
-
ACL - Discriminative Language Modeling with Conditional Random Fields and the Perceptron Algorithm
Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics - ACL '04, 2004Co-Authors: Brian Roark, Michael Collins, Murat Saraclar, Mark JohnsonAbstract:This paper describes discriminative language modeling for a large vocabulary speech recognition task. We contrast two parameter estimation methods: the Perceptron Algorithm, and a method based on conditional random fields (CRFs). The models are encoded as deterministic weighted finite state automata, and are applied by intersecting the automata with word-lattices that are the output from a baseline recognizer. The Perceptron Algorithm has the benefit of automatically selecting a relatively small feature set in just a couple of passes over the training data. However, using the feature set output from the Perceptron Algorithm (initialized with their weights), CRF training provides an additional 0.5% reduction in word error rate, for a total 1.8% absolute reduction from the baseline of 39.2%.