The Experts below are selected from a list of 53343 Experts worldwide ranked by ideXlab platform
Phillip A. Regalia - One of the best experts on this subject based on the ideXlab platform.
-
A Gradient search interpretation of the super-exponential algorithm
IEEE Transactions on Information Theory, 2000Co-Authors: Mamadou Mboup, Phillip A. RegaliaAbstract:This article reviews the super-exponential algorithm proposed by Shalvi and Weinstein (1993) for blind channel equalization. The principle of this algorithm-Hadamard exponentiation, projection over the set of attainable combined channel-equalizer impulse responses followed by a normalization-is shown to coincide with a Gradient search of an extremum of a cost function. The cost function belongs to the family of functions given as the ratio of the standard l2p and l2 sequence norms, where p>1. This family is very relevant in blind channel equalization, tracing back to Donoho's (1981) work on minimum entropy deconvolution and also underlying the Godard (1980) (or constant modulus) and the earlier Shalvi-Weinstein algorithms. Using this Gradient search interpretation, which is more tractable for analytical study, we give a simple proof of convergence for the super-exponential algorithm. Finally, we show that the Gradient Step-size choice giving rise to the super-exponential algorithm is optimal
-
ICASSP - On the equivalence between the super-exponential algorithm and a Gradient search method
1999 IEEE International Conference on Acoustics Speech and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258), 1999Co-Authors: Mamadou Mboup, Phillip A. RegaliaAbstract:This paper reviews the super-exponential algorithm proposed by Shalvi and Weinstein (1993) for blind channel equalization. We show that the algorithm coincides with a Gradient search of a maximum of a cost function, which belongs to a family of functions very relevant in blind channel equalization. This family traces back to Donoho's (1981) work on minimum entropy deconvolution, and also underlies the Godard (1980) (or constant modulus) and the Shalvi-Weinstein algorithms. Using this Gradient search interpretation, we give a simple proof of convergence for the super-exponential algorithm. Finally, we show that the Gradient Step-size choice giving rise to the super-exponential algorithm is optimal.
Matthew D. Hoffman - One of the best experts on this subject based on the ideXlab platform.
-
ICML - A trust-region method for stochastic variational inference with applications to streaming data
2015Co-Authors: Lucas Theis, Matthew D. HoffmanAbstract:Stochastic variational inference allows for fast posterior inference in complex Bayesian models. However, the algorithm is prone to local optima which can make the quality of the posterior approximation sensitive to the choice of hyperparameters and initialization. We address this problem by replacing the natural Gradient Step of stochastic varitional inference with a trust-region update. We show that this leads to generally better results and reduced sensitivity to hyperparameters. We also describe a new strategy for variational inference on streaming data and show that here our trust-region method is crucial for getting good performance.
-
A trust-region method for stochastic variational inference with applications to streaming data
arXiv: Machine Learning, 2015Co-Authors: Lucas Theis, Matthew D. HoffmanAbstract:Stochastic variational inference allows for fast posterior inference in complex Bayesian models. However, the algorithm is prone to local optima which can make the quality of the posterior approximation sensitive to the choice of hyperparameters and initialization. We address this problem by replacing the natural Gradient Step of stochastic varitional inference with a trust-region update. We show that this leads to generally better results and reduced sensitivity to hyperparameters. We also describe a new strategy for variational inference on streaming data and show that here our trust-region method is crucial for getting good performance.
-
NIPS - Online Learning for Latent Dirichlet Allocation
2010Co-Authors: Matthew D. Hoffman, Francis Bach, David M. BleiAbstract:We develop an online variational Bayes (VB) algorithm for Latent Dirichlet Allocation (LDA). Online LDA is based on online stochastic optimization with a natural Gradient Step, which we show converges to a local optimum of the VB objective function. It can handily analyze massive document collections, including those arriving in a stream. We study the performance of online LDA in several ways, including by fitting a 100-topic topic model to 3.3M articles from Wikipedia in a single pass. We demonstrate that online LDA finds topic models as good or better than those found with batch VB, and in a fraction of the time.
Mamadou Mboup - One of the best experts on this subject based on the ideXlab platform.
-
A Gradient search interpretation of the super-exponential algorithm
IEEE Transactions on Information Theory, 2000Co-Authors: Mamadou Mboup, Phillip A. RegaliaAbstract:This article reviews the super-exponential algorithm proposed by Shalvi and Weinstein (1993) for blind channel equalization. The principle of this algorithm-Hadamard exponentiation, projection over the set of attainable combined channel-equalizer impulse responses followed by a normalization-is shown to coincide with a Gradient search of an extremum of a cost function. The cost function belongs to the family of functions given as the ratio of the standard l2p and l2 sequence norms, where p>1. This family is very relevant in blind channel equalization, tracing back to Donoho's (1981) work on minimum entropy deconvolution and also underlying the Godard (1980) (or constant modulus) and the earlier Shalvi-Weinstein algorithms. Using this Gradient search interpretation, which is more tractable for analytical study, we give a simple proof of convergence for the super-exponential algorithm. Finally, we show that the Gradient Step-size choice giving rise to the super-exponential algorithm is optimal
-
ICASSP - On the equivalence between the super-exponential algorithm and a Gradient search method
1999 IEEE International Conference on Acoustics Speech and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258), 1999Co-Authors: Mamadou Mboup, Phillip A. RegaliaAbstract:This paper reviews the super-exponential algorithm proposed by Shalvi and Weinstein (1993) for blind channel equalization. We show that the algorithm coincides with a Gradient search of a maximum of a cost function, which belongs to a family of functions very relevant in blind channel equalization. This family traces back to Donoho's (1981) work on minimum entropy deconvolution, and also underlies the Godard (1980) (or constant modulus) and the Shalvi-Weinstein algorithms. Using this Gradient search interpretation, we give a simple proof of convergence for the super-exponential algorithm. Finally, we show that the Gradient Step-size choice giving rise to the super-exponential algorithm is optimal.
Lucas Theis - One of the best experts on this subject based on the ideXlab platform.
-
ICML - A trust-region method for stochastic variational inference with applications to streaming data
2015Co-Authors: Lucas Theis, Matthew D. HoffmanAbstract:Stochastic variational inference allows for fast posterior inference in complex Bayesian models. However, the algorithm is prone to local optima which can make the quality of the posterior approximation sensitive to the choice of hyperparameters and initialization. We address this problem by replacing the natural Gradient Step of stochastic varitional inference with a trust-region update. We show that this leads to generally better results and reduced sensitivity to hyperparameters. We also describe a new strategy for variational inference on streaming data and show that here our trust-region method is crucial for getting good performance.
-
A trust-region method for stochastic variational inference with applications to streaming data
arXiv: Machine Learning, 2015Co-Authors: Lucas Theis, Matthew D. HoffmanAbstract:Stochastic variational inference allows for fast posterior inference in complex Bayesian models. However, the algorithm is prone to local optima which can make the quality of the posterior approximation sensitive to the choice of hyperparameters and initialization. We address this problem by replacing the natural Gradient Step of stochastic varitional inference with a trust-region update. We show that this leads to generally better results and reduced sensitivity to hyperparameters. We also describe a new strategy for variational inference on streaming data and show that here our trust-region method is crucial for getting good performance.
Alexander J Smola - One of the best experts on this subject based on the ideXlab platform.
-
Step size adaptation in reproducing kernel hilbert space
Journal of Machine Learning Research, 2006Co-Authors: S V N Vishwanathan, Nicol N Schraudolph, Alexander J SmolaAbstract:This paper presents an online support vector machine (SVM) that uses the stochastic meta-descent (SMD) algorithm to adapt its Step size automatically. We formulate the online learning problem as a stochastic Gradient descent in reproducing kernel Hilbert space (RKHS) and translate SMD to the nonparametric setting, where its Gradient trace parameter is no longer a coefficient vector but an element of the RKHS. We derive efficient updates that allow us to perform the Step size adaptation in linear time. We apply the online SVM framework to a variety of loss functions, and in particular show how to handle structured output spaces and achieve efficient online multiclass classification. Experiments show that our algorithm outperforms more primitive methods for setting the Gradient Step size.
