The Experts below are selected from a list of 399234 Experts worldwide ranked by ideXlab platform
Michael I. Jordan - One of the best experts on this subject based on the ideXlab platform.
-
Tree-dependent Component Analysis
arXiv: Learning, 2012Co-Authors: Francis Bach, Michael I. JordanAbstract:We present a generalization of independent component analysis (ICA), where instead of looking for a linear transform that makes the data components independent, we look for a transform that makes the data components well fit by a tree-structured graphical model. Treating the problem as a semiparametric statistical problem, we show that the optimal transform is found by minimizing a Contrast Function based on mutual information, a Function that directly extends the Contrast Function used for classical ICA. We provide two approximations of this Contrast Function, one using kernel density estimation, and another using kernel generalized variance. This tree-dependent component analysis framework leads naturally to an efficient general multivariate density estimation technique where only bivariate density estimation needs to be performed.
-
dimensionality reduction for supervised learning with reproducing kernel hilbert spaces
International Conference on Artificial Intelligence and Statistics, 2004Co-Authors: Kenji Fukumizu, Francis Bach, Michael I. JordanAbstract:We propose a novel method of dimensionality reduction for supervised learning problems. Given a regression or classification problem in which we wish to predict a response variable Y from an explanatory variable X, we treat the problem of dimensionality reduction as that of finding a low-dimensional "effective subspace" for X which retains the statistical relationship between X and Y. We show that this problem can be formulated in terms of conditional independence. To turn this formulation into an optimization problem we establish a general nonparametric characterization of conditional independence using covariance operators on reproducing kernel Hilbert spaces. This characterization allows us to derive a Contrast Function for estimation of the effective subspace. Unlike many conventional methods for dimensionality reduction in supervised learning, the proposed method requires neither assumptions on the marginal distribution of X, nor a parametric model of the conditional distribution of Y. We present experiments that compare the performance of the method with conventional methods.
-
UAI - Tree-dependent component analysis
2002Co-Authors: Francis Bach, Michael I. JordanAbstract:We present a generalization of independent component analysis (ICA), where instead of looking for a linear transform that makes the data components independent, we look for a transform that makes the data components well fit by a tree-structured graphical model. Treating the problem as a semiparametric statistical problem, we show that the optimal transform is found by minimizing a Contrast Function based on mutual information, a Function that directly extends the Contrast Function used for classical ICA. We provide two approximations of this Contrast Function, one using kernel density estimation, and another using kernel generalized variance. This tree-dependent component analysis framework leads naturally to an efficient general multivariate density estimation technique where only bivariate density estimation needs to be performed.
Vicente Zarzoso - One of the best experts on this subject based on the ideXlab platform.
-
Solving Independent Component Analysis Contrast Functions with Particle Swarm Optimization
2010Co-Authors: Jorge Igual, Jehad Ababneh, Julio Miró-borrás, Raúl Llinares, Vicente ZarzosoAbstract:Independent Component Analysis (ICA) is a statistical computation method that transforms a random vector in another one whose components are independent. Because the marginal distributions are usually unknown, the final problem is reduced to an optimization of a Contrast Function, a Function that measures the independence of the components. In this paper, the stochastic global Particle Swarm Optimization (PSO) algorithm is used to solve the opti- mization problem. The PSO is used to separate some selected benchmarks signals based on two different Contrast Functions. The results obtained using the PSO are compared with classical ICA algorithms. It is shown that the PSO is a more powerful and robust technique and capable of finding the original signals or sources when classical ICA algorithms give poor results or fail to converge
-
a Contrast Function for independent component analysis without permutation ambiguity
IEEE Transactions on Neural Networks, 2010Co-Authors: Vicente Zarzoso, Pierre Comon, Ronald PhlypoAbstract:This brief deals with the problem of blind source separation (BSS) via independent component analysis (ICA). We prove that a linear combination of the separator output fourth-order marginal cumulants (kurtoses) is a valid Contrast Function for ICA under prewhitening if the weights have the same sign as the source kurtoses. If, in addition, the source kurtoses are different and so are the linear combination weights, the Contrast eliminates the permutation ambiguity typical to ICA, as the estimated sources are sorted at the separator output according to their kurtosis values in the same order as the weights. If the weights equal the source kurtoses, the Contrast is a cumulant matching criterion based on the maximum-likelihood principle. The Contrast can be maximized by means of a cost-efficient Jacobi-type pairwise iteration. In the real-valued two-signal case, the asymptotic variance of the resulting Givens angle estimator is determined in closed form, leading to the Contrast weights with optimal finite-sample performance. A fully blind solution can be implemented by computing the optimum weights from the initial source estimates obtained by a classical ICA stage. An experimental study validates the features of the proposed technique and shows its superior performance compared to related previous methods.
-
Iterative Algorithms
2010Co-Authors: Vicente Zarzoso, Hyvärinen AapoAbstract:The present chapter surveys computational algorithms for solving the independent component analysis (ICA) problem. Most of these algorithms rely on gradient or Newton iterations for Contrast Function maximization, and can work either in batch or adaptive processing mode. After briefly summarizing the common tools employed in their design and analysis, the chapter reviews a variety of iterative techniques ranging from pioneering neural network approaches and relative (or natural) gradient methods to Newton-like fixed-point algorithms as well as methods based on some form of optimal step-size coefficient.
-
ICANN (2) - Solving independent component analysis Contrast Functions with particle swarm optimization
Artificial Neural Networks – ICANN 2010, 2010Co-Authors: Jorge Igual, Jehad Ababneh, Raul Llenares, Julio Miró-borrás, Vicente ZarzosoAbstract:Independent Component Analysis (ICA) is a statistical computation method that transforms a random vector in another one whose components are independent. Because the marginal distributions are usually unknown, the final problem is reduced to an optimization of a Contrast Function, a Function that measures the independence of the components. In this paper, the stochastic global Particle Swarm Optimization (PSO) algorithm is used to solve the optimization problem. The PSO is used to separate some selected benchmarks signals based on two different Contrast Functions. The results obtained using the PSO are compared with classical ICA algorithms. It is shown that the PSO is a more powerful and robust technique and capable of finding the original signals or sources when classical ICA algorithms give poor results or fail to converge.
-
A Contrast for Independent Component Analysis with Priors on Source Kurtosis Signs
IEEE Signal Processing Letters, 2008Co-Authors: Vicente Zarzoso, Ronald Phlypo, Pierre ComonAbstract:A Contrast Function for Independent Component Analysis (ICA) is presented incorporating the prior knowledge on the sub-Gaussian or super-Gaussian character of the sources as described by their kurtosis signs. The Contrast is related to the maximum likelihood principle, reduces the permutation indeterminacy typical of ICA, and proves particularly useful in the direct extraction of a source signal with distinct kurtosis sign. In addition, its numerical maximization can be performed cost-effectively by a Jacobi-like pairwise iteration. Extensions to standardized cumulants of orders other than four are also given.
Eric Moreau - One of the best experts on this subject based on the ideXlab platform.
-
New kurtosis optimization schemes for MISO equalization
IEEE Transactions on Signal Processing, 2012Co-Authors: Marc Castella, Eric MoreauAbstract:This paper deals with efficient optimization of cumulant based Contrast Functions. Such a problem occurs in the blind source separation framework, where Contrast Functions are criteria to be maximized in order to retrieve the sources. More precisely, we focus on the extraction of one source signal and our method applies in deflation approaches, where the sources are extracted one by one. We propose new methods to maximize the kurtosis Contrast Function. These methods are intermediate between a gradient and an iterative "fixed-point" optimization of so-called reference Contrasts. They rely on iterative updates of the parameters which monotonically increase the Contrast Function value: we point out the strong similarity with the Expectation-Maximization (EM) method and with recent generalizations referred to as Minimization-Maximization (MM). We also prove the global convergence of the algorithm to a stationary point. Simulations confirm the convergence of our methods to a separating solution. They also show experimentally that our methods have a much lower computational cost than former classical optimization methods. Finally, simulations suggest that the methods remain valid under weaker conditions than those required for proving convergence.
-
Gradient algorithm for reference-based cubic Contrast Function in a deflation scenario
2011 IEEE Statistical Signal Processing Workshop (SSP), 2011Co-Authors: Fadoua Brahim, Remi Dubroca, Christophe De Luigi, Eric MoreauAbstract:The paper deals with the problem of blind source separation of a MIMO convolutive mixture by a deflation procedure. A criterion based on high order statistics and showing a cubic dependence w.r.t. the unknown equalizer parameters has been recently proposed. In order to optimize efficiently this criterion in a classical deflation scenario, we propose a new algorithm based on a fixed step size gradient. Computer simulations illustrate the good behavior and the usefulness of our algorithm in comparison with other approaches.
-
A new method for kurtosis maximization and source separation
2010Co-Authors: Marc Castella, Eric MoreauAbstract:This paper introduces a new method to maximize kurtosis-based Contrast Functions. Such Contrast Functions appear in the problem of blind source separation of convolutively mixed sources: the corresponding methods recover the sources one by one using a deflation approach. The proposed maximization algorithm is based on the particular nature of the criterion. The method is similar in spirit to a gradient ascent method, but differs in the fact that a "reference" Contrast Function is considered at each line search. The convergence of the method to a stationary point of the criterion can be proved. The theoretical result is illustrated by simulation
-
a one stage self adaptive algorithm for source separation
International Conference on Acoustics Speech and Signal Processing, 1994Co-Authors: Eric Moreau, Odile MacchiAbstract:In order to perform separation of a mixture of sources, an interesting approach is to maximise a Contrast Function: e.g. the Contrast of Comon. This paper brings two novel contributions (i) a novel algorithm as proposed in order to adaptively maximise Comon's Contrast. However it requires a preprocessing whitening operation which is awkward when the mixture is ill-conditioned. (ii) A new criterion is defined that is free of the prewhitening step. In the case of two sources it can be proved that this criterion is a Contrast. This Contrast can also be adaptively maximized and has the additional advantage not to require identical signs for the fourth-order cumulants of the sources. Achievement of these two adaptive algorithms is demonstrated using a new performance index. >
-
ICASSP (2) - A quadratic MISO Contrast Function for blind equalization
2004 IEEE International Conference on Acoustics Speech and Signal Processing, 1Co-Authors: Marc Castella, Eric Moreau, Jean-christophe PesquetAbstract:The paper is concerned with blind separation of convolutive mixtures of mutually independent signals. We consider the MISO extraction of one source signal based on the maximization of a Contrast Function (CF); a new, so-called "reference" CF is proposed, which is based on cross-statistics between the estimated output and a reference signal. The proposed CF is valid both for i.i.d. and non i.i.d. sources. It presents the advantage over other CFs to be a quadratic Function, which makes its optimization much easier to realize. Finally, simulations demonstrate the validity of this CF and show that it leads to improved separation performances.
P L Indovina - One of the best experts on this subject based on the ideXlab platform.
-
Time-resolved Contrast Function and optical characterization of spatially varying absorptive inclusions at different depths in diffusing media.
Physical review. E Statistical nonlinear and soft matter physics, 2004Co-Authors: S De Nicola, R Esposito, M Lepore, P L IndovinaAbstract:The role of a spatially varying absorptive inhomogeneity located at different depths within a turbid material has been investigated. This inhomogeneity has been characterized by a spatially dependent Gaussian distribution of its absorption coefficient. The present study has been performed calculating the time-resolved Contrast Function in the framework of the first-order perturbative approach to the diffusion equation for a slab geometry and a coaxial measurement scheme. The model has allowed us to take into account different locations of the inclusion along the source-detector axis. The accuracy of time-resolved Contrast predictions has been analyzed through comparisons with results of the finite element method that has been used to numerically solve the diffusion equation. Recovery of the absorption perturbation parameter of the inhomogeneity for different axial positions has also been investigated.
Francis Bach - One of the best experts on this subject based on the ideXlab platform.
-
Tree-dependent Component Analysis
arXiv: Learning, 2012Co-Authors: Francis Bach, Michael I. JordanAbstract:We present a generalization of independent component analysis (ICA), where instead of looking for a linear transform that makes the data components independent, we look for a transform that makes the data components well fit by a tree-structured graphical model. Treating the problem as a semiparametric statistical problem, we show that the optimal transform is found by minimizing a Contrast Function based on mutual information, a Function that directly extends the Contrast Function used for classical ICA. We provide two approximations of this Contrast Function, one using kernel density estimation, and another using kernel generalized variance. This tree-dependent component analysis framework leads naturally to an efficient general multivariate density estimation technique where only bivariate density estimation needs to be performed.
-
dimensionality reduction for supervised learning with reproducing kernel hilbert spaces
International Conference on Artificial Intelligence and Statistics, 2004Co-Authors: Kenji Fukumizu, Francis Bach, Michael I. JordanAbstract:We propose a novel method of dimensionality reduction for supervised learning problems. Given a regression or classification problem in which we wish to predict a response variable Y from an explanatory variable X, we treat the problem of dimensionality reduction as that of finding a low-dimensional "effective subspace" for X which retains the statistical relationship between X and Y. We show that this problem can be formulated in terms of conditional independence. To turn this formulation into an optimization problem we establish a general nonparametric characterization of conditional independence using covariance operators on reproducing kernel Hilbert spaces. This characterization allows us to derive a Contrast Function for estimation of the effective subspace. Unlike many conventional methods for dimensionality reduction in supervised learning, the proposed method requires neither assumptions on the marginal distribution of X, nor a parametric model of the conditional distribution of Y. We present experiments that compare the performance of the method with conventional methods.
-
UAI - Tree-dependent component analysis
2002Co-Authors: Francis Bach, Michael I. JordanAbstract:We present a generalization of independent component analysis (ICA), where instead of looking for a linear transform that makes the data components independent, we look for a transform that makes the data components well fit by a tree-structured graphical model. Treating the problem as a semiparametric statistical problem, we show that the optimal transform is found by minimizing a Contrast Function based on mutual information, a Function that directly extends the Contrast Function used for classical ICA. We provide two approximations of this Contrast Function, one using kernel density estimation, and another using kernel generalized variance. This tree-dependent component analysis framework leads naturally to an efficient general multivariate density estimation technique where only bivariate density estimation needs to be performed.