The Experts below are selected from a list of 2019 Experts worldwide ranked by ideXlab platform
Hiroyuki Kasai - One of the best experts on this subject based on the ideXlab platform.
-
accelerated stochastic Multiplicative Update with gradient averaging for nonnegative matrix factorizations
European Signal Processing Conference, 2018Co-Authors: Hiroyuki KasaiAbstract:Nonnegative matrix factorization (NMF) is a powerful tool in data analysis by discovering latent features and part-based patterns from high-dimensional data, and is a special case in which factor matrices have low-rank nonnegative constraints. Applying NMF into huge-size matrices, we specifically address stochastic Multiplicative Update (MU) rule, which is the most popular, but which has slow convergence property. This present paper introduces a gradient averaging technique of stochastic gradient on the stochastic MU rule, and proposes an accelerated stochastic Multiplicative Update rule: SAGMU. Extensive computational experiments using both synthetic and real-world datasets demonstrate the effectiveness of SAGMU.
-
EUSIPCO - Accelerated stochastic Multiplicative Update with gradient averaging for nonnegative matrix factorizations
2018 26th European Signal Processing Conference (EUSIPCO), 2018Co-Authors: Hiroyuki KasaiAbstract:Nonnegative matrix factorization (NMF) is a powerful tool in data analysis by discovering latent features and part-based patterns from high-dimensional data, and is a special case in which factor matrices have low-rank nonnegative constraints. Applying NMF into huge-size matrices, we specifically address stochastic Multiplicative Update (MU) rule, which is the most popular, but which has slow convergence property. This present paper introduces a gradient averaging technique of stochastic gradient on the stochastic MU rule, and proposes an accelerated stochastic Multiplicative Update rule: SAGMU. Extensive computational experiments using both synthetic and real-world datasets demonstrate the effectiveness of SAGMU.
-
ICASSP - Stochastic Variance Reduced Multiplicative Update for Nonnegative Matrix Factorization
2018 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP), 2018Co-Authors: Hiroyuki KasaiAbstract:Nonnegative matrix factorization (NMF), a dimensionality reduction and factor analysis method, is a special case in which factor matrices have low-rank nonnegative constraints. Considering the stochastic learning in NMF, we specifically address the Multiplicative Update (MU) rule, which is the most popular, but which has slow convergence property. This present paper introduces on the stochastic MU rule a variance-reduced technique of stochastic gradient. Numerical comparisons suggest that our proposed algorithms robustly outperform state-of-the-art algorithms across different synthetic and real-world datasets.
-
Stochastic variance reduced Multiplicative Update for nonnegative matrix factorization
arXiv: Numerical Analysis, 2017Co-Authors: Hiroyuki KasaiAbstract:Nonnegative matrix factorization (NMF), a dimensionality reduction and factor analysis method, is a special case in which factor matrices have low-rank nonnegative constraints. Considering the stochastic learning in NMF, we specifically address the Multiplicative Update (MU) rule, which is the most popular, but which has slow convergence property. This present paper introduces on the stochastic MU rule a variance-reduced technique of stochastic gradient. Numerical comparisons suggest that our proposed algorithms robustly outperform state-of-the-art algorithms across different synthetic and real-world datasets.
Yannick Deville - One of the best experts on this subject based on the ideXlab platform.
-
an nmf based approach for hyperspectral unmixing using a new Multiplicative tuning linear mixing model to address spectral variability
European Signal Processing Conference, 2019Co-Authors: Fatima Zohra Benhalouche, Moussa Sofiane Karoui, Yannick DevilleAbstract:In this work, a new approach is presented for unmixing remote sensing hyperspectral data. This approach considers a linear mixing model that is introduced in these investigations to handle the spectral variability phenomenon, which is usually observed in the considered data and which is here modeled in a Multiplicative form. The proposed algorithm, which is based on a pixel-by-pixel non-negative matrix factorization method, uses Multiplicative Update rules for minimizing a cost function that takes into account the introduced linear mixing model. Tests, by means of realistic synthetic data, are conducted to evaluate the performance of the proposed approach, and the obtained results are compared to those of methods from the literature. These test results show that the proposed approach outperforms all other tested methods.
-
EUSIPCO - An NMF-Based Approach for Hyperspectral Unmixing Using a New Multiplicative-tuning Linear Mixing Model to Address Spectral Variability
2019 27th European Signal Processing Conference (EUSIPCO), 2019Co-Authors: Fatima Zohra Benhalouche, Moussa Sofiane Karoui, Yannick DevilleAbstract:In this work, a new approach is presented for unmixing remote sensing hyperspectral data. This approach considers a linear mixing model that is introduced in these investigations to handle the spectral variability phenomenon, which is usually observed in the considered data and which is here modeled in a Multiplicative form. The proposed algorithm, which is based on a pixel-by-pixel non-negative matrix factorization method, uses Multiplicative Update rules for minimizing a cost function that takes into account the introduced linear mixing model. Tests, by means of realistic synthetic data, are conducted to evaluate the performance of the proposed approach, and the obtained results are compared to those of methods from the literature. These test results show that the proposed approach outperforms all other tested methods.
Roland Badeau - One of the best experts on this subject based on the ideXlab platform.
-
EUSIPCO - Multiplicative Updates for modeling mixtures of non-stationary signals in the time-frequency domain
2013Co-Authors: Roland Badeau, Alexey OzerovAbstract:We recently introduced the high-resolution nonnegative matrix factorization (HR-NMF) model for representing mixtures of non-stationary signals in the time-frequency domain, and we highlighted its capability to both reach a high spectral resolution and reconstruct high quality audio signals. An expectation-maximization (EM) algorithm was also proposed for estimating its parameters. In this paper, we replace the maximization step by Multiplicative Update rules (MUR), in order to improve the convergence rate. We also introduce general MUR that are not limited to nonnegative parameters, and we propose a new insight into the EM algorithm, which shows that MUR and EM actually belong to the same family. We thus introduce a continuum of algorithms between them. Experiments confirm that the proposed approach permits to overcome the convergence rate of the EM algorithm.
-
Multiplicative Updates for modeling mixtures of non-stationary signals in the time-frequency domain
2013Co-Authors: Roland Badeau, Alexey OzerovAbstract:We recently introduced the high-resolution nonnegative matrix factorization (HR-NMF) model for representing mixtures of non-stationary signals in the time-frequency domain, and we highlighted its capability to both reach a high spectral resolution and reconstruct high quality audio signals. An expectation-maximization (EM) algorithm was also proposed for estimating its parameters. In this paper, we replace the maximization step by Multiplicative Update rules (MUR), in order to improve the convergence rate. We also introduce general MUR that are not limited to nonnegative parameters, and we propose a new insight into the EM algorithm, which shows that MUR and EM actually belong to the same family. We thus introduce a continuum of algorithms between them. Experiments confirm that the proposed approach permits to overcome the convergence rate of the EM algorithm.
-
Stability analysis of Multiplicative Update algorithms for non-negative matrix factorization
2011Co-Authors: Roland Badeau, Nancy Bertin, Emmanuel VincentAbstract:Multiplicative Update algorithms have encountered a great success to solve optimization problems with non-negativity constraints, such as the famous non-negative matrix factorization (NMF) and its many variants. However, despite several years of research on the topic, the understanding of their convergence properties is still to be improved. In this paper, we show that Lyapunov's stability theory provides a very enlightening viewpoint on the problem. We prove the stability of supervised NMF and study the more difficult case of unsupervised NMF. Numerical simulations illustrate those theoretical results, and the convergence speed of NMF Multiplicative Updates is analyzed.
-
ICASSP - Stability analysis of Multiplicative Update algorithms for non-negative matrix factorization
2011 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP), 2011Co-Authors: Roland Badeau, Nancy Bertin, Emmanuel VincentAbstract:Multiplicative Update algorithms have encountered a great success to solve optimization problems with non-negativity constraints, such as the famous non-negative matrix factorization (NMF) and its many variants. However, despite several years of research on the topic, the understanding of their convergence properties is still to be improved. In this paper, we show that Lyapunov's stability theory provides a very enlightening viewpoint on the problem. We prove the stability of supervised NMF and study the more difficult case of unsupervised NMF. Numerical simulations illustrate those theoretical results, and the convergence speed of NMF Multiplicative Updates is analyzed.
-
Stability Analysis of Multiplicative Update Algorithms and Application to Nonnegative Matrix Factorization
IEEE transactions on neural networks, 2010Co-Authors: Roland Badeau, Nancy Bertin, Emmanuel VincentAbstract:Multiplicative Update algorithms have proved to be a great success in solving optimization problems with nonnegativity constraints, such as the famous nonnegative matrix factorization (NMF) and its many variants. However, despite several years of research on the topic, the understanding of their convergence properties is still to be improved. In this paper, we show that Lyapunov's stability theory provides a very enlightening viewpoint on the problem. We prove the exponential or asymptotic stability of the solutions to general optimization problems with nonnegative constraints, including the particular case of supervised NMF, and finally study the more difficult case of unsupervised NMF. The theoretical results presented in this paper are confirmed by numerical simulations involving both supervised and unsupervised NMF, and the convergence speed of NMF Multiplicative Updates is investigated.
Renbo Zhao - One of the best experts on this subject based on the ideXlab platform.
-
a unified convergence analysis of the Multiplicative Update algorithm for regularized nonnegative matrix factorization
IEEE Transactions on Signal Processing, 2018Co-Authors: Renbo ZhaoAbstract:The Multiplicative Update (MU) algorithm has been extensively used to estimate the basis and coefficient matrices in nonnegative matrix factorization (NMF) problems under a wide range of divergences and regularizers. However, theoretical convergence guarantees have only been derived for a few special divergences without regularization. In this work, we provide a conceptually simple, self-contained, and unified proof for the convergence of the MU algorithm applied on NMF with a wide range of divergences and regularizers. Our main result shows the sequence of iterates (i.e., pairs of basis and coefficient matrices) produced by the MU algorithm converges to the set of stationary points of the nonconvex NMF optimization problem. Our proof strategy has the potential to open up new avenues for analyzing similar problems in machine learning and signal processing.
-
A unified convergence analysis of the Multiplicative Update algorithm for nonnegative matrix factorization
2017 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP), 2017Co-Authors: Renbo ZhaoAbstract:The Multiplicative Update (MU) algorithm has been used extensively to estimate the basis and coefficient matrices in nonnegative matrix factorization (NMF) problems under a wide range of divergences and regularizations. However, theoretical convergence guarantees have only been derived for a few special divergences. In this work, we provide a conceptually simple, self-contained, and unified proof for the convergence of the MU algorithm applied on NMF with a wide range of divergences and regularizations. Our result shows the sequence of iterates (i.e., pairs of basis and coefficient matrices) produced by the MU algorithm converges to the set of stationary points of the NMF (optimization) problem. Our proof strategy has the potential to open up new avenues for analyzing similar problems.
-
ICASSP - A unified convergence analysis of the Multiplicative Update algorithm for nonnegative matrix factorization
2017 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP), 2017Co-Authors: Renbo Zhao, Vincent Y. F. TanAbstract:The Multiplicative Update (MU) algorithm has been used extensively to estimate the basis and coefficient matrices in nonnegative matrix factorization (NMF) problems under a wide range of divergences and regularizations. However, theoretical convergence guarantees have only been derived for a few special divergences. In this work, we provide a conceptually simple, self-contained, and unified proof for the convergence of the MU algorithm applied on NMF with a wide range of divergences and regularizations. Our result shows the sequence of iterates (i.e., pairs of basis and coefficient matrices) produced by the MU algorithm converges to the set of stationary points of the NMF (optimization) problem. Our proof strategy has the potential to open up new avenues for analyzing similar problems.
-
A Unified Convergence Analysis of the Multiplicative Update Algorithm for Regularized NMF with General Divergences
arXiv: Optimization and Control, 2016Co-Authors: Renbo Zhao, Vincent Y. F. TanAbstract:The Multiplicative Update (MU) algorithm has been used extensively to estimate the basis and coefficient matrices in nonnegative matrix factorization (NMF) problems under a wide range of divergences and regularizations. However, theoretical convergence guarantees have only been derived for a few special divergences and without regularizers. We provide a conceptually simple, self-contained, and unified proof for the convergence of the MU algorithm applied on NMF with a wide range of divergences and with $\ell_1$ and Tikhonov regularizations. Our result shows the sequence of iterates (i.e., pairs of basis and coefficient matrices) produced by the MU algorithm converges to the set of stationary points of the NMF (optimization) problem. Our proof strategy has the potential to open up new avenues for analyzing similar problems.
Emmanuel Vincent - One of the best experts on this subject based on the ideXlab platform.
-
Stability analysis of Multiplicative Update algorithms for non-negative matrix factorization
2011Co-Authors: Roland Badeau, Nancy Bertin, Emmanuel VincentAbstract:Multiplicative Update algorithms have encountered a great success to solve optimization problems with non-negativity constraints, such as the famous non-negative matrix factorization (NMF) and its many variants. However, despite several years of research on the topic, the understanding of their convergence properties is still to be improved. In this paper, we show that Lyapunov's stability theory provides a very enlightening viewpoint on the problem. We prove the stability of supervised NMF and study the more difficult case of unsupervised NMF. Numerical simulations illustrate those theoretical results, and the convergence speed of NMF Multiplicative Updates is analyzed.
-
ICASSP - Stability analysis of Multiplicative Update algorithms for non-negative matrix factorization
2011 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP), 2011Co-Authors: Roland Badeau, Nancy Bertin, Emmanuel VincentAbstract:Multiplicative Update algorithms have encountered a great success to solve optimization problems with non-negativity constraints, such as the famous non-negative matrix factorization (NMF) and its many variants. However, despite several years of research on the topic, the understanding of their convergence properties is still to be improved. In this paper, we show that Lyapunov's stability theory provides a very enlightening viewpoint on the problem. We prove the stability of supervised NMF and study the more difficult case of unsupervised NMF. Numerical simulations illustrate those theoretical results, and the convergence speed of NMF Multiplicative Updates is analyzed.
-
Stability Analysis of Multiplicative Update Algorithms and Application to Nonnegative Matrix Factorization
IEEE transactions on neural networks, 2010Co-Authors: Roland Badeau, Nancy Bertin, Emmanuel VincentAbstract:Multiplicative Update algorithms have proved to be a great success in solving optimization problems with nonnegativity constraints, such as the famous nonnegative matrix factorization (NMF) and its many variants. However, despite several years of research on the topic, the understanding of their convergence properties is still to be improved. In this paper, we show that Lyapunov's stability theory provides a very enlightening viewpoint on the problem. We prove the exponential or asymptotic stability of the solutions to general optimization problems with nonnegative constraints, including the particular case of supervised NMF, and finally study the more difficult case of unsupervised NMF. The theoretical results presented in this paper are confirmed by numerical simulations involving both supervised and unsupervised NMF, and the convergence speed of NMF Multiplicative Updates is investigated.
-
Stability Analysis of Multiplicative Update Algorithms and Application to Nonnegative
2010Co-Authors: Roland Badeau, Nancy Bertin, Emmanuel VincentAbstract:Multiplicative Update algorithms have proved to be a great success in solving optimization problems with non- negativity constraints, such as the famous nonnegative matrix factorization (NMF) and its many variants. However, despite several years of research on the topic, the understanding of their convergence properties is still to be improved. In this paper, we show that Lyapunov's stability theory provides a very enlightening viewpoint on the problem. We prove the exponential or asymptotic stability of the solutions to general optimization problems with nonnegative constraints, including the particular case of supervised NMF, and finally study the more difficult case of unsupervised NMF. The theoretical results presented in this paper are confirmed by numerical simulations involving both supervised and unsupervised NMF, and the convergence speed of NMF Multiplicative Updates is investigated.
-
stability analysis of Multiplicative Update algorithms and application to non negative matrix factorization analyse de la stabilite des regles de mises a jour Multiplicatives et application a la factorisation en matrices positives
2010Co-Authors: Roland Badeau, Nancy Bertin, Emmanuel VincentAbstract:Multiplicative Update algorithms have encountered a great success to solve optimization problems with nonnegativity constraints, such as the famous non-negative matrix factorization (NMF) and its many variants. However, despite several years of research on the topic, the understanding of their convergence properties is still to be improved. In this paper, we show that Lyapunov’s stability theory provides a very enlightening viewpoint on the problem. We prove the exponential or asymptotic stability of the solutions to general optimization problems with non-negative constraints, including the particular case of supervised NMF, and finally study the more difficult case of unsupervised NMF. The theoretical results presented in the paper are confirmed by numerical simulations involving both supervised and unsupervised NMF, and the convergence speed of NMF Multiplicative Updates is investigated.
