The Experts below are selected from a list of 295992 Experts worldwide ranked by ideXlab platform

Matthias Schulist - One of the best experts on this subject based on the ideXlab platform.

  • fir filter design with additional constraints using complex chebyshev Approximation
    Signal Processing, 1993
    Co-Authors: Matthias Schulist
    Abstract:

    Abstract A powerful single exchange algorithm for the solution of the complex Chebyshev Approximation Problem was introduced by Tang. It was picked up and modified by Alkhairy et al. for the design of digital FIR filters. In this paper we extend this algorithm to solve the Approximation Problem in conjunction with additional constraints, such as constraints on the filter coefficients, constraints on the magnitude response and / or its derivatives or constraints on the group delay. Since the algorithm deals with a linear optimization Problem, all constraints have to be linear with respect to the filter coefficients. We show this linearization and the inclusion into the complex Approximation Problem and the algorithm as well. A final example will demonstrate results achieved with the modified algorithm.

Lekheng Lim - One of the best experts on this subject based on the ideXlab platform.

  • Nonnegative Approximations of nonnegative tensors
    Journal of Chemometrics, 2009
    Co-Authors: Lekheng Lim, Pierre Comon
    Abstract:

    We study the decomposition of a nonnegative tensor into a minimal sum of outer product of nonnegative vectors and the associated parsimonious naive Bayes probabilistic model. We show that the corresponding Approximation Problem, which is central to nonnegative Parafac, will always have optimal solutions. The result holds for any choice of norms and, under a mild assumption, even Bregman divergences.

  • nonnegative Approximations of nonnegative tensors
    Journal of Chemometrics, 2009
    Co-Authors: Lekheng Lim, Pierre Comon
    Abstract:

    We study the decomposition of a nonnegative tensor into a minimal sum of outer product of nonnegative vectors and the associated parsimonious naive Bayes probabilistic model. We show that the corresponding Approximation Problem, which is central to nonnegative PARAFAC, will always have optimal solutions. The result holds for any choice of norms and, under a mild assumption, even Bregman divergences. Copyright © 2009 John Wiley & Sons, Ltd.

  • tensor rank and the ill posedness of the best low rank Approximation Problem
    SIAM Journal on Matrix Analysis and Applications, 2008
    Co-Authors: Vin De Silva, Lekheng Lim
    Abstract:

    There has been continued interest in seeking a theorem describing optimal low-rank Approximations to tensors of order 3 or higher that parallels the Eckart-Young theorem for matrices. In this paper, we argue that the naive approach to this Problem is doomed to failure because, unlike matrices, tensors of order 3 or higher can fail to have best rank-$r$ Approximations. The phenomenon is much more widespread than one might suspect: examples of this failure can be constructed over a wide range of dimensions, orders, and ranks, regardless of the choice of norm (or even Bregman divergence). Moreover, we show that in many instances these counterexamples have positive volume: they cannot be regarded as isolated phenomena.  In one extreme case, we exhibit a tensor space in which no rank-3 tensor has an optimal rank-2 Approximation. The notable exceptions to this misbehavior are rank-1 tensors and order-2 tensors (i.e., matrices). In a more positive spirit, we propose a natural way of overcoming the ill-posedness of the low-rank Approximation Problem, by using weak solutions when true solutions do not exist. For this to work, it is necessary to characterize the set of weak solutions, and we do this  in the case of rank 2, order 3 (in arbitrary dimensions). In our work we emphasize the importance of closely studying concrete low-dimensional examples as a first step toward more general results. To this end, we present a detailed analysis of equivalence classes of $2 \times 2 \times 2$ tensors, and we develop methods for extending results upward to higher orders and dimensions. Finally, we link our work to existing studies of tensors from an algebraic geometric point of view. The rank of a tensor can in theory be given a semialgebraic description; in other words, it can be determined by a system of polynomial inequalities. We study some of these polynomials in cases of interest to us; in particular, we make extensive use of the hyperdeterminant $\Delta$ on $\mathbb{R}^{2\times 2 \times 2}$.

  • tensor rank and the ill posedness of the best low rank Approximation Problem
    arXiv: Numerical Analysis, 2006
    Co-Authors: Vin De Silva, Lekheng Lim
    Abstract:

    There has been continued interest in seeking a theorem describing optimal low-rank Approximations to tensors of order 3 or higher, that parallels the Eckart-Young theorem for matrices. In this paper, we argue that the naive approach to this Problem is doomed to failure because, unlike matrices, tensors of order 3 or higher can fail to have best rank-r Approximations. The phenomenon is much more widespread than one might suspect: examples of this failure can be constructed over a wide range of dimensions, orders and ranks, regardless of the choice of norm (or even Bregman divergence). Moreover, we show that in many instances these counterexamples have positive volume: they cannot be regarded as isolated phenomena. In one extreme case, we exhibit a tensor space in which no rank-3 tensor has an optimal rank-2 Approximation. The notable exceptions to this misbehavior are rank-1 tensors and order-2 tensors. In a more positive spirit, we propose a natural way of overcoming the ill-posedness of the low-rank Approximation Problem, by using weak solutions when true solutions do not exist. In our work we emphasize the importance of closely studying concrete low-dimensional examples as a first step towards more general results. To this end, we present a detailed analysis of equivalence classes of 2-by-2-by-2 tensors, and we develop methods for extending results upwards to higher orders and dimensions. Finally, we link our work to existing studies of tensors from an algebraic geometric point of view. The rank of a tensor can in theory be given a semialgebraic description; i.e., can be determined by a system of polynomial inequalities. In particular we make extensive use of the 2-by-2-by-2 hyperdeterminant.

Nick Vannieuwenhoven - One of the best experts on this subject based on the ideXlab platform.

  • a riemannian trust region method for the canonical tensor rank Approximation Problem
    Siam Journal on Optimization, 2018
    Co-Authors: Paul Breiding, Nick Vannieuwenhoven
    Abstract:

    The canonical tensor rank Approximation Problem (TAP) consists of approximating a real-valued tensor by one of low canonical rank, which is a challenging nonlinear, nonconvex, constrained optimizat...

  • a riemannian trust region method for the canonical tensor rank Approximation Problem
    arXiv: Numerical Analysis, 2017
    Co-Authors: Paul Breiding, Nick Vannieuwenhoven
    Abstract:

    The canonical tensor rank Approximation Problem (TAP) consists of approximating a real-valued tensor by one of low canonical rank, which is a challenging non-linear, non-convex, constrained optimization Problem, where the constraint set forms a non-smooth semi-algebraic set. We introduce a Riemannian Gauss-Newton method with trust region for solving small-scale, dense TAPs. The novelty of our approach is threefold. First, we parametrize the constraint set as the Cartesian product of Segre manifolds, hereby formulating the TAP as a Riemannian optimization Problem, and we argue why this parametrization is among the theoretically best possible. Second, an original ST-HOSVD-based retraction operator is proposed. Third, we introduce a hot restart mechanism that efficiently detects when the optimization process is tending to an ill-conditioned tensor rank decomposition and which often yields a quick escape path from such spurious decompositions. Numerical experiments show improvements of up to three orders of magnitude in terms of the expected time to compute a successful solution over existing state-of-the-art methods.

Joel A Tropp - One of the best experts on this subject based on the ideXlab platform.

  • algorithms for simultaneous sparse Approximation part ii convex relaxation
    Signal Processing, 2006
    Co-Authors: Joel A Tropp
    Abstract:

    A simultaneous sparse Approximation Problem requests a good Approximation of several input signals at once using different linear combinations of the same elementary signals. At the same time, the Problem balances the error in Approximation against the total number of elementary signals that participate. These elementary signals typically model coherent structures in the input signals, and they are chosen from a large, linearly dependent collection.The first part of this paper presents theoretical and numerical results for a greedy pursuit algorithm, called simultaneous orthogonal matching pursuit.The second part of the paper develops another algorithmic approach called convex relaxation. This method replaces the combinatorial simultaneous sparse Approximation Problem with a closely related convex program that can be solved efficiently with standard mathematical programming software. The paper develops conditions under which convex relaxation computes good solutions to simultaneous sparse Approximation Problems.

Inderjit S Dhillon - One of the best experts on this subject based on the ideXlab platform.

  • fast projection based methods for the least squares nonnegative matrix Approximation Problem
    Statistical Analysis and Data Mining, 2008
    Co-Authors: Dongmin Kim, Suvrit Sra, Inderjit S Dhillon
    Abstract:

    Nonnegative matrix Approximation (NNMA) is a popular matrix decomposition technique that has proven to be useful across a diverse variety of fields with applications ranging from document analysis and image processing to bioinformatics and signal processing. Over the years, several algorithms for NNMA have been proposed, e.g. Lee and Seung's multiplicative updates, alternating least squares (ALS), and gradient descent-based procedures. However, most of these procedures suffer from either slow convergence, numerical instability, or at worst, serious theoretical drawbacks. In this paper, we develop a new and improved algorithmic framework for the least-squares NNMA Problem, which is not only theoretically well-founded, but also overcomes many deficiencies of other methods. Our framework readily admits powerful optimization techniques and as concrete realizations we present implementations based on the Newton, BFGS and conjugate gradient methods. Our algorithms provide numerical results superior to both Lee and Seung's method as well as to the alternating least squares heuristic, which was reported to work well in some situations but has no theoretical guarantees [1]. Our approach extends naturally to include regularization and box-constraints without sacrificing convergence guarantees. We present experimental results on both synthetic and real-world datasets that demonstrate the superiority of our methods, both in terms of better Approximations as well as computational efficiency. Copyright © 2007 Wiley Periodicals, Inc., A Wiley Company Statistical Analy Data Mining 1: 000-000, 2007

  • fast newton type methods for the least squares nonnegative matrix Approximation Problem
    SIAM International Conference on Data Mining, 2007
    Co-Authors: Dongmin Kim, Suvrit Sra, Inderjit S Dhillon
    Abstract:

    Nonnegative Matrix Approximation is an effective matrix decomposition technique that has proven to be useful for a wide variety of applications ranging from document analysis and image processing to bioinformatics. There exist a few algorithms for nonnegative matrix Approximation (NNMA), for example, Lee & Seung’s multiplicative updates, alternating least squares, and certain gradient descent based procedures. All of these procedures suffer from either slow convergence, numerical instabilities, or at worst, theoretical unsoundness. In this paper we present new and improved algorithms for the least-squares NNMA Problem, which are not only theoretically well-founded, but also overcome many of the deficiencies of other methods. In particular, we use non-diagonal gradient scaling to obtain rapid convergence. Our methods provide numerical results superior to both Lee & Seung’s method as well to the alternating least squares (ALS) heuristic, which is known to work well in some situations but has no theoretical guarantees (Berry et al. 2006). Our approach extends naturally to include regularization and box-constraints, without sacrificing convergence guarantees. We present experimental results on both synthetic and realworld datasets to demonstrate the superiority of our methods, in terms of better Approximations as well as efficiency.