The Experts below are selected from a list of 24603 Experts worldwide ranked by ideXlab platform

Vishal Monga - One of the best experts on this subject based on the ideXlab platform.

  • fast low rank shared Dictionary Learning for image classification
    IEEE Transactions on Image Processing, 2017
    Co-Authors: Vishal Monga
    Abstract:

    Despite the fact that different objects possess distinct class-specific features, they also usually share common patterns. This observation has been exploited partially in a recently proposed Dictionary Learning framework by separating the particularity and the commonality (COPAR). Inspired by this, we propose a novel method to explicitly and simultaneously learn a set of common patterns as well as class-specific features for classification with more intuitive constraints. Our Dictionary Learning framework is hence characterized by both a shared Dictionary and particular (class-specific) dictionaries. For the shared Dictionary, we enforce a low-rank constraint, i.e., claim that its spanning subspace should have low dimension and the coefficients corresponding to this Dictionary should be similar. For the particular dictionaries, we impose on them the well-known constraints stated in the Fisher discrimination Dictionary Learning (FDDL). Furthermore, we develop new fast and accurate algorithms to solve the subproblems in the Learning step, accelerating its convergence. The said algorithms could also be applied to FDDL and its extensions. The efficiencies of these algorithms are theoretically and experimentally verified by comparing their complexities and running time with those of other well-known Dictionary Learning methods. Experimental results on widely used image data sets establish the advantages of our method over the state-of-the-art Dictionary Learning methods.

  • fast low rank shared Dictionary Learning for image classification
    arXiv: Computer Vision and Pattern Recognition, 2016
    Co-Authors: Vishal Monga
    Abstract:

    Despite the fact that different objects possess distinct class-specific features, they also usually share common patterns. This observation has been exploited partially in a recently proposed Dictionary Learning framework by separating the particularity and the commonality (COPAR). Inspired by this, we propose a novel method to explicitly and simultaneously learn a set of common patterns as well as class-specific features for classification with more intuitive constraints. Our Dictionary Learning framework is hence characterized by both a shared Dictionary and particular (class-specific) dictionaries. For the shared Dictionary, we enforce a low-rank constraint, i.e. claim that its spanning subspace should have low dimension and the coefficients corresponding to this Dictionary should be similar. For the particular dictionaries, we impose on them the well-known constraints stated in the Fisher discrimination Dictionary Learning (FDDL). Further, we develop new fast and accurate algorithms to solve the subproblems in the Learning step, accelerating its convergence. The said algorithms could also be applied to FDDL and its extensions. The efficiencies of these algorithms are theoretically and experimentally verified by comparing their complexities and running time with those of other well-known Dictionary Learning methods. Experimental results on widely used image datasets establish the advantages of our method over state-of-the-art Dictionary Learning methods.

Michael Elad - One of the best experts on this subject based on the ideXlab platform.

  • dual graph regularized Dictionary Learning
    IEEE Transactions on Signal and Information Processing over Networks, 2016
    Co-Authors: Yael Yankelevsky, Michael Elad
    Abstract:

    Dictionary Learning (DL) techniques aim to find sparse signal representations that capture prominent characteristics in a given data. Such methods operate on a data matrix $Y\in \mathbb {R}^{N\times M}$ , where each of its columns $y_i\in \mathbb {R}^N$ constitutes a training sample, and these columns together represent a sampling from the data manifold. For signals $y\in \mathbb {R}^N$ residing on weighted graphs, an additional challenge is incorporating the underlying geometric structure of the data domain into the Learning process. In such cases, the topological graph structure may provide a crucial interpretation for the columns, while the data manifold itself may also possess a low-dimensional intrinsic structure that should be taken into account. In this work, we propose a novel Dictionary Learning algorithm for graph signals that simultaneously takes into account the underlying structure in both the signal and the manifold domains. Specifically, we require that the Dictionary atoms are smooth with respect to the graph topology, as encapsulated by the graph Laplacian matrix. Furthermore, we propose to learn this graph Laplacian within the Dictionary Learning process, adapting it to promote the desired smoothness. Utilizing the manifold structure, we propose to encourage the smoothness of the sparse representations on the data manifold in a similar manner. Both these smoothness forces implicitly enhance the learned Dictionary. The efficiency of the proposed approach is demonstrated on synthetic examples as well as on real data, showing that it outperforms other Dictionary Learning methods in typical problems such as resistance to noise and data completion.

  • Trainlets: Dictionary Learning in High Dimensions
    IEEE Transactions on Signal Processing, 2016
    Co-Authors: Jeremias Sulam, Boaz Ophir, Michael Zibulevsky, Michael Elad
    Abstract:

    Sparse representation has shown to be a very powerful model for real world signals, and has enabled the development of applications with notable performance. Combined with the ability to learn a Dictionary from signal examples, sparsity-inspired algorithms are often achieving state-of-the-art results in a wide variety of tasks. These methods have traditionally been restricted to small dimensions mainly due to the computational constraints that the Dictionary Learning problem entails. In the context of image processing, this implies handling small image patches. In this work we show how to efficiently handle bigger dimensions and go beyond the small patches in sparsity-based signal and image processing methods. We build our approach based on a new cropped Wavelet decomposition, which enables a multi-scale analysis with virtually no border effects. We then employ this as the base Dictionary within a double sparsity model to enable the training of adaptive dictionaries. To cope with the increase of training data, while at the same time improving the training performance, we present an Online Sparse Dictionary Learning (OSDL) algorithm to train this model effectively, enabling it to handle millions of examples. This work shows that Dictionary Learning can be up-scaled to tackle a new level of signal dimensions, obtaining large adaptable atoms that we call Trainlets.

  • multi scale Dictionary Learning using wavelets
    IEEE Journal of Selected Topics in Signal Processing, 2011
    Co-Authors: Boaz Ophir, Michael Lustig, Michael Elad
    Abstract:

    In this paper, we present a multi-scale Dictionary Learning paradigm for sparse and redundant signal representations. The appeal of such a Dictionary is obvious-in many cases data naturally comes at different scales. A multi-scale Dictionary should be able to combine the advantages of generic multi-scale representations (such as Wavelets), with the power of learned dictionaries, in capturing the intrinsic characteristics of a family of signals. Using such a Dictionary would allow representing the data in a more efficient, i.e., sparse, manner, allowing applications to take a more global look at the signal. In this paper, we aim to achieve this goal without incurring the costs of an explicit Dictionary with large atoms. The K-SVD using Wavelets approach presented here applies Dictionary Learning in the analysis domain of a fixed multi-scale operator. This way, sub-dictionaries at different data scales, consisting of small atoms, are trained. These dictionaries can then be efficiently used in sparse coding for various image processing applications, potentially outperforming both single-scale trained dictionaries and multi-scale analytic ones. In this paper, we demonstrate this construction and discuss its potential through several experiments performed on fingerprint and coastal scenery images.

Peter L Bartlett - One of the best experts on this subject based on the ideXlab platform.

  • alternating minimization for Dictionary Learning local convergence guarantees
    arXiv: Machine Learning, 2017
    Co-Authors: Niladri S Chatterji, Peter L Bartlett
    Abstract:

    We present theoretical guarantees for an alternating minimization algorithm for the Dictionary Learning/sparse coding problem. The Dictionary Learning problem is to factorize vector samples $y^{1},y^{2},\ldots, y^{n}$ into an appropriate basis (Dictionary) $A^*$ and sparse vectors $x^{1*},\ldots,x^{n*}$. Our algorithm is a simple alternating minimization procedure that switches between $\ell_1$ minimization and gradient descent in alternate steps. Dictionary Learning and specifically alternating minimization algorithms for Dictionary Learning are well studied both theoretically and empirically. However, in contrast to previous theoretical analyses for this problem, we replace a condition on the operator norm (that is, the largest magnitude singular value) of the true underlying Dictionary $A^*$ with a condition on the matrix infinity norm (that is, the largest magnitude term). Our guarantees are under a reasonable generative model that allows for dictionaries with growing operator norms, and can handle an arbitrary level of overcompleteness, while having sparsity that is information theoretically optimal. We also establish upper bounds on the sample complexity of our algorithm.

  • alternating minimization for Dictionary Learning with random initialization
    arXiv: Machine Learning, 2017
    Co-Authors: Niladri S Chatterji, Peter L Bartlett
    Abstract:

    We present theoretical guarantees for an alternating minimization algorithm for the Dictionary Learning/sparse coding problem. The Dictionary Learning problem is to factorize vector samples $y^{1},y^{2},\ldots, y^{n}$ into an appropriate basis (Dictionary) $A^*$ and sparse vectors $x^{1*},\ldots,x^{n*}$. Our algorithm is a simple alternating minimization procedure that switches between $\ell_1$ minimization and gradient descent in alternate steps. Dictionary Learning and specifically alternating minimization algorithms for Dictionary Learning are well studied both theoretically and empirically. However, in contrast to previous theoretical analyses for this problem, we replace the condition on the operator norm (that is, the largest magnitude singular value) of the true underlying Dictionary $A^*$ with a condition on the matrix infinity norm (that is, the largest magnitude term). This not only allows us to get convergence rates for the error of the estimated Dictionary measured in the matrix infinity norm, but also ensures that a random initialization will provably converge to the global optimum. Our guarantees are under a reasonable generative model that allows for dictionaries with growing operator norms, and can handle an arbitrary level of overcompleteness, while having sparsity that is information theoretically optimal. We also establish upper bounds on the sample complexity of our algorithm.

  • alternating minimization for Dictionary Learning with random initialization
    Neural Information Processing Systems, 2017
    Co-Authors: Niladri S Chatterji, Peter L Bartlett
    Abstract:

    We present theoretical guarantees for an alternating minimization algorithm for the Dictionary Learning/sparse coding problem. The Dictionary Learning problem is to factorize vector samples $y^{1},y^{2},\ldots, y^{n}$ into an appropriate basis (Dictionary) $A^*$ and sparse vectors $x^{1*},\ldots,x^{n*}$. Our algorithm is a simple alternating minimization procedure that switches between $\ell_1$ minimization and gradient descent in alternate steps. Dictionary Learning and specifically alternating minimization algorithms for Dictionary Learning are well studied both theoretically and empirically. However, in contrast to previous theoretical analyses for this problem, we replace a condition on the operator norm (that is, the largest magnitude singular value) of the true underlying Dictionary $A^*$ with a condition on the matrix infinity norm (that is, the largest magnitude term). This not only allows us to get convergence rates for the error of the estimated Dictionary measured in the matrix infinity norm, but also ensures that a random initialization will provably converge to the global optimum. Our guarantees are under a reasonable generative model that allows for dictionaries with growing operator norms, and can handle an arbitrary level of overcompleteness, while having sparsity that is information theoretically optimal. We also establish upper bounds on the sample complexity of our algorithm.

David Zhang - One of the best experts on this subject based on the ideXlab platform.

  • a survey of Dictionary Learning algorithms for face recognition
    IEEE Access, 2017
    Co-Authors: Jian Yang, David Zhang
    Abstract:

    During the past several years, as one of the most successful applications of sparse coding and Dictionary Learning, Dictionary-based face recognition has received significant attention. Although some surveys of sparse coding and Dictionary Learning have been reported, there is no specialized survey concerning Dictionary Learning algorithms for face recognition. This paper provides a survey of Dictionary Learning algorithms for face recognition. To provide a comprehensive overview, we not only categorize existing Dictionary Learning algorithms for face recognition but also present details of each category. Since the number of atoms has an important impact on classification performance, we also review the algorithms for selecting the number of atoms. Specifically, we select six typical Dictionary Learning algorithms with different numbers of atoms to perform experiments on face databases. In summary, this paper provides a broad view of Dictionary Learning algorithms for face recognition and advances study in this field. It is very useful for readers to understand the profiles of this subject and to grasp the theoretical rationales and potentials as well as their applicability to different cases of face recognition.

  • a locality constrained and label embedding Dictionary Learning algorithm for image classification
    IEEE Transactions on Neural Networks, 2017
    Co-Authors: Zhengming Li, Jian Yang, Zhihui Lai, Yong Xu, David Zhang
    Abstract:

    Locality and label information of training samples play an important role in image classification. However, previous Dictionary Learning algorithms do not take the locality and label information of atoms into account together in the Learning process, and thus their performance is limited. In this paper, a discriminative Dictionary Learning algorithm, called the locality-constrained and label embedding Dictionary Learning (LCLE-DL) algorithm, was proposed for image classification. First, the locality information was preserved using the graph Laplacian matrix of the learned Dictionary instead of the conventional one derived from the training samples. Then, the label embedding term was constructed using the label information of atoms instead of the classification error term, which contained discriminating information of the learned Dictionary. The optimal coding coefficients derived by the locality-based and label-based reconstruction were effective for image classification. Experimental results demonstrated that the LCLE-DL algorithm can achieve better performance than some state-of-the-art algorithms.

  • sparse representation based fisher discrimination Dictionary Learning for image classification
    International Journal of Computer Vision, 2014
    Co-Authors: Meng Yang, Lei Zhang, Xiangchu Feng, David Zhang
    Abstract:

    The employed Dictionary plays an important role in sparse representation or sparse coding based image reconstruction and classification, while Learning dictionaries from the training data has led to state-of-the-art results in image classification tasks. However, many Dictionary Learning models exploit only the discriminative information in either the representation coefficients or the representation residual, which limits their performance. In this paper we present a novel Dictionary Learning method based on the Fisher discrimination criterion. A structured Dictionary, whose atoms have correspondences to the subject class labels, is learned, with which not only the representation residual can be used to distinguish different classes, but also the representation coefficients have small within-class scatter and big between-class scatter. The classification scheme associated with the proposed Fisher discrimination Dictionary Learning (FDDL) model is consequently presented by exploiting the discriminative information in both the representation residual and the representation coefficients. The proposed FDDL model is extensively evaluated on various image datasets, and it shows superior performance to many state-of-the-art Dictionary Learning methods in a variety of classification tasks.

Larry S Davis - One of the best experts on this subject based on the ideXlab platform.

  • online discriminative Dictionary Learning for visual tracking
    Workshop on Applications of Computer Vision, 2014
    Co-Authors: Fan Yang, Zhuolin Jiang, Larry S Davis
    Abstract:

    Dictionary Learning has been applied to various computer vision problems, such as image restoration, object classification and face recognition. In this work, we propose a tracking framework based on sparse representation and online discriminative Dictionary Learning. By associating Dictionary items with label information, the learned Dictionary is both reconstructive and discriminative, which better distinguishes target objects from the background. During tracking, the best target candidate is selected by a joint decision measure. Reliable tracking results and augmented training samples are accumulated into two sets to update the Dictionary. Both online Dictionary Learning and the proposed joint decision measure are important for the final tracking performance. Experiments show that our approach outperforms several recently proposed trackers.

  • online semi supervised discriminative Dictionary Learning for sparse representation
    Asian Conference on Computer Vision, 2012
    Co-Authors: Guangxiao Zhang, Zhuolin Jiang, Larry S Davis
    Abstract:

    We present an online semi-supervised Dictionary Learning algorithm for classification tasks. Specifically, we integrate the reconstruction error of labeled and unlabeled data, the discriminative sparse-code error, and the classification error into an objective function for online Dictionary Learning, which enhances the Dictionary's representative and discriminative power. In addition, we propose a probabilistic model over the sparse codes of input signals, which allows us to expand the labeled set. As a consequence, the Dictionary and the classifier learned from the enlarged labeled set yield lower generalization error on unseen data. Our approach learns a single Dictionary and a predictive linear classifier jointly. Experimental results demonstrate the effectiveness of our approach in face and object category recognition applications.