The Experts below are selected from a list of 53616 Experts worldwide ranked by ideXlab platform
Marco Loog - One of the best experts on this subject based on the ideXlab platform.
-
Implicitly Constrained Semi-Supervised linear discriminant analysis
arXiv: Machine Learning, 2014Co-Authors: Jesse H. Krijthe, Marco LoogAbstract:Semi-supervised learning is an important and active topic of research in pattern recognition. For classification using linear discriminant analysis specifically, several semi-supervised variants have been proposed. Using any one of these methods is not guaranteed to outperform the supervised classifier which does not take the additional unlabeled data into account. In this work we compare traditional Expectation Maximization type approaches for semi-supervised linear discriminant analysis with approaches based on intrinsic constraints and propose a new principled approach for semi-supervised linear discriminant analysis, using so-called implicit constraints. We explore the relationships between these methods and consider the question if and in what sense we can expect improvement in performance over the supervised procedure. The constraint based approaches are more robust to misspecification of the model, and may outperform alternatives that make more assumptions on the data, in terms of the log-likelihood of unseen objects.
-
semi supervised linear discriminant analysis through moment constraint parameter estimation
Pattern Recognition Letters, 2014Co-Authors: Marco LoogAbstract:A semi-supervised version of classical linear discriminant analysis is presented. As opposed to most current approaches to semi-supervised learning, no additional extrinsic assumptions are made to tie information coming from labeled and unlabeled data together. Our approach exploits the fact that the parameters that are to be estimated fulfill particular relations, intrinsic to the classifier, that link label-dependent with label-independent quantities. In this way, the latter type of parameters, which can be estimated based on unlabeled data, impose constraints on the former and lead to a reduction in variability of the label dependent estimates. As a result, the performance of our semi-supervised linear discriminant is typically expected to improve over that of its regular supervised match. Possibly more important, our semi-supervised linear discriminant analysis does not show the severe deteriorations other approaches frequently display with increasing numbers of unlabeled data. This work recapitulates, corrects, extends, and revises our previous work that has been published as part of the First IAPR TC3 Workshop on Partially Supervised Learning. The main novelty it provides over our earlier work is an affine invariant approach to semi-supervised learning befitting linear discriminant analysis. Besides, more elaborate and convincing experimental evidence of the potential of our general approach is provided. We essentially believe that the general principle of intrinsic constraints is of interest as such and may inspire other novel semi-supervised methods.
Jesse H. Krijthe - One of the best experts on this subject based on the ideXlab platform.
-
Implicitly Constrained Semi-Supervised linear discriminant analysis
arXiv: Machine Learning, 2014Co-Authors: Jesse H. Krijthe, Marco LoogAbstract:Semi-supervised learning is an important and active topic of research in pattern recognition. For classification using linear discriminant analysis specifically, several semi-supervised variants have been proposed. Using any one of these methods is not guaranteed to outperform the supervised classifier which does not take the additional unlabeled data into account. In this work we compare traditional Expectation Maximization type approaches for semi-supervised linear discriminant analysis with approaches based on intrinsic constraints and propose a new principled approach for semi-supervised linear discriminant analysis, using so-called implicit constraints. We explore the relationships between these methods and consider the question if and in what sense we can expect improvement in performance over the supervised procedure. The constraint based approaches are more robust to misspecification of the model, and may outperform alternatives that make more assumptions on the data, in terms of the log-likelihood of unseen objects.
Xiaoyan Wang - One of the best experts on this subject based on the ideXlab platform.
-
Regularized orthogonal linear discriminant analysis
Pattern Recognition, 2012Co-Authors: Wai-ki Ching, Delin Chu, Li-zhi Liao, Xiaoyan WangAbstract:In this paper the regularized orthogonal linear discriminant analysis (ROLDA) is studied. The major issue of the regularized linear discriminant analysis is to choose an appropriate regularization parameter. In existing regularized linear discriminant analysis methods, they all select the ''best'' regularization parameter from a given parameter candidate set by using cross-validation for classification. An obvious limitation of such regularized linear discriminant analysis methods is that it is not clear how to choose an appropriate candidate set. Therefore, up to now, there is no concrete mathematical theory available in selecting an appropriate regularization parameter in practical applications of the regularized linear discriminant analysis. The present work is to fill this gap. Here we derive the mathematical relationship between orthogonal linear discriminant analysis and the regularized orthogonal linear discriminant analysis first, and then by means of this relationship we find a mathematical criterion for selecting the regularization parameter in ROLDA and consequently we develop a new regularized orthogonal linear discriminant analysis method, in which no candidate set of regularization parameter is needed. The effectiveness of our proposed regularized orthogonal linear discriminant analysis is illustrated by some real-world data sets.
Tongming Yin - One of the best experts on this subject based on the ideXlab platform.
-
Fast orthogonal linear discriminant analysis with application to image classification
Neurocomputing, 2015Co-Authors: Tongming YinAbstract:Compared to linear discriminant analysis (LDA), its orthogonalized version is a more effective statistical learning tool for dimension reduction, which devotes to better separating the data points from different classes in the lower-dimensional subspace. However, existing orthogonalized LDA techniques suffer from various drawbacks, including the requirement for expensive computing time. This paper develops an efficient orthogonal dimension reduction approach, referred to as fast orthogonal linear discriminant analysis (FOLDA), which is based on existing orthogonal linear discriminant analysis (OLDA) algorithms. However, different from previous efforts, the new approach applies the QR decomposition and the regression to solve for a new orthogonal projection vector at each iteration, leading to the by far cheaper computational cost. FOLDA achieves comparable recognition rate to existing OLDA algorithms due to the incorporation of the idea and spirit behind the latter ones. Experimental results on image databases, such as MINST, COIL20, MEPG-7 and OUTEX, show the effectiveness and efficiency of our algorithm.
Yan Chen - One of the best experts on this subject based on the ideXlab platform.
-
Robust Sparse linear discriminant analysis
IEEE Transactions on Circuits and Systems for Video Technology, 2019Co-Authors: Jie Wen, Xiaozhao Fang, Jinrong Cui, Lunke Fei, Ke Yan, Yan ChenAbstract:linear discriminant analysis (LDA) is a very popular supervised feature extraction method and has been extended to different variants. However, classical LDA has the following problems: 1) The obtained discriminant projection does not have good interpretability for features; 2) LDA is sensitive to noise; and 3) LDA is sensitive to the selection of number of projection directions. In this paper, a novel feature extraction method called robust sparse linear discriminant analysis (RSLDA) is proposed to solve the above problems. Specifically, RSLDA adaptively selects the most discriminative features for discriminant analysis by introducing the $l_{2,1}$ norm. An orthogonal matrix and a sparse matrix are also simultaneously introduced to guarantee that the extracted features can hold the main energy of the original data and enhance the robustness to noise, and thus RSLDA has the potential to perform better than other discriminant methods. Extensive experiments on six databases demonstrate that the proposed method achieves the competitive performance compared with other state-of-the-art feature extraction methods. Moreover, the proposed method is robust to the noisy data.