The Experts below are selected from a list of 150603 Experts worldwide ranked by ideXlab platform
Haesun Park - One of the best experts on this subject based on the ideXlab platform.
-
a comparison of generalized linear Discriminant Analysis algorithms
Pattern Recognition, 2008Co-Authors: Cheong Hee Park, Haesun ParkAbstract:Linear Discriminant Analysis (LDA) is a dimension reduction method which finds an optimal linear transformation that maximizes the class separability. However, in undersampled problems where the number of data samples is smaller than the dimension of data space, it is difficult to apply LDA due to the singularity of scatter matrices caused by high dimensionality. In order to make LDA applicable, several generalizations of LDA have been proposed recently. In this paper, we present theoretical and algorithmic relationships among several generalized LDA algorithms and compare their computational complexities and performances in text classification and face recognition. Towards a practical dimension reduction method for high dimensional data, an efficient algorithm is proposed, which reduces the computational complexity greatly while achieving competitive prediction accuracies. We also present nonlinear extensions of these LDA algorithms based on kernel methods. It is shown that a generalized eigenvalue problem can be formulated in the kernel-based feature space, and generalized LDA algorithms are applied to solve the generalized eigenvalue problem, resulting in nonlinear Discriminant Analysis. Performances of these linear and nonlinear Discriminant Analysis algorithms are compared extensively.
-
nonlinear Discriminant Analysis using kernel functions and the generalized singular value decomposition
SIAM Journal on Matrix Analysis and Applications, 2005Co-Authors: Cheong Hee Park, Haesun ParkAbstract:Linear Discriminant Analysis (LDA) has been widely used for linear dimension reduction. However, LDA has limitations in that one of the scatter matrices is required to be nonsingular and the nonlinearly clustered structure is not easily captured. In order to overcome the problems caused by the singularity of the scatter matrices, a generalization of LDA based on the generalized singular value decomposition (GSVD) was recently developed. In this paper, we propose a nonlinear Discriminant Analysis based on the kernel method and the GSVD. The GSVD is applied to solve the generalized eigenvalue problem which is formulated in the feature space defined by a nonlinear mapping through kernel functions. Our GSVD-based kernel Discriminant Analysis is theoretically compared with other kernel-based nonlinear Discriminant Analysis algorithms. The experimental results show that our method is an effective nonlinear dimension reduction method.
-
fingerprint classification using fast fourier transform and nonlinear Discriminant Analysis
Pattern Recognition, 2005Co-Authors: Cheong Hee Park, Haesun ParkAbstract:In this paper, we present a new approach for fingerprint classification based on discrete Fourier transform (DFT) and nonlinear Discriminant Analysis. Utilizing the DFT and directional filters, a reliable and efficient directional image is constructed from each fingerprint image, and then nonlinear Discriminant Analysis is applied to the constructed directional images, reducing the dimension dramatically and extracting the Discriminant features. The proposed method explores the capability of DFT and directional filtering in dealing with low-quality images and the effectiveness of nonlinear feature extraction method in fingerprint classification. Experimental results demonstrates competitive performance compared with other published results.
Trevor Hastie - One of the best experts on this subject based on the ideXlab platform.
-
sparse Discriminant Analysis
Technometrics, 2011Co-Authors: Line Katrine Harder Clemmensen, Daniela Witten, Trevor Hastie, Bjarne Kjær ErsbøllAbstract:We consider the problem of performing interpretable classification in the high-dimensional setting, in which the number of features is very large and the number of observations is limited. This setting has been studied extensively in the chemometrics literature, and more recently has become commonplace in biological and medical applications. In this setting, a traditional approach involves performing feature selection before classification. We propose sparse Discriminant Analysis, a method for performing linear Discriminant Analysis with a sparseness criterion imposed such that classification and feature selection are performed simultaneously. Sparse Discriminant Analysis is based on the optimal scoring interpretation of linear Discriminant Analysis, and can be extended to perform sparse discrimination via mixtures of Gaussians if boundaries between classes are nonlinear or if subgroups are present within each class. Our proposal also provides low-dimensional views of the discriminative directions.
-
regularized linear Discriminant Analysis and its application in microarrays
Biostatistics, 2007Co-Authors: Trevor Hastie, Robert TibshiraniAbstract:SUMMARY In this paper, we introduce a modified version of linear Discriminant Analysis, called the “shrunken centroids regularized Discriminant Analysis” (SCRDA). This method generalizes the idea of the “nearest shrunken centroids” (NSC) (Tibshirani and others, 2003) into the classical Discriminant Analysis. The SCRDA method is specially designed for classification problems in high dimension low sample size situations, for example, microarray data. Through both simulated data and real life data, it is shown that this method performs very well in multivariate classification problems, often outperforms the PAM method (using the NSC algorithm) and can be as competitive as the support vector machines classifiers. It is also suitable for feature elimination purpose and can be used as gene selection method. The open source R package for this method (named “rda”) is available on CRAN (http://www.r-project.org) for download and testing.
-
Functional linear Discriminant Analysis for irregularly sampled curves
Journal of the Royal Statistical Society: Series B (Statistical Methodology), 2001Co-Authors: Gareth M. James, Trevor HastieAbstract:We introduce a technique for extending the classical method of linear Discriminant Analysis (LDA) to data sets where the predictor variables are curves or functions. This procedure, which we call functional linear Discriminant Analysis (FLDA), is particularly useful when only fragments of the curves are observed. All the techniques associated with LDA can be extended for use with FLDA. In particular FLDA can be used to produce classifications on new (test) curves, give an estimate of the Discriminant function between classes and provide a one- or two-dimensional pictorial representation of a set of curves. We also extend this procedure to provide generalizations of quadratic and regularized Discriminant Analysis.
-
Discriminant Analysis by gaussian mixtures
Journal of the royal statistical society series b-methodological, 1996Co-Authors: Trevor Hastie, Robert TibshiraniAbstract:Fisher-Rao linear Discriminant Analysis (LDA) is a valuable tool for multigroup classification. LDA is equivalent to maximum likelihood classification assuming Gaussian distributions for each class. In this paper, we fit Gaussian mixtures to each class to facilitate effective classification in non-normal settings, especially when the classes are clustered. Low dimensional views are an important by-product of LDA-our new techniques inherit this feature. We can control the within-class spread of the subclass centres relative to the between-class spread. Our technique for fitting these models permits a natural blend with nonparametric versions of LDA.
-
penalized Discriminant Analysis
Annals of Statistics, 1995Co-Authors: Trevor Hastie, Andreas BujaAbstract:Fisher's linear Discriminant Analysis (LDA) is a popular data-analytic tool for studying the relationship between a set of predictors and a categorical response. In this paper we describe a penalized version of LDA. It is designed for situations in which there are many highly correlated predictors, such as those obtained by discretizing a function, or the grey-scale values of the pixels in a series of images. In cases such as these it is natural, efficient and sometimes essential to impose a spatial smoothness constraint on the coefficients, both for improved prediction performance and interpretability. We cast the classification problem into a regression framework via optimal scoring. Using this, our proposal facilitates the use of any penalized regression technique in the classification setting. The technique is illustrated with examples in speech recognition and handwritten character recognition.
Shan Zeng - One of the best experts on this subject based on the ideXlab platform.
-
multiple kernel based Discriminant Analysis via support vectors for dimension reduction
IEEE Access, 2019Co-Authors: Shan Zeng, Liang Jiang, Chongjun Gao, Xiuying Wang, Dagan FengAbstract:Kernel-based Discriminant Analysis is an effective nonlinear mechanism for pattern Analysis. Conventional kernel-based Discriminant Analysis mainly based on a single kernel function may be insufficient when dealing with datasets with complicated geometric structures. A combination of multiple kernels is able to represent the complementary information of the original data from multiple views and thereby improves recognition performance. However, the Discriminant Analysis methods based on the combination of multiple kernels face the challenges of optimizing the weights of the “base kernels” and the heavy computational burden. To address these challenges, this paper proposes a novel multi-kernel Discriminant Analysis method based on support vectors (MKDASV) to represent the data structure more effectively by incorporating the between-class and within-class information. First, the multi-kernel SVM algorithm is utilized to obtain the weight of each “base kernel” and the support vectors; and then the criteria of Discriminant Analysis method are constructed by taking into account both the margin maximizing classification theory of SVM and the expression of the within-class scatter in LDA algorithm; and finally, to effectively reduce the amount of computation, only the support vectors are used as the training samples to participate in the dimensionality reduction operation. The experimental results on six standard databases validated that our proposed method outperformed the other five methods in terms of classification accuracy and the computational efficiency as well.
-
multiple kernel fuzzy Discriminant Analysis for hyperspectral imaging classification
IEEE International Conference on Fuzzy Systems, 2017Co-Authors: Shan Zeng, Jun Bai, Liang Jiang, Zhen KangAbstract:The classical fuzzy Discriminant Analysis with kernel methods (KFDA) is an effective method of solving nonlinearity pattern Analysis problem. In some complicated cases, the kernel machine constituted by a single kernel function is not able to meet some practical application requirements, such as heterogeneous information or unnormalised data, non-flat distribution of samples, etc. By searching for an appropriate linear combination of base kernel functions or matrices, multiple kernel learning (MKL) is able to improve the performance in some extent. So it is a necessary choice to introduce multiple kernel learning into KFDA in order to get better results. In this study, multiple kernel fuzzy Discriminant Analysis (MKFDA) is proposed. Our method obtains the projection matrix from fuzzy Discriminant Analysis with multiple kernel, and then feature extraction and classification are made based on the projection matrix. The experiment on the AVIRIS image was performed, and the results showed that the performance of fuzzy Discriminant Analysis with multiple kernels is better than that of fuzzy Discriminant with single kernel for the Hyperspectral images' feature extraction and classification.
Wilfried Philips - One of the best experts on this subject based on the ideXlab platform.
-
semisupervised local Discriminant Analysis for feature extraction in hyperspectral images
IEEE Transactions on Geoscience and Remote Sensing, 2013Co-Authors: Wenzhi Liao, Aleksandra Pizurica, Paul Scheunders, Wilfried Philips, Youguo PiAbstract:We propose a novel semisupervised local Discriminant Analysis method for feature extraction in hyperspectral remote sensing imagery, with improved performance in both ill-posed and poor-posed conditions. The proposed method combines unsupervised methods (local linear feature extraction methods and supervised method (linear Discriminant Analysis) in a novel framework without any free parameters. The underlying idea is to design an optimal projection matrix, which preserves the local neighborhood information inferred from unlabeled samples, while simultaneously maximizing the class discrimination of the data inferred from the labeled samples. Experimental results on four real hyperspectral images demonstrate that the proposed method compares favorably with conventional feature extraction methods.
-
semisupervised local Discriminant Analysis for feature extraction in hyperspectral images
IEEE Transactions on Geoscience and Remote Sensing, 2013Co-Authors: Wenzhi Liao, Aleksandra Pizurica, Paul Scheunders, Wilfried PhilipsAbstract:We propose a novel semisupervised local Discriminant Analysis method for feature extraction in hyperspectral remote sensing imagery, with improved performance in both ill-posed and poor-posed conditions. The proposed method combines unsupervised methods (local linear feature extraction methods and supervised method (linear Discriminant Analysis) in a novel framework without any free parameters. The underlying idea is to design an optimal projection matrix, which preserves the local neighborhood information inferred from unlabeled samples, while simultaneously maximizing the class discrimination of the data inferred from the labeled samples. Experimental results on four real hyperspectral images demonstrate that the proposed method compares favorably with conventional feature extraction methods.
Dagan Feng - One of the best experts on this subject based on the ideXlab platform.
-
multiple kernel based Discriminant Analysis via support vectors for dimension reduction
IEEE Access, 2019Co-Authors: Shan Zeng, Liang Jiang, Chongjun Gao, Xiuying Wang, Dagan FengAbstract:Kernel-based Discriminant Analysis is an effective nonlinear mechanism for pattern Analysis. Conventional kernel-based Discriminant Analysis mainly based on a single kernel function may be insufficient when dealing with datasets with complicated geometric structures. A combination of multiple kernels is able to represent the complementary information of the original data from multiple views and thereby improves recognition performance. However, the Discriminant Analysis methods based on the combination of multiple kernels face the challenges of optimizing the weights of the “base kernels” and the heavy computational burden. To address these challenges, this paper proposes a novel multi-kernel Discriminant Analysis method based on support vectors (MKDASV) to represent the data structure more effectively by incorporating the between-class and within-class information. First, the multi-kernel SVM algorithm is utilized to obtain the weight of each “base kernel” and the support vectors; and then the criteria of Discriminant Analysis method are constructed by taking into account both the margin maximizing classification theory of SVM and the expression of the within-class scatter in LDA algorithm; and finally, to effectively reduce the amount of computation, only the support vectors are used as the training samples to participate in the dimensionality reduction operation. The experimental results on six standard databases validated that our proposed method outperformed the other five methods in terms of classification accuracy and the computational efficiency as well.