The Experts below are selected from a list of 33900 Experts worldwide ranked by ideXlab platform
Rave Harpaz - One of the best experts on this subject based on the ideXlab platform.
-
model based Linear Manifold clustering
2008Co-Authors: Robert M Haralick, Rave HarpazAbstract:A new paradigm of clustering called “Linear Manifold Clustering” which is based on Linear Manifolds is designed, analyzed, and evaluated throughout this thesis. A Linear Manifold is a translated subspace. Linear Manifold clustering seeks to identify groups of points that are embedded in lower dimensional Linear Manifolds. The “birth” of this paradigm of clustering is a consequence of what we believe is a need for an important yet overlooked cluster model, and as a result of what we identify as an acute need to sufficiently address certain clustering requirements that current state of art methods are unable to address. In many problem domains it assumed that Linear models are sufficient enough to describe and capture the data's inherent structure. Yet very few remote attempts have been made to devise clustering methods able to identify or learn mixtures of Linear Manifolds. None of these attempts posed the problem in a model-based statistical setting intended to model and understand the underlying “process” responsible for generating sets of points that lie on lower dimensional Linear Manifolds. In this thesis we introduce a formal stochastic Linear Manifold cluster model. Based on this model we present a series of results and techniques demonstrating the applicability of the Linear Manifold clustering paradigm to a wide range of applications. We show that this model is a generalization of other more common and somewhat limited cluster models. This generalization allows for less assumptions to be imposed on the data, which typically yield biased results, and more freedom for the data to “speak for itself”. An emphasis is put on the paradigms of pattern and correlation clustering and on the application of DNA microarray analysis, where we show that pattern clusters or correlations manifest themselves as Linear Manifolds in the data space. Based on the Linear Manifold cluster model we present two clustering algorithms: one for clustering or learning mixtures of Linear Manifolds, and the other tailored to the application of correlation clustering and DNA micro array analysis. The efficacy of these techniques is demonstrated by a series of experiments on synthetic and real data sets. Most clustering methods focus only on the grouping aspects of clustering and lack the ability to provide any scientific content. In this thesis, we also present two Linear Manifold based modeling techniques that deliver scientific content and with which data can be described. One based on a probabilistic density estimation model with which statistical inference such as predictions can be based upon. The other, a model which describes the Linear dependencies in the data in the form of a set of Linear equations.
-
Linear Manifold clustering in high dimensional spaces by stochastic search
Pattern Recognition, 2007Co-Authors: Robert M Haralick, Rave HarpazAbstract:Classical clustering algorithms are based on the concept that a cluster center is a single point. Clusters which are not compact around a single point are not candidates for classical clustering approaches. In this paper we present a new clustering paradigm in which the cluster center is a Linear Manifold. Clusters are groups of points compact around a Linear Manifold. A Linear Manifold of dimension 0 is a point. So clustering around a center point is a special case of Linear Manifold clustering. Linear Manifold clustering (LMCLUS) identifies subsets of the data which are embedded in arbitrary oriented lower dimensional Linear Manifolds. Minimal subsets of points are repeatedly sampled to construct trial Linear Manifolds of various dimensions. Histograms of the distances of the points to each trial Manifold are computed. The sampling corresponding to the histogram having the best separation between a mode near zero and the rest is selected and the data points are partitioned on the basis of the best separation. The repeated sampling then continues recursively on each block of the partitioned data. A broad evaluation of some 100 experiments over real and synthetic data sets demonstrates the general superiority of this algorithm over any of the competing algorithms in terms of accuracy and computation time. Its expected computational time is Linearly proportional to the data set dimension and data set size. Its accuracy ranges from near 0.90 to 0.99 depending on the experiment and is generally much higher than the accuracy of the competing clustering algorithms.
-
modeling high dimensional probability distributions via Linear Manifold clusters
2007Co-Authors: Rave Harpaz, Robert M HaralickAbstract:One of the ultimate goals of cluster analysis is not only to reveal structure but also to understand it. Most clustering methods focus only on the grouping aspect and do not provide a descriptive model with which the population underlying the data can be described or with which statistical inference such as predictions can be made. Linear Manifold clustering seeks to identify groups of points that lie on lower dimensional Linear Manifolds. In this paper we present a non-parametric density estimation modeling technique by which data that lies in a mixture of Linear Manifolds can be described and with which statistical inference can based on. The efficacy of this technique is demonstrated by a target recognition experiment, where image pixels represented by high-dimensional feature vectors are classified with an error rate close to 0.1 using a probabilistic classifier constructed from mixture models of Linear Manifolds.
-
Linear Manifold clustering
Lecture Notes in Computer Science, 2005Co-Authors: Robert M Haralick, Rave HarpazAbstract:In this paper we describe a new cluster model which is based on the concept of Linear Manifolds. The method identifies subsets of the data which are embedded in arbitrary oriented lower dimensional Linear Manifolds. Minimal subsets of points are repeatedly sampled to construct trial Linear Manifolds of various dimensions. Histograms of the distances of the points to each trial Manifold are computed. The sampling corresponding to the histogram having the best separation between a mode near zero and the rest is selected and the data points are partitioned on the basis of the best separation. The repeated sampling then continues recursively on each block of the partitioned data. A broad evaluation of some hundred experiments over real and synthetic data sets demonstrates the general superiority of this algorithm over any of the competing algorithms in terms of stability, accuracy, and computation time.
Maneesh Sahani - One of the best experts on this subject based on the ideXlab platform.
-
bayesian Manifold learning the locally Linear latent variable model
Neural Information Processing Systems, 2015Co-Authors: Mijung Park, Wittawat Jitkrittum, Ahmad Qamar, Z Szabo, Lars Buesing, Maneesh SahaniAbstract:We introduce the Locally Linear Latent Variable Model (LL-LVM), a probabilistic model for non-Linear Manifold discovery that describes a joint distribution over observations, their Manifold coordinates and locally Linear maps conditioned on a set of neighbourhood relationships. The model allows straightforward variational optimisation of the posterior distribution on coordinates and locally Linear maps from the latent space to the observation space given the data. Thus, the LL-LVM encapsulates the local-geometry preserving intuitions that underlie non-probabilistic methods such as locally Linear embedding (LLE). Its probabilistic semantics make it easy to evaluate the quality of hypothesised neighbourhood relationships, select the intrinsic dimensionality of the Manifold, construct out-of-sample extensions and to combine the Manifold model with additional probabilistic models that capture the structure of coordinates within the Manifold.
-
bayesian Manifold learning the locally Linear latent variable model ll lvm
arXiv: Machine Learning, 2014Co-Authors: Mijung Park, Wittawat Jitkrittum, Ahmad Qamar, Z Szabo, Lars Buesing, Maneesh SahaniAbstract:We introduce the Locally Linear Latent Variable Model (LL-LVM), a probabilistic model for non-Linear Manifold discovery that describes a joint distribution over observations, their Manifold coordinates and locally Linear maps conditioned on a set of neighbourhood relationships. The model allows straightforward variational optimisation of the posterior distribution on coordinates and locally Linear maps from the latent space to the observation space given the data. Thus, the LL-LVM encapsulates the local-geometry preserving intuitions that underlie non-probabilistic methods such as locally Linear embedding (LLE). Its probabilistic semantics make it easy to evaluate the quality of hypothesised neighbourhood relationships, select the intrinsic dimensionality of the Manifold, construct out-of-sample extensions and to combine the Manifold model with additional probabilistic models that capture the structure of coordinates within the Manifold.
Robert M Haralick - One of the best experts on this subject based on the ideXlab platform.
-
inexact mdl for Linear Manifold clusters
International Conference on Pattern Recognition, 2016Co-Authors: Robert M Haralick, Art Diky, Nancy Y KiangAbstract:We present a regularization technique based on the minimum description length (MDL) principle for the Linear Manifold clustering. We suggest an inexact minimum description length method based on describing the data structure as Linear Manifold clusters. We examine the behavior of the proposed method and compare it performance against simulated clustering results of various dimensionality and structure. Finally, we empirically evaluate the proposed technique on a climate data.
-
model based Linear Manifold clustering
2008Co-Authors: Robert M Haralick, Rave HarpazAbstract:A new paradigm of clustering called “Linear Manifold Clustering” which is based on Linear Manifolds is designed, analyzed, and evaluated throughout this thesis. A Linear Manifold is a translated subspace. Linear Manifold clustering seeks to identify groups of points that are embedded in lower dimensional Linear Manifolds. The “birth” of this paradigm of clustering is a consequence of what we believe is a need for an important yet overlooked cluster model, and as a result of what we identify as an acute need to sufficiently address certain clustering requirements that current state of art methods are unable to address. In many problem domains it assumed that Linear models are sufficient enough to describe and capture the data's inherent structure. Yet very few remote attempts have been made to devise clustering methods able to identify or learn mixtures of Linear Manifolds. None of these attempts posed the problem in a model-based statistical setting intended to model and understand the underlying “process” responsible for generating sets of points that lie on lower dimensional Linear Manifolds. In this thesis we introduce a formal stochastic Linear Manifold cluster model. Based on this model we present a series of results and techniques demonstrating the applicability of the Linear Manifold clustering paradigm to a wide range of applications. We show that this model is a generalization of other more common and somewhat limited cluster models. This generalization allows for less assumptions to be imposed on the data, which typically yield biased results, and more freedom for the data to “speak for itself”. An emphasis is put on the paradigms of pattern and correlation clustering and on the application of DNA microarray analysis, where we show that pattern clusters or correlations manifest themselves as Linear Manifolds in the data space. Based on the Linear Manifold cluster model we present two clustering algorithms: one for clustering or learning mixtures of Linear Manifolds, and the other tailored to the application of correlation clustering and DNA micro array analysis. The efficacy of these techniques is demonstrated by a series of experiments on synthetic and real data sets. Most clustering methods focus only on the grouping aspects of clustering and lack the ability to provide any scientific content. In this thesis, we also present two Linear Manifold based modeling techniques that deliver scientific content and with which data can be described. One based on a probabilistic density estimation model with which statistical inference such as predictions can be based upon. The other, a model which describes the Linear dependencies in the data in the form of a set of Linear equations.
-
Linear Manifold clustering in high dimensional spaces by stochastic search
Pattern Recognition, 2007Co-Authors: Robert M Haralick, Rave HarpazAbstract:Classical clustering algorithms are based on the concept that a cluster center is a single point. Clusters which are not compact around a single point are not candidates for classical clustering approaches. In this paper we present a new clustering paradigm in which the cluster center is a Linear Manifold. Clusters are groups of points compact around a Linear Manifold. A Linear Manifold of dimension 0 is a point. So clustering around a center point is a special case of Linear Manifold clustering. Linear Manifold clustering (LMCLUS) identifies subsets of the data which are embedded in arbitrary oriented lower dimensional Linear Manifolds. Minimal subsets of points are repeatedly sampled to construct trial Linear Manifolds of various dimensions. Histograms of the distances of the points to each trial Manifold are computed. The sampling corresponding to the histogram having the best separation between a mode near zero and the rest is selected and the data points are partitioned on the basis of the best separation. The repeated sampling then continues recursively on each block of the partitioned data. A broad evaluation of some 100 experiments over real and synthetic data sets demonstrates the general superiority of this algorithm over any of the competing algorithms in terms of accuracy and computation time. Its expected computational time is Linearly proportional to the data set dimension and data set size. Its accuracy ranges from near 0.90 to 0.99 depending on the experiment and is generally much higher than the accuracy of the competing clustering algorithms.
-
modeling high dimensional probability distributions via Linear Manifold clusters
2007Co-Authors: Rave Harpaz, Robert M HaralickAbstract:One of the ultimate goals of cluster analysis is not only to reveal structure but also to understand it. Most clustering methods focus only on the grouping aspect and do not provide a descriptive model with which the population underlying the data can be described or with which statistical inference such as predictions can be made. Linear Manifold clustering seeks to identify groups of points that lie on lower dimensional Linear Manifolds. In this paper we present a non-parametric density estimation modeling technique by which data that lies in a mixture of Linear Manifolds can be described and with which statistical inference can based on. The efficacy of this technique is demonstrated by a target recognition experiment, where image pixels represented by high-dimensional feature vectors are classified with an error rate close to 0.1 using a probabilistic classifier constructed from mixture models of Linear Manifolds.
-
Linear Manifold clustering
Lecture Notes in Computer Science, 2005Co-Authors: Robert M Haralick, Rave HarpazAbstract:In this paper we describe a new cluster model which is based on the concept of Linear Manifolds. The method identifies subsets of the data which are embedded in arbitrary oriented lower dimensional Linear Manifolds. Minimal subsets of points are repeatedly sampled to construct trial Linear Manifolds of various dimensions. Histograms of the distances of the points to each trial Manifold are computed. The sampling corresponding to the histogram having the best separation between a mode near zero and the rest is selected and the data points are partitioned on the basis of the best separation. The repeated sampling then continues recursively on each block of the partitioned data. A broad evaluation of some hundred experiments over real and synthetic data sets demonstrates the general superiority of this algorithm over any of the competing algorithms in terms of stability, accuracy, and computation time.
Mijung Park - One of the best experts on this subject based on the ideXlab platform.
-
bayesian Manifold learning the locally Linear latent variable model
Neural Information Processing Systems, 2015Co-Authors: Mijung Park, Wittawat Jitkrittum, Ahmad Qamar, Z Szabo, Lars Buesing, Maneesh SahaniAbstract:We introduce the Locally Linear Latent Variable Model (LL-LVM), a probabilistic model for non-Linear Manifold discovery that describes a joint distribution over observations, their Manifold coordinates and locally Linear maps conditioned on a set of neighbourhood relationships. The model allows straightforward variational optimisation of the posterior distribution on coordinates and locally Linear maps from the latent space to the observation space given the data. Thus, the LL-LVM encapsulates the local-geometry preserving intuitions that underlie non-probabilistic methods such as locally Linear embedding (LLE). Its probabilistic semantics make it easy to evaluate the quality of hypothesised neighbourhood relationships, select the intrinsic dimensionality of the Manifold, construct out-of-sample extensions and to combine the Manifold model with additional probabilistic models that capture the structure of coordinates within the Manifold.
-
bayesian Manifold learning the locally Linear latent variable model ll lvm
arXiv: Machine Learning, 2014Co-Authors: Mijung Park, Wittawat Jitkrittum, Ahmad Qamar, Z Szabo, Lars Buesing, Maneesh SahaniAbstract:We introduce the Locally Linear Latent Variable Model (LL-LVM), a probabilistic model for non-Linear Manifold discovery that describes a joint distribution over observations, their Manifold coordinates and locally Linear maps conditioned on a set of neighbourhood relationships. The model allows straightforward variational optimisation of the posterior distribution on coordinates and locally Linear maps from the latent space to the observation space given the data. Thus, the LL-LVM encapsulates the local-geometry preserving intuitions that underlie non-probabilistic methods such as locally Linear embedding (LLE). Its probabilistic semantics make it easy to evaluate the quality of hypothesised neighbourhood relationships, select the intrinsic dimensionality of the Manifold, construct out-of-sample extensions and to combine the Manifold model with additional probabilistic models that capture the structure of coordinates within the Manifold.
Takanori Maehara - One of the best experts on this subject based on the ideXlab platform.
-
Revisiting Graph Neural Networks: All We Have is Low-Pass Filters
arXiv: Machine Learning, 2019Co-Authors: Hoang Nt, Takanori MaeharaAbstract:Graph neural networks have become one of the most important techniques to solve machine learning problems on graph-structured data. Recent work on vertex classification proposed deep and distributed learning models to achieve high performance and scalability. However, we find that the feature vectors of benchmark datasets are already quite informative for the classification task, and the graph structure only provides a means to denoise the data. In this paper, we develop a theoretical framework based on graph signal processing for analyzing graph neural networks. Our results indicate that graph neural networks only perform low-pass filtering on feature vectors and do not have the non-Linear Manifold learning property. We further investigate their resilience to feature noise and propose some insights on GCN-based graph neural network design.