The Experts below are selected from a list of 351 Experts worldwide ranked by ideXlab platform

Idris A Eckley - One of the best experts on this subject based on the ideXlab platform.

  • minimum spectral connectivity projection pursuit divisive clustering using optimal projections for spectral clustering
    Statistics and Computing, 2019
    Co-Authors: David P Hofmeyr, Nicos G Pavlidis, Idris A Eckley
    Abstract:

    We study the problem of determining the optimal low-dimensional projection for maximising the separability of a binary partition of an unlabelled dataset, as measured by spectral graph theory. This is achieved by finding projections which minimise the second eigenvalue of the graph Laplacian of the projected data, which corresponds to a non-convex, non-smooth optimisation problem. We show that the optimal univariate projection based on spectral connectivity converges to the vector normal to the Maximum Margin Hyperplane through the data, as the scaling parameter is reduced to zero. This establishes a connection between connectivity as measured by spectral graph theory and maximal Euclidean separation. The computational cost associated with each eigen problem is quadratic in the number of data. To mitigate this issue, we propose an approximation method using microclusters with provable approximation error bounds. Combining multiple binary partitions within a divisive hierarchical model allows us to construct clustering solutions admitting clusters with varying scales and lying within different subspaces. We evaluate the performance of the proposed method on a large collection of benchmark datasets and find that it compares favourably with existing methods for projection pursuit and dimension reduction for data clustering. Applying the proposed approach for a decreasing sequence of scaling parameters allows us to obtain large Margin clustering solutions, which are found to be competitive with those from dedicated Maximum Margin clustering algorithms.

Manfred K. Warmuth - One of the best experts on this subject based on the ideXlab platform.

  • Maximizing the Margin with Boosting Gunnar Rätsch ¡
    2008
    Co-Authors: Manfred K. Warmuth
    Abstract:

    Abstract. AdaBoost produces a linear combination of weak hypotheses. It has been observed that the generalization error of the algorithm continues to improve even after all examples are classified correctly by the current linear combination, i.e. by a Hyperplane in feature space spanned by the weak hypotheses. The improvement is attributed to the experimental observation that the distances (Margins) of the examples to the separating Hyperplane are increasing even when the training error is already zero, that is all examples are on the correct side of the Hyperplane. We give an iterative version of AdaBoost that explicitly maximizes the minimum Margin of the examples. We bound the number of iterations and the number of hypotheses used in the final linear combination which approximates the Maximum Margin Hyperplane with a certain precision. Our modified algorithm essentially retains the exponential convergence properties of AdaBoost and our result does not depend on the size of the hypothesis class.

  • Active learning with support vector machines in the drug discovery process
    Journal of Chemical Information and Computer Sciences, 2003
    Co-Authors: Manfred K. Warmuth, Santosh Putta, M. Mathieson, Jun Liao, Gunnar Ratsch, Christian Lemmen
    Abstract:

    We investigate the following data mining problem from computer-aided drug design: From a large collection of compounds, find those that bind to a target molecule in as few iterations of biochemical testing as possible. In each iteration a comparatively small batch of compounds is screened for binding activity toward this target. We employed the so-called "active learning paradigm" from Machine Learning for selecting the successive batches. Our main selection strategy is based on the Maximum Margin Hyperplane-generated by "Support Vector Machines". This Hyperplane separates the current set of active from the inactive compounds and has the largest possible distance from any labeled compound. We perform a thorough comparative study of various other selection strategies on data sets provided by DuPont Pharmaceuticals and show that the strategies based on the Maximum Margin Hyperplane clearly outperform the simpler ones.

  • Support Vector Machines for Active Learning in the Drug Discovery Process
    2003
    Co-Authors: Manfred K. Warmuth, Santosh Putta, M. Mathieson, Jun Liao, Gunnar Ratsch, Christian Lemmen
    Abstract:

    We investigate the following data mining problem from computeraided drug design: From a large collection of compounds, find those that bind to a target molecule in as few iterations of biochemical testing as possible. In each iteration a comparatively small batch of compounds is screened for binding activity towards this target. We employed the so-called "active learning paradigm" from Machine Learning for selecting the successive batches. Our main selection strategy is based on the Maximum Margin Hyperplane -- generated by "Support Vector Machines". This Hyperplane separates the current set of active from the inactive compounds and has the largest possible distance from any labeled compound

  • Maximizing the Margin with Boosting
    2002
    Co-Authors: Gunnar Ratsch, Manfred K. Warmuth
    Abstract:

    AdaBoost produces a linear combination of weak hypotheses. It has been observed that the generalization error of the algorithm continues to improve even after all examples are classified correctly by the current linear combination, i.e. by a Hyperplane in feature space spanned by the weak hypotheses. The improvement is attributed to the experimental observation that the distances (Margins) of the examples to the separating Hyperplane are increasing even when the training error is already zero, that is all examples are on the correct side of the Hyperplane. We give an iterative version of AdaBoost that explicitly maximizes the minimum Margin of the examples. We bound the number of iterations and the number of hypotheses used in the final linear combination which approximates the Maximum Margin Hyperplane with a certain precision. Our modified algorithm essentially retains the exponential convergence properties of AdaBoost and our result does not depend on the size of the hypothesis class

  • Active Learning and Feature Selection in the Drug Discovery Process
    2002
    Co-Authors: Manfred K. Warmuth
    Abstract:

    Non-technical: In collaboration with the computational chemists at Telik, we will develop and apply novel approaches of Machine Learning to the charac-terization and classification of organic molecules with respect to their potential as pharmaceutical agents. In preliminary research we have already shown that our methods greatly improve the efficiency of the drug discovery cycle. In particular, we will develop search methods that identify small sets of chemical features of the compounds that are likely to be responsible for the relevant pharmaceutical properties. Technical: We propose to use modern Machine Learning techniques to help speed up the drug discovery cycle. Candidate compounds are repre-sented as high-dimensional descriptor vectors. The algorithms are to decide which batch of compounds should be tested next and which features are responsible for the activity of the compounds. We use the Maximum Margin Hyperplane separating the labeled compounds for se-lecting the next batch of unlabeled compounds. An alternate method based on the Voted Perceptron is more suitable for high-dimensional data. We also determine small sets of relevant features using the Max-imum Entropy principle

Daniel D Lee - One of the best experts on this subject based on the ideXlab platform.

  • University of Pennsylvania Multiplicative updates for nonnegative quadratic programming in support vector machines
    2008
    Co-Authors: Fei Sha, Lawrence K Saul, Daniel D Lee
    Abstract:

    We derive multiplicative updates for solving the nonnegative quadratic programming problem in support vector machines (SVMs). The updates have a simple closed form, and we prove that they converge monotonically to the solution of the Maximum Margin Hyperplane. The updates optimize the traditionally proposed objective function for SVMs. They do not involve any heuristics such as choosing a learning rate or deciding which variables to update at each iteration. They can be used to adjust all the quadratic programming variables in parallel with a guarantee of improvement at each iteration. We analyze the asymptotic convergence of the updates and show that the coefficients of non-support vectors decay geometrically to zero at a rate that depends on their Margins. In practice, the updates converge very rapidly to good classifiers.

  • multiplicative updates for nonnegative quadratic programming in support vector machines
    Neural Information Processing Systems, 2002
    Co-Authors: Fei Sha, Lawrence K Saul, Daniel D Lee
    Abstract:

    We derive multiplicative updates for solving the nonnegative quadratic programming problem in support vector machines (SVMs). The updates have a simple closed form, and we prove that they converge monotonically to the solution of the Maximum Margin Hyperplane. The updates optimize the traditionally proposed objective function for SVMs. They do not involve any heuristics such as choosing a learning rate or deciding which variables to update at each iteration. They can be used to adjust all the quadratic programming variables in parallel with a guarantee of improvement at each iteration. We analyze the asymptotic convergence of the updates and show that the coefficients of non-support vectors decay geometrically to zero at a rate that depends on their Margins. In practice, the updates converge very rapidly to good classifiers.

David P Hofmeyr - One of the best experts on this subject based on the ideXlab platform.

  • minimum spectral connectivity projection pursuit divisive clustering using optimal projections for spectral clustering
    Statistics and Computing, 2019
    Co-Authors: David P Hofmeyr, Nicos G Pavlidis, Idris A Eckley
    Abstract:

    We study the problem of determining the optimal low-dimensional projection for maximising the separability of a binary partition of an unlabelled dataset, as measured by spectral graph theory. This is achieved by finding projections which minimise the second eigenvalue of the graph Laplacian of the projected data, which corresponds to a non-convex, non-smooth optimisation problem. We show that the optimal univariate projection based on spectral connectivity converges to the vector normal to the Maximum Margin Hyperplane through the data, as the scaling parameter is reduced to zero. This establishes a connection between connectivity as measured by spectral graph theory and maximal Euclidean separation. The computational cost associated with each eigen problem is quadratic in the number of data. To mitigate this issue, we propose an approximation method using microclusters with provable approximation error bounds. Combining multiple binary partitions within a divisive hierarchical model allows us to construct clustering solutions admitting clusters with varying scales and lying within different subspaces. We evaluate the performance of the proposed method on a large collection of benchmark datasets and find that it compares favourably with existing methods for projection pursuit and dimension reduction for data clustering. Applying the proposed approach for a decreasing sequence of scaling parameters allows us to obtain large Margin clustering solutions, which are found to be competitive with those from dedicated Maximum Margin clustering algorithms.

Christian Lemmen - One of the best experts on this subject based on the ideXlab platform.

  • Active learning with support vector machines in the drug discovery process
    Journal of Chemical Information and Computer Sciences, 2003
    Co-Authors: Manfred K. Warmuth, Santosh Putta, M. Mathieson, Jun Liao, Gunnar Ratsch, Christian Lemmen
    Abstract:

    We investigate the following data mining problem from computer-aided drug design: From a large collection of compounds, find those that bind to a target molecule in as few iterations of biochemical testing as possible. In each iteration a comparatively small batch of compounds is screened for binding activity toward this target. We employed the so-called "active learning paradigm" from Machine Learning for selecting the successive batches. Our main selection strategy is based on the Maximum Margin Hyperplane-generated by "Support Vector Machines". This Hyperplane separates the current set of active from the inactive compounds and has the largest possible distance from any labeled compound. We perform a thorough comparative study of various other selection strategies on data sets provided by DuPont Pharmaceuticals and show that the strategies based on the Maximum Margin Hyperplane clearly outperform the simpler ones.

  • Support Vector Machines for Active Learning in the Drug Discovery Process
    2003
    Co-Authors: Manfred K. Warmuth, Santosh Putta, M. Mathieson, Jun Liao, Gunnar Ratsch, Christian Lemmen
    Abstract:

    We investigate the following data mining problem from computeraided drug design: From a large collection of compounds, find those that bind to a target molecule in as few iterations of biochemical testing as possible. In each iteration a comparatively small batch of compounds is screened for binding activity towards this target. We employed the so-called "active learning paradigm" from Machine Learning for selecting the successive batches. Our main selection strategy is based on the Maximum Margin Hyperplane -- generated by "Support Vector Machines". This Hyperplane separates the current set of active from the inactive compounds and has the largest possible distance from any labeled compound

  • active learning in the drug discovery process
    Neural Information Processing Systems, 2001
    Co-Authors: Manfred K. Warmuth, M. Mathieson, Jun Liao, Gunnar Ratsch, Christian Lemmen
    Abstract:

    We investigate the following data mining problem from Computational Chemistry: From a large data set of compounds, find those that bind to a target molecule in as few iterations of biological testing as possible. In each iteration a comparatively small batch of compounds is screened for binding to the target. We apply active learning techniques for selecting the successive batches. One selection strategy picks unlabeled examples closest to the Maximum Margin Hyperplane. Another produces many weight vectors by running perceptrons over multiple permutations of the data. Each weight vector votes with its ± prediction and we pick the unlabeled examples for which the prediction is most evenly split between + and -. For a third selection strategy note that each unlabeled example bisects the version space of consistent weight vectors. We estimate the volume on both sides of the split by bouncing a billiard through the version space and select un-labeled examples that cause the most even split of the version space. We demonstrate that on two data sets provided by DuPont Pharmaceuticals that all three selection strategies perform comparably well and are much better than selecting random batches for testing.