The Experts below are selected from a list of 10239 Experts worldwide ranked by ideXlab platform

Andrew Mccallum - One of the best experts on this subject based on the ideXlab platform.

  • Alternating Projections for Learning with Expectation Constraints
    arXiv: Learning, 2012
    Co-Authors: Kedar Bellare, Gregory Druck, Andrew Mccallum
    Abstract:

    We present an objective function for learning with unlabeled data that utilizes auxiliary Expectation constraints. We optimize this objective function using a procedure that alternates between information and moment projections. Our method provides an alternate interpretation of the posterior regularization framework (Graca et al., 2008), maintains uncertainty during optimization unlike constraint-driven learning (Chang et al., 2007), and is more efficient than Generalized Expectation criteria (Mann & McCallum, 2008). Applications of this framework include minimally supervised learning, semisupervised learning, and learning with constraints that are more expressive than the underlying model. In experiments, we demonstrate comparable accuracy to Generalized Expectation criteria for minimally supervised learning, and use expressive structural constraints to guide semi-supervised learning, providing a 3%-6% improvement over stateof-the-art constraint-driven learning.

  • Generalized Expectation criteria for lightly supervised learning
    2011
    Co-Authors: Andrew Mccallum, Gregory Druck
    Abstract:

    Machine learning has facilitated many recent advances in natural language processing and information extraction. Unfortunately, most machine learning methods rely on costly labeled data, which impedes their application to new problems. Even in the absence of labeled data we often have a wealth of prior knowledge about these problems. For example, we may know which labels particular words are likely to indicate for a sequence labeling task, or we may have linguistic knowledge suggesting probable dependencies for syntactic analysis. This thesis focuses on incorporating such prior knowledge into learning, with the goal of reducing annotation effort for information extraction and natural language processing tasks. We advocate constraints on Expectations as a flexible and interpretable language for encoding prior knowledge. We focus on the development of Generalized Expectation (GE), a method for learning with Expectation constraints and unlabeled data. We explore the various flexibilities afforded by GE criteria, derive efficient algorithms for GE training, and relate GE to other methods for incorporating prior knowledge into learning. We then use GE to develop lightly supervised approaches to text classification, dependency parsing, sequence labeling, and entity resolution that yield accurate models for these tasks with minimal human effort. We also consider the incorporation of GE into interactive training systems that actively solicit prior knowledge from the user and assist the user in evaluating and analyzing model predictions.

  • Generalized Expectation Criteria for Semi-Supervised Learning with Weakly Labeled Data
    Journal of Machine Learning Research, 2010
    Co-Authors: Gideon S. Mann, Andrew Mccallum
    Abstract:

    In this paper, we present an overview of Generalized Expectation criteria (GE), a simple, robust, scalable method for semi-supervised training using weakly-labeled data. GE fits model parameters by favoring models that match certain Expectation constraints, such as marginal label distributions, on the unlabeled data. This paper shows how to apply Generalized Expectation criteria to two classes of parametric models: maximum entropy models and conditional random fields. Experimental results demonstrate accuracy improvements over supervised training and a number of other state-of-the-art semi-supervised learning methods for these models.

  • Generalized Expectation criteria for bootstrapping extractors using record text alignment
    Empirical Methods in Natural Language Processing, 2009
    Co-Authors: Kedar Bellare, Andrew Mccallum
    Abstract:

    Traditionally, machine learning approaches for information extraction require human annotated data that can be costly and time-consuming to produce. However, in many cases, there already exists a database (DB) with schema related to the desired output, and records related to the expected input text. We present a conditional random field (CRF) that aligns tokens of a given DB record and its realization in text. The CRF model is trained using only the available DB and unlabeled text with Generalized Expectation criteria. An annotation of the text induced from inferred alignments is used to train an information extractor. We evaluate our method on a citation extraction task in which alignments between DBLP database records and citation texts are used to train an extractor. Experimental results demonstrate an error reduction of 35% over a previous state-of-the-art method that uses heuristic alignments.

  • semi supervised learning of dependency parsers using Generalized Expectation criteria
    International Joint Conference on Natural Language Processing, 2009
    Co-Authors: Gregory Druck, Gideon S. Mann, Andrew Mccallum
    Abstract:

    In this paper, we propose a novel method for semi-supervised learning of non-projective log-linear dependency parsers using directly expressed linguistic prior knowledge (e.g. a noun's parent is often a verb). Model parameters are estimated using a Generalized Expectation (GE) objective function that penalizes the mismatch between model predictions and linguistic Expectation constraints. In a comparison with two prominent "unsupervised" learning methods that require indirect biasing toward the correct syntactic structure, we show that GE can attain better accuracy with as few as 20 intuitive constraints. We also present positive experimental results on longer sentences in multiple languages.

Gideon S. Mann - One of the best experts on this subject based on the ideXlab platform.

  • Generalized Expectation Criteria for Semi-Supervised Learning with Weakly Labeled Data
    Journal of Machine Learning Research, 2010
    Co-Authors: Gideon S. Mann, Andrew Mccallum
    Abstract:

    In this paper, we present an overview of Generalized Expectation criteria (GE), a simple, robust, scalable method for semi-supervised training using weakly-labeled data. GE fits model parameters by favoring models that match certain Expectation constraints, such as marginal label distributions, on the unlabeled data. This paper shows how to apply Generalized Expectation criteria to two classes of parametric models: maximum entropy models and conditional random fields. Experimental results demonstrate accuracy improvements over supervised training and a number of other state-of-the-art semi-supervised learning methods for these models.

  • semi supervised learning of dependency parsers using Generalized Expectation criteria
    International Joint Conference on Natural Language Processing, 2009
    Co-Authors: Gregory Druck, Gideon S. Mann, Andrew Mccallum
    Abstract:

    In this paper, we propose a novel method for semi-supervised learning of non-projective log-linear dependency parsers using directly expressed linguistic prior knowledge (e.g. a noun's parent is often a verb). Model parameters are estimated using a Generalized Expectation (GE) objective function that penalizes the mismatch between model predictions and linguistic Expectation constraints. In a comparison with two prominent "unsupervised" learning methods that require indirect biasing toward the correct syntactic structure, we show that GE can attain better accuracy with as few as 20 intuitive constraints. We also present positive experimental results on longer sentences in multiple languages.

  • ACL/IJCNLP - Semi-supervised Learning of Dependency Parsers using Generalized Expectation Criteria
    Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of, 2009
    Co-Authors: Gregory Druck, Gideon S. Mann, Andrew Mccallum
    Abstract:

    In this paper, we propose a novel method for semi-supervised learning of non-projective log-linear dependency parsers using directly expressed linguistic prior knowledge (e.g. a noun's parent is often a verb). Model parameters are estimated using a Generalized Expectation (GE) objective function that penalizes the mismatch between model predictions and linguistic Expectation constraints. In a comparison with two prominent "unsupervised" learning methods that require indirect biasing toward the correct syntactic structure, we show that GE can attain better accuracy with as few as 20 intuitive constraints. We also present positive experimental results on longer sentences in multiple languages.

  • learning from labeled features using Generalized Expectation criteria
    International ACM SIGIR Conference on Research and Development in Information Retrieval, 2008
    Co-Authors: Gregory Druck, Gideon S. Mann, Andrew Mccallum
    Abstract:

    It is difficult to apply machine learning to new domains because often we lack labeled problem instances. In this paper, we provide a solution to this problem that leverages domain knowledge in the form of affinities between input features and classes. For example, in a baseball vs. hockey text classification problem, even without any labeled data, we know that the presence of the word puck is a strong indicator of hockey. We refer to this type of domain knowledge as a labeled feature. In this paper, we propose a method for training discriminative probabilistic models with labeled features and unlabeled instances. Unlike previous approaches that use labeled features to create labeled pseudo-instances, we use labeled features directly to constrain the model's predictions on unlabeled instances. We express these soft constraints using Generalized Expectation (GE) criteria --- terms in a parameter estimation objective function that express preferences on values of a model Expectation. In this paper we train multinomial logistic regression models using GE criteria, but the method we develop is applicable to other discriminative probabilistic models. The complete objective function also includes a Gaussian prior on parameters, which encourages generalization by spreading parameter weight to unlabeled features. Experimental results on text classification data sets show that this method outperforms heuristic approaches to training classifiers with labeled features. Experiments with human annotators show that it is more beneficial to spend limited annotation time labeling features rather than labeling instances. For example, after only one minute of labeling features, we can achieve 80% accuracy on the ibm vs. mac text classification problem using GE-FL, whereas ten minutes labeling documents results in an accuracy of only 77%

  • Generalized Expectation criteria for semi supervised learning of conditional random fields
    Meeting of the Association for Computational Linguistics, 2008
    Co-Authors: Gideon S. Mann, Andrew Mccallum
    Abstract:

    This paper presents a semi-supervised training method for linear-chain conditional random fields that makes use of labeled features rather than labeled instances. This is accomplished by using Generalized Expectation criteria to express a preference for parameter settings in which the model’s distribution on unlabeled data matches a target distribution. We induce target conditional probability distributions of labels given features from both annotated feature occurrences in context and adhoc feature majority label assignment. The use of Generalized Expectation criteria allows for a dramatic reduction in annotation time by shifting from traditional instance-labeling to feature-labeling, and the methods presented outperform traditional CRF training and other semi-supervised methods when limited human effort is available.

Gregory Druck - One of the best experts on this subject based on the ideXlab platform.

  • Alternating Projections for Learning with Expectation Constraints
    arXiv: Learning, 2012
    Co-Authors: Kedar Bellare, Gregory Druck, Andrew Mccallum
    Abstract:

    We present an objective function for learning with unlabeled data that utilizes auxiliary Expectation constraints. We optimize this objective function using a procedure that alternates between information and moment projections. Our method provides an alternate interpretation of the posterior regularization framework (Graca et al., 2008), maintains uncertainty during optimization unlike constraint-driven learning (Chang et al., 2007), and is more efficient than Generalized Expectation criteria (Mann & McCallum, 2008). Applications of this framework include minimally supervised learning, semisupervised learning, and learning with constraints that are more expressive than the underlying model. In experiments, we demonstrate comparable accuracy to Generalized Expectation criteria for minimally supervised learning, and use expressive structural constraints to guide semi-supervised learning, providing a 3%-6% improvement over stateof-the-art constraint-driven learning.

  • Generalized Expectation criteria for lightly supervised learning
    2011
    Co-Authors: Andrew Mccallum, Gregory Druck
    Abstract:

    Machine learning has facilitated many recent advances in natural language processing and information extraction. Unfortunately, most machine learning methods rely on costly labeled data, which impedes their application to new problems. Even in the absence of labeled data we often have a wealth of prior knowledge about these problems. For example, we may know which labels particular words are likely to indicate for a sequence labeling task, or we may have linguistic knowledge suggesting probable dependencies for syntactic analysis. This thesis focuses on incorporating such prior knowledge into learning, with the goal of reducing annotation effort for information extraction and natural language processing tasks. We advocate constraints on Expectations as a flexible and interpretable language for encoding prior knowledge. We focus on the development of Generalized Expectation (GE), a method for learning with Expectation constraints and unlabeled data. We explore the various flexibilities afforded by GE criteria, derive efficient algorithms for GE training, and relate GE to other methods for incorporating prior knowledge into learning. We then use GE to develop lightly supervised approaches to text classification, dependency parsing, sequence labeling, and entity resolution that yield accurate models for these tasks with minimal human effort. We also consider the incorporation of GE into interactive training systems that actively solicit prior knowledge from the user and assist the user in evaluating and analyzing model predictions.

  • semi supervised learning of dependency parsers using Generalized Expectation criteria
    International Joint Conference on Natural Language Processing, 2009
    Co-Authors: Gregory Druck, Gideon S. Mann, Andrew Mccallum
    Abstract:

    In this paper, we propose a novel method for semi-supervised learning of non-projective log-linear dependency parsers using directly expressed linguistic prior knowledge (e.g. a noun's parent is often a verb). Model parameters are estimated using a Generalized Expectation (GE) objective function that penalizes the mismatch between model predictions and linguistic Expectation constraints. In a comparison with two prominent "unsupervised" learning methods that require indirect biasing toward the correct syntactic structure, we show that GE can attain better accuracy with as few as 20 intuitive constraints. We also present positive experimental results on longer sentences in multiple languages.

  • ACL/IJCNLP - Semi-supervised Learning of Dependency Parsers using Generalized Expectation Criteria
    Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of, 2009
    Co-Authors: Gregory Druck, Gideon S. Mann, Andrew Mccallum
    Abstract:

    In this paper, we propose a novel method for semi-supervised learning of non-projective log-linear dependency parsers using directly expressed linguistic prior knowledge (e.g. a noun's parent is often a verb). Model parameters are estimated using a Generalized Expectation (GE) objective function that penalizes the mismatch between model predictions and linguistic Expectation constraints. In a comparison with two prominent "unsupervised" learning methods that require indirect biasing toward the correct syntactic structure, we show that GE can attain better accuracy with as few as 20 intuitive constraints. We also present positive experimental results on longer sentences in multiple languages.

  • UAI - Alternating projections for learning with Expectation constraints
    2009
    Co-Authors: Kedar Bellare, Gregory Druck, Andrew Mccallum
    Abstract:

    We present an objective function for learning with unlabeled data that utilizes auxiliary Expectation constraints. We optimize this objective function using a procedure that alternates between information and moment projections. Our method provides an alternate interpretation of the posterior regularization framework (Graca et al., 2008), maintains uncertainty during optimization unlike constraint-driven learning (Chang et al., 2007), and is more efficient than Generalized Expectation criteria (Mann & McCallum, 2008). Applications of this framework include minimally supervised learning, semi-supervised learning, and learning with constraints that are more expressive than the underlying model. In experiments, we demonstrate comparable accuracy to Generalized Expectation criteria for minimally supervised learning, and use expressive structural constraints to guide semi-supervised learning, providing a 3%-6% improvement over state-of-the-art constraint-driven learning.

Xiaolong Wang - One of the best experts on this subject based on the ideXlab platform.

  • Biomedical named entity recognition using Generalized Expectation criteria
    International Journal of Machine Learning and Cybernetics, 2011
    Co-Authors: Yan Wu, Xiaolong Wang, Xuan Wang
    Abstract:

    It is difficult to apply machine learning to a domain which is short of labeled training data, such as biomedical named entity recognition (NER) which remains a challenging task because of its extraordinary complex nomenclature. In this paper, we proposed a semi-supervised method which can train condition random field (CRF) models using Generalized Expectation (GE) criteria to solve biomedical named entity recognition problem. In the proposed method, instead of “instance” labeling, the “feature” labeling is applied to get the training data which can save lots of labeling time. Latent Dirichlet Allocation (LDA) model was involved to choose the features for labeling. Experiment results show that the proposed method can dramatically improve the performance of biomedical NER through incorporating unlabeled data by feature labeling.

  • semi supervised biomedical relation classification using Generalized Expectation criteria
    International Conference on Machine Learning and Cybernetics, 2011
    Co-Authors: Chengjie Sun, Lin Yao, Lei Lin, Xuejun Sha, Xiaolong Wang
    Abstract:

    Semi-supervised learning is necessary for the extensive application of machine learning in practice. It can use unlabeled data to improve the performance of existing supervised machine learning method. In this work, we addressed the biomedical relation classification problem by utilizing a semi-supervised method which can train Maximum Entropy models according to Generalized Expectation criteria. In the proposed method, instead of "instance labeling" used in previous works, the "feature labeling" was applied to get the training data which can save lots of labeling time. A topic model was involved to choose the features for labeling. Experiment results show that the proposed method can dramatically improve the performance of biomedical relation classification through incorporating unlabeled data by feature labeling.

  • ICMLC - Semi-supervised biomedical relation classification using Generalized Expectation criteria
    2011 International Conference on Machine Learning and Cybernetics, 2011
    Co-Authors: Chengjie Sun, Lin Yao, Lei Lin, Xuejun Sha, Xiaolong Wang
    Abstract:

    Semi-supervised learning is necessary for the extensive application of machine learning in practice. It can use unlabeled data to improve the performance of existing supervised machine learning method. In this work, we addressed the biomedical relation classification problem by utilizing a semi-supervised method which can train Maximum Entropy models according to Generalized Expectation criteria. In the proposed method, instead of "instance labeling" used in previous works, the "feature labeling" was applied to get the training data which can save lots of labeling time. A topic model was involved to choose the features for labeling. Experiment results show that the proposed method can dramatically improve the performance of biomedical relation classification through incorporating unlabeled data by feature labeling.

Jinfeng Yang - One of the best experts on this subject based on the ideXlab platform.

  • horror text recognition based on Generalized Expectation criteria
    International Conference on Intelligent Science and Big Data Engineering, 2013
    Co-Authors: Guoqi Liu, Jinfeng Yang
    Abstract:

    Along with the growth of Internet, Web is glutted with more and more illegal and harmful information, such as pornography, violence, horror information. For a long time, researchers pay little attention to horror information relative to pornography. Horror information as well as pornography harms youngsters’ health seriously. In order to recognize horror information, in this paper, we propose a horror text recognition algorithm based on two classifiers, in which one is text title classifier, and the other is text content classifier based on Generalized Expectation (GE) criteria. A Generalized Expectation criterion is a term in a parameter estimation objective function that assigns scores which can express preferences to values of a model Expectation. In this paper, this parameter estimation objective function is used for measuring the correlation between features and sentiment labels.

  • IScIDE - Horror Text Recognition Based on Generalized Expectation Criteria
    Lecture Notes in Computer Science, 2013
    Co-Authors: Guoqi Liu, Jinfeng Yang
    Abstract:

    Along with the growth of Internet, Web is glutted with more and more illegal and harmful information, such as pornography, violence, horror information. For a long time, researchers pay little attention to horror information relative to pornography. Horror information as well as pornography harms youngsters’ health seriously. In order to recognize horror information, in this paper, we propose a horror text recognition algorithm based on two classifiers, in which one is text title classifier, and the other is text content classifier based on Generalized Expectation (GE) criteria. A Generalized Expectation criterion is a term in a parameter estimation objective function that assigns scores which can express preferences to values of a model Expectation. In this paper, this parameter estimation objective function is used for measuring the correlation between features and sentiment labels.