The Experts below are selected from a list of 11814 Experts worldwide ranked by ideXlab platform
Raymond J Mooney - One of the best experts on this subject based on the ideXlab platform.
-
multi prototype Vector Space Models of word meaning
North American Chapter of the Association for Computational Linguistics, 2010Co-Authors: Joseph Reisinger, Raymond J MooneyAbstract:Current Vector-Space Models of lexical semantics create a single "prototype" Vector to represent the meaning of a word. However, due to lexical ambiguity, encoding word meaning with a single Vector is problematic. This paper presents a method that uses clustering to produce multiple "sense-specific" Vectors for each word. This approach provides a context-dependent Vector representation of word meaning that naturally accommodates homonymy and polysemy. Experimental comparisons to human judgements of semantic similarity for both isolated words as well as words in sentential contexts demonstrate the superiority of this approach over both prototype and exemplar based Vector-Space Models.
-
HLT-NAACL - Multi-Prototype Vector-Space Models of Word Meaning
2010Co-Authors: Joseph Reisinger, Raymond J MooneyAbstract:Current Vector-Space Models of lexical semantics create a single "prototype" Vector to represent the meaning of a word. However, due to lexical ambiguity, encoding word meaning with a single Vector is problematic. This paper presents a method that uses clustering to produce multiple "sense-specific" Vectors for each word. This approach provides a context-dependent Vector representation of word meaning that naturally accommodates homonymy and polysemy. Experimental comparisons to human judgements of semantic similarity for both isolated words as well as words in sentential contexts demonstrate the superiority of this approach over both prototype and exemplar based Vector-Space Models.
Tom M. Mitchell - One of the best experts on this subject based on the ideXlab platform.
-
a compositional and interpretable semantic Space
North American Chapter of the Association for Computational Linguistics, 2015Co-Authors: Alona Fyshe, Leila Wehbe, Partha Pratim Talukdar, Brian Murphy, Tom M. MitchellAbstract:Vector Space Models (VSMs) of Semantics are useful tools for exploring the semantics of single words, and the composition of words to make phrasal meaning. While many methods can estimate the meaning (i.e. Vector) of a phrase, few do so in an interpretable way. We introduce a new method (CNNSE) that allows word and phrase Vectors to adapt to the notion of composition. Our method learns a VSM that is both tailored to support a chosen semantic composition operation, and whose resulting features have an intuitive interpretation. Interpretability allows for the exploration of phrasal semantics, which we leverage to analyze performance on a behavioral task.
-
documents and dependencies an exploration of Vector Space Models for semantic composition
Conference on Computational Natural Language Learning, 2013Co-Authors: Alona Fyshe, Partha Pratim Talukdar, Brian Murphy, Tom M. MitchellAbstract:In most previous research on distributional semantics, Vector Space Models (VSMs) of words are built either from topical information (e.g., documents in which a word is present), or from syntactic/semantic types of words (e.g., dependency parse links of a word in sentences), but not both. In this paper, we explore the utility of combining these two representations to build VSM for the task of semantic composition of adjective-noun phrases. Through extensive experiments on benchmark datasets, we find that even though a type-based VSM is effective for semantic composition, it is often outperformed by a VSM built using a combination of topic- and type-based statistics. We also introduce a new evaluation task wherein we predict the composed Vector representation of a phrase from the brain activity of a human subject reading that phrase. We exploit a large syntactically parsed corpus of 16 billion tokens to build our VSMs, with Vectors for both phrases and words, and make them publicly available.
-
Vector Space semantic parsing a framework for compositional Vector Space Models
Meeting of the Association for Computational Linguistics, 2013Co-Authors: Jayant Krishnamurthy, Tom M. MitchellAbstract:We present Vector Space semantic parsing (VSSP), a framework for learning compositional Models of Vector Space semantics. Our framework uses Combinatory Categorial Grammar (CCG) to define a correspondence between syntactic categories and semantic representations, which are Vectors and functions on Vectors. The complete correspondence is a direct consequence of minimal assumptions about the semantic representations of basic syntactic categories (e.g., nouns are Vectors), and CCG’s tight coupling of syntax and semantics. Furthermore, this correspondence permits nonuniform semantic representations and more expressive composition operations than previous work. VSSP builds a CCG semantic parser respecting this correspondence; this semantic parser parses text into lambda calculus formulas that evaluate to Vector Space representations. In these formulas, the meanings of words are represented by parameters that can be trained in a task-specific fashion. We present experiments using noun-verbnoun and adverb-adjective-noun phrases which demonstrate that VSSP can learn composition operations that RNN (Socher et al., 2011) and MV-RNN (Socher et al., 2012) cannot.
Phil Blunsom - One of the best experts on this subject based on the ideXlab platform.
-
new directions in Vector Space Models of meaning
Meeting of the Association for Computational Linguistics, 2014Co-Authors: Edward Grefenstette, Karl Moritz Hermann, Georgiana Dinu, Phil BlunsomAbstract:Symbolic approaches have dominated NLP as a means to model syntactic and semantic aspects of natural language. While powerful inferential tools exist for such Models, they suffer from an inability to capture correlation between words and to provide a continuous model for word, phrase, and document similarity. Distributed representations are one mechanism to overcome these constraints. This tutorial will supply NLP researchers with the mathematical and conceptual background to make use of Vector-based Models of meaning in their own research. We will begin by motivating the need for a transition from symbolic representations to distributed ones. We will briefly cover how collocational (distributional) Vectors can be used and manipulated to model word meaning. We will discuss the progress from distributional to distributed representations, and how neural networks allow us to learn word Vectors and condition them on metadata such as parallel texts, topic labels, or sentiment labels. Finally, we will present various forms of semantic Vector composition, and discuss their relative strengths and weaknesses, and their application to problems such as language modelling, paraphrasing, machine translation and document classification. This tutorial aims to bring researchers up to speed with recent developments in this fastmoving field. It aims to strike a balance between providing a general introduction to Vectorbased Models of meaning, an analysis of diverging strands of research in the field, and also being a hands-on tutorial to equip NLP researchers with the necessary tools and background knowledge to start working on such Models. Attendees should be comfortable with basic probability, linear algebra, and continuous mathematics. No substantial knowledge of machine learning is required.
-
the role of syntax in Vector Space Models of compositional semantics
Meeting of the Association for Computational Linguistics, 2013Co-Authors: Karl Moritz Hermann, Phil BlunsomAbstract:Modelling the compositional process by which the meaning of an utterance arises from the meaning of its parts is a fundamental task of Natural Language Processing. In this paper we draw upon recent advances in the learning of Vector Space representations of sentential semantics and the transparent interface between syntax and semantics provided by Combinatory Categorial Grammar to introduce Combinatory Categorial Autoencoders. This model leverages the CCG combinatory operators to guide a non-linear transformation of meaning within a sentence. We use this model to learn high dimensional embeddings for sentences and evaluate them in a range of tasks, demonstrating that the incorporation of syntax allows a concise model to learn representations that are both effective and general.
Joseph Reisinger - One of the best experts on this subject based on the ideXlab platform.
-
multi prototype Vector Space Models of word meaning
North American Chapter of the Association for Computational Linguistics, 2010Co-Authors: Joseph Reisinger, Raymond J MooneyAbstract:Current Vector-Space Models of lexical semantics create a single "prototype" Vector to represent the meaning of a word. However, due to lexical ambiguity, encoding word meaning with a single Vector is problematic. This paper presents a method that uses clustering to produce multiple "sense-specific" Vectors for each word. This approach provides a context-dependent Vector representation of word meaning that naturally accommodates homonymy and polysemy. Experimental comparisons to human judgements of semantic similarity for both isolated words as well as words in sentential contexts demonstrate the superiority of this approach over both prototype and exemplar based Vector-Space Models.
-
HLT-NAACL - Multi-Prototype Vector-Space Models of Word Meaning
2010Co-Authors: Joseph Reisinger, Raymond J MooneyAbstract:Current Vector-Space Models of lexical semantics create a single "prototype" Vector to represent the meaning of a word. However, due to lexical ambiguity, encoding word meaning with a single Vector is problematic. This paper presents a method that uses clustering to produce multiple "sense-specific" Vectors for each word. This approach provides a context-dependent Vector representation of word meaning that naturally accommodates homonymy and polysemy. Experimental comparisons to human judgements of semantic similarity for both isolated words as well as words in sentential contexts demonstrate the superiority of this approach over both prototype and exemplar based Vector-Space Models.
Sebastiano Panichella - One of the best experts on this subject based on the ideXlab platform.
-
Using IR methods for labeling source code artifacts: Is it worthwhile?
2012 20th IEEE International Conference on Program Comprehension (ICPC), 2012Co-Authors: Andrea De Lucia, Massimiliano Di Penta, Rocco Oliveto, Annibale Panichella, Sebastiano PanichellaAbstract:Information Retrieval (IR) techniques have been used for various software engineering tasks, including the labeling of software artifacts by extracting “keywords” from them. Such techniques include Vector Space Models, Latent Semantic Indexing, Latent Dirichlet Allocation, as well as customized heuristics extracting words from specific source code elements. This paper investigates how source code artifact labeling performed by IR techniques would overlap (and differ) from labeling performed by humans. This has been done by asking a group of subjects to label 20 classes from two Java software systems, JHotDraw and eXVantage. Results indicate that, in most cases, automatic labeling would be more similar to human-based labeling if using simpler techniques - e.g., using words from class and method names - that better reflect how humans behave. Instead, clustering-based approaches (LSI and LDA) are much more worthwhile to be used on source code artifacts having a high verbosity, as well as for artifacts requiring more effort to be manually labeled.
-
ICPC - Using IR methods for labeling source code artifacts: Is it worthwhile?
2012 20th IEEE International Conference on Program Comprehension (ICPC), 2012Co-Authors: Andrea De Lucia, Massimiliano Di Penta, Rocco Oliveto, Annibale Panichella, Sebastiano PanichellaAbstract:Information Retrieval (IR) techniques have been used for various software engineering tasks, including the labeling of software artifacts by extracting “keywords” from them. Such techniques include Vector Space Models, Latent Semantic Indexing, Latent Dirichlet Allocation, as well as customized heuristics extracting words from specific source code elements. This paper investigates how source code artifact labeling performed by IR techniques would overlap (and differ) from labeling performed by humans. This has been done by asking a group of subjects to label 20 classes from two Java software systems, JHotDraw and eXVantage. Results indicate that, in most cases, automatic labeling would be more similar to human-based labeling if using simpler techniques — e.g., using words from class and method names — that better reflect how humans behave. Instead, clustering-based approaches (LSI and LDA) are much more worthwhile to be used on source code artifacts having a high verbosity, as well as for artifacts requiring more effort to be manually labeled.