The Experts below are selected from a list of 264 Experts worldwide ranked by ideXlab platform
Afra Alishahi - One of the best experts on this subject based on the ideXlab platform.
-
CoNLL - Online Entropy-Based Model of Lexical Category Acquisition
2010Co-Authors: Grzegorz Chrupala, Afra AlishahiAbstract:Children learn a robust representation of Lexical categories at a young age. We propose an incremental model of this process which efficiently groups words into Lexical categories based on their local context using an information-theoretic criterion. We train our model on a corpus of child-directed speech from CHILDES and show that the model learns a fine-grained set of intuitive word categories. Furthermore, we propose a novel evaluation approach by comparing the efficiency of our induced categories against other Category sets (including traditional part of speech tags) in a variety of language tasks. We show the categories induced by our model typically outperform the other Category sets.
-
online entropy based model of Lexical Category acquisition
Conference on Computational Natural Language Learning, 2010Co-Authors: Grzegorz Chrupala, Afra AlishahiAbstract:Children learn a robust representation of Lexical categories at a young age. We propose an incremental model of this process which efficiently groups words into Lexical categories based on their local context using an information-theoretic criterion. We train our model on a corpus of child-directed speech from CHILDES and show that the model learns a fine-grained set of intuitive word categories. Furthermore, we propose a novel evaluation approach by comparing the efficiency of our induced categories against other Category sets (including traditional part of speech tags) in a variety of language tasks. We show the categories induced by our model typically outperform the other Category sets.
-
Lexical Category acquisition as an incremental process
2009Co-Authors: Afra Alishahi, Grzegorz ChrupalaAbstract:Psycholinguistic studies suggest that early on children acquire robust knowledge of the abstract Lexical categories such as nouns, verbs and determiners (e.g., Gelman & Taylor, 1984; Kemp et al., 2005). Children’s grouping of words into categories might be based on various cues, including the phonological and morphological properties of a word, the distributional information about its surrounding context, and its semantic features. Among these, the distributional properties of the local context of a word have been shown to be a reliable cue for the formation of the Lexical categories (Redington et al., 1998; Mintz, 2003). Several computational models have used distributional information for categorizing words (e.g. Brown et al., 1992; Schutze, 1993; Redington et al., 1998; Clark, 2000; Mintz, 2002). The majority of these models use iterative, unsupervised methods that partition the vocabulary into a set of optimum clusters (e.g., Brown et al., 1992; Clark, 2000). The generated clusters are intuitive, and can be used in different tasks such as word prediction and parsing. Moreover, these models confirm the learnability of abstract word categories, and hint at distributional cues as a useful source of information for this purpose. The process of learning word categories by children is necessarily incremental. Human language acquisition is bounded by memory and processing limitations, and it is implausible that humans process large volumes of text at once and induce an optimum set of categories. Efficient online computational models must be developed to investigate whether the distributional information is equally powerful in an online process of word categorization. There have only been a few previous attempts at applying an incremental method to Category acquisition. The model of Cartwright & Brent (1997) uses an algorithm which incrementally merges word clusters so that a Minimum Description Length criterion for a template grammar is optimized. The model treats whole sentences as contextual units, which sacrifices a degree of incrementality, as well as making it less robust to noise in the input. The model proposed by Parisien et al. (2008) uses a Bayesian clustering algorithm that can cope with ambiguity, and shows the developmental trends observed in children (e.g. the order of acquisition of different categories). However, their fully Bayesian implementation is computationally expensive. Moreover, when measuring the similarity between two contexts, the model is sensitive to mismatches between any pair of context features, which results in the creation of sparse clusters. To overcome the problem, they introduce a bootstrapping mechanism which improves the performance, but adds substantially to the computational load. We propose an efficient incremental model for clustering words into categories based on their local context. Each word of a sentence is processed and categorized individually based on the similarity of its content (the word itself) and its context (the surrounding words) to the existing clusters. We test our model on a corpus of child-directed speech from CHILDES (MacWhinney, 2000). Over time, the model learns a finegrained set of word categories that are intuitive and can be used in a variety of tasks. We evaluate our model on a word prediction task, where a missing word is guessed based on its context. We also use our model to infer the semantic properties of a novel word based on the context it appears in. In both tasks, we show that our induced categories outperform the part of speech tags used for annotating the corpus.
Grzegorz Chrupala - One of the best experts on this subject based on the ideXlab platform.
-
CoNLL - Online Entropy-Based Model of Lexical Category Acquisition
2010Co-Authors: Grzegorz Chrupala, Afra AlishahiAbstract:Children learn a robust representation of Lexical categories at a young age. We propose an incremental model of this process which efficiently groups words into Lexical categories based on their local context using an information-theoretic criterion. We train our model on a corpus of child-directed speech from CHILDES and show that the model learns a fine-grained set of intuitive word categories. Furthermore, we propose a novel evaluation approach by comparing the efficiency of our induced categories against other Category sets (including traditional part of speech tags) in a variety of language tasks. We show the categories induced by our model typically outperform the other Category sets.
-
online entropy based model of Lexical Category acquisition
Conference on Computational Natural Language Learning, 2010Co-Authors: Grzegorz Chrupala, Afra AlishahiAbstract:Children learn a robust representation of Lexical categories at a young age. We propose an incremental model of this process which efficiently groups words into Lexical categories based on their local context using an information-theoretic criterion. We train our model on a corpus of child-directed speech from CHILDES and show that the model learns a fine-grained set of intuitive word categories. Furthermore, we propose a novel evaluation approach by comparing the efficiency of our induced categories against other Category sets (including traditional part of speech tags) in a variety of language tasks. We show the categories induced by our model typically outperform the other Category sets.
-
Lexical Category acquisition as an incremental process
2009Co-Authors: Afra Alishahi, Grzegorz ChrupalaAbstract:Psycholinguistic studies suggest that early on children acquire robust knowledge of the abstract Lexical categories such as nouns, verbs and determiners (e.g., Gelman & Taylor, 1984; Kemp et al., 2005). Children’s grouping of words into categories might be based on various cues, including the phonological and morphological properties of a word, the distributional information about its surrounding context, and its semantic features. Among these, the distributional properties of the local context of a word have been shown to be a reliable cue for the formation of the Lexical categories (Redington et al., 1998; Mintz, 2003). Several computational models have used distributional information for categorizing words (e.g. Brown et al., 1992; Schutze, 1993; Redington et al., 1998; Clark, 2000; Mintz, 2002). The majority of these models use iterative, unsupervised methods that partition the vocabulary into a set of optimum clusters (e.g., Brown et al., 1992; Clark, 2000). The generated clusters are intuitive, and can be used in different tasks such as word prediction and parsing. Moreover, these models confirm the learnability of abstract word categories, and hint at distributional cues as a useful source of information for this purpose. The process of learning word categories by children is necessarily incremental. Human language acquisition is bounded by memory and processing limitations, and it is implausible that humans process large volumes of text at once and induce an optimum set of categories. Efficient online computational models must be developed to investigate whether the distributional information is equally powerful in an online process of word categorization. There have only been a few previous attempts at applying an incremental method to Category acquisition. The model of Cartwright & Brent (1997) uses an algorithm which incrementally merges word clusters so that a Minimum Description Length criterion for a template grammar is optimized. The model treats whole sentences as contextual units, which sacrifices a degree of incrementality, as well as making it less robust to noise in the input. The model proposed by Parisien et al. (2008) uses a Bayesian clustering algorithm that can cope with ambiguity, and shows the developmental trends observed in children (e.g. the order of acquisition of different categories). However, their fully Bayesian implementation is computationally expensive. Moreover, when measuring the similarity between two contexts, the model is sensitive to mismatches between any pair of context features, which results in the creation of sparse clusters. To overcome the problem, they introduce a bootstrapping mechanism which improves the performance, but adds substantially to the computational load. We propose an efficient incremental model for clustering words into categories based on their local context. Each word of a sentence is processed and categorized individually based on the similarity of its content (the word itself) and its context (the surrounding words) to the existing clusters. We test our model on a corpus of child-directed speech from CHILDES (MacWhinney, 2000). Over time, the model learns a finegrained set of word categories that are intuitive and can be used in a variety of tasks. We evaluate our model on a word prediction task, where a missing word is guessed based on its context. We also use our model to infer the semantic properties of a novel word based on the context it appears in. In both tasks, we show that our induced categories outperform the part of speech tags used for annotating the corpus.
Morten H. Christiansen - One of the best experts on this subject based on the ideXlab platform.
-
SPECIAL SECTION: COMPUTATIONAL PRINCIPLES OF LANGUAGE ACQUISITION The secret is in the sound: from unsegmented speech to Lexical categories
2020Co-Authors: Morten H. Christiansen, Luca Onnis, Stephen HockemaAbstract:When learning language, young children are faced with many seemingly formidable challenges, including discovering words embedded in a continuous stream of sounds and determining what role these words play in syntactic constructions. We suggest that knowledge of phoneme distributions may play a crucial part in helping children segment words and determine their Lexical Category, and we propose an integrated model of how children might go from unsegmented speech to Lexical categories. We corroborated this theoretical model using a two-stage computational analysis of a large corpus of English child-directed speech. First, we used transition probabilities between phonemes to find words in unsegmented speech. Second, we used distributional information about word edges ‐ the beginning and ending phonemes of words ‐ to predict whether the segmented words from the first stage were nouns, verbs, or something else. The results indicate that discovering Lexical units and their associated syntactic Category in child-directed speech is possible by attending to the statistics of single phoneme transitions and word-initial and final phonemes. Thus, we suggest that a core computational principle in language acquisition is that the same source of information is used to learn about different aspects of linguistic structure.
-
ICONIC VERSUS ARBITRARY MAPPINGS AND THE CULTURAL TRANSMISSION OF LANGUAGE
The Evolution of Language, 2020Co-Authors: Padraic Monaghan, Morten H. ChristiansenAbstract:Most theories of language evolution assume that the ability to use symbols was a crucial step towards modern language (for a review see, e.g., Christiansen & Kirby, 2003). Following de Saussure, symbol use is typically construed as the capacity for establishing arbitrary mappings from sounds or gestures to specific concepts and/or percepts for the purpose of communication. Although intuition suggest that iconic relationships between form and meaning should make the learning of such mappings easier (e.g., sound symbolism), recent simulations by Gasser (2004) have demonstrated that, for large vocabularies, the learning advantage is for arbitrary relationships. Because systematic iconic mappings between forms and meanings require strong constraints on the space of possible pairings (e.g., a particular onset phoneme is restricted to only co-occur with a particular facet of meaning) it is only possible to encode efficiently a relatively small number of words. In contrast, arbitrary mappings between form and meaning impose fewer constraints and therefore permit the learning of a large and extendable vocabulary, which is the hallmark of human language. However, the cost of arbitrariness is that generalities about the language structure, such as the Lexical Category of a word, are not readily learnable from the sounds of the language. Such systematicity has been seen as advantageous, perhaps even necessary, for learning categories (Braine, 1987). In this paper, we hypothesize that cultural transmission has shaped language so as to incorporate certain systematic properties of iconic mappings in order to facilitate the learning of Lexical categories. Importantly, the iconic mapping is not between form and meaning but between form and Lexical Category.
-
Using Phoneme Distributions to Discover Words and Lexical Categories in Unsegmented Speech
2017Co-Authors: Morten H. Christiansen, Stephen Hockema, Luca OnnisAbstract:Using Phoneme Distributions to Discover Words and Lexical Categories in Unsegmented Speech Morten H. Christiansen (mhc27@cornell.edu) Department of Psychology, Cornell University Ithaca, NY 14853 USA Stephen A. Hockema (shockema@indiana.edu) Department of Psychological and Brain Sciences, Indiana University Bloomington, IN 47405 USA Luca Onnis (lo35@cornell.edu) Department of Psychology, Cornell University Ithaca, NY 14853 USA Abstract how words are put together to form meaningful sentences. An initial step in this direction involves determining what syntactic roles individual words may play in sentences. Several types of information may be useful for the discovery of Lexical categories, such as nouns and verbs, including distributions of word co-occurrences (e.g., Redington, Chater & Finch, 1998), frequent word frames (e.g., I X it; Mintz, 2003), and phonological cues (Kelly, 1992; Monaghan, Chater & Christiansen, 2005). Indeed, merely paying attention to the first and last phoneme of a word has been shown to be useful for predicting Lexical categories across different language such as English, Dutch, French and Japanese (Onnis & Christiansen, 2005). During the first year of life, infants become perceptually attuned to the sound structure of their native language (see e.g., Jusczyk, 1997; Kuhl, 1999, for reviews). We suggest that this attunement to native phonology is crucial not only for word segmentation but also for the discovery of syntactic structure. Specifically, we hypothesize that phoneme distributions may be a highly useful source of information that a child is likely to utilize in both tasks. In this paper, we test this hypothesis by carrying out a two-step corpus analysis in which information about phoneme distribution is used first in Experiment 1 to segment words out of a large corpus of phonologically-transcribed child- directed speech and then in Experiment 2 to predict the Lexical Category of these words (noun, verb, or other). The results show that it is possible to get from unsegmented speech to Lexical categories with a reasonably high accuracy and completeness using only information about the distribution of phonemes in the input. When learning language young children are faced with many formidable challenges, including discovering words embedded in a continuous stream of sounds and determining what role these words play in syntactic constructions. We suggest that knowledge of phoneme distributions may play a crucial part in helping children segment words and determining their Lexical Category. We performed a two-step analysis of a large corpus of English child-directed speech. First, we used transition probabilities between phonemes to find words in unsegmented speech. Second, we used distributional information about word edges—the beginning and ending phonemes of words—to predict whether the segmented words were nouns, verbs, or something else. These results indicate that discovering Lexical units and their associated syntactic Category in child-directed speech is possible by attending to the statistics of single phoneme transitions and word-initial and final phonemes. Introduction One of the first tasks facing an infant embarking on language development is to discover where the words are in fluent speech. This is not a trivial problem because there are no acoustic equivalents in speech of the white spaces placed between words in written text. To find the words, infants appear to be utilizing several different cues, including Lexical stress (Curtin, Mintz & Christiansen, 2005), transitional probabilities between syllables (Saffran, Aslin & Newport, 1996), and phonotactic constraints on phoneme combinations in words (Jusczyk, Friederici & Svenkerud, 1993). Among these word segmentation cues, computational models and statistical analyses have indicated that, at least in English, phoneme distributions may be the single most useful source of information for the discovery of word boundaries (e.g., Brent & Cartwright, 1996; Hockema, 2006), especially when combined with information about Lexical stress patterns (Christiansen, Allen & Seidenberg, Discovering words is, however, only one of the first steps in language acquisition. The child also needs to discover Experiment 1: Discovering Words Infants are proficient statistical learners, sensitive to sequential sound probabilities in artificial (Saffran et al., 1996) and natural language (Jusczyk et al., 1993). Such statistical learning abilities would be most useful for word segmentation if natural speech was primarily made up of two types of sound sequences: ones that occur within words
-
Phonological typicality influences sentence processing in predictive contexts: reply to Staub, Grant, Clifton, and Rayner (2009).
Journal of Experimental Psychology: Learning Memory and Cognition, 2011Co-Authors: Thomas A. Farmer, Padraic Monaghan, Jennifer B. Misyak, Morten H. ChristiansenAbstract:In 2 separate self-paced reading experiments, Farmer, Christiansen, and Monaghan (2006) found that the degree to which a word's phonology is typical of other words in its Lexical Category influences online processing of nouns and verbs in predictive contexts. Staub, Grant, Clifton. and Rayner (2009) failed to find an effect of phonological typicality when they combined stimuli from the separate experiments into a single experiment. We replicated Staub et al.'s experiment and found that the combination of stimulus sets affects the predictiveness of the syntactic context; this reduces the phonological typicality effect as the experiment proceeds, although the phonological typicality effect was still evident early in the experiment. Although an ambiguous context may diminish sensitivity to the probabilistic relationship between the sound of a word and its Lexical Category. phonological typicality does influence online sentence processing during normal reading when the syntactic context is predictive of the Lexical Category of upcoming words.
-
The secret is in the sound: from unsegmented speech to Lexical categories.
Developmental Science, 2009Co-Authors: Morten H. Christiansen, Luca Onnis, Stephen HockemaAbstract:: When learning language, young children are faced with many seemingly formidable challenges, including discovering words embedded in a continuous stream of sounds and determining what role these words play in syntactic constructions. We suggest that knowledge of phoneme distributions may play a crucial part in helping children segment words and determine their Lexical Category, and we propose an integrated model of how children might go from unsegmented speech to Lexical categories. We corroborated this theoretical model using a two-stage computational analysis of a large corpus of English child-directed speech. First, we used transition probabilities between phonemes to find words in unsegmented speech. Second, we used distributional information about word edges--the beginning and ending phonemes of words--to predict whether the segmented words from the first stage were nouns, verbs, or something else. The results indicate that discovering Lexical units and their associated syntactic Category in child-directed speech is possible by attending to the statistics of single phoneme transitions and word-initial and final phonemes. Thus, we suggest that a core computational principle in language acquisition is that the same source of information is used to learn about different aspects of linguistic structure.
Gérard Bailly - One of the best experts on this subject based on the ideXlab platform.
-
Adaptive latency for part-of-speech tagging in incremental text-to-speech synthesis
Proceedings of the Annual Conference of the International Speech Communication Association INTERSPEECH, 2016Co-Authors: Maël Pouget, Olha Nahorna, Thomas Hueber, Gérard BaillyAbstract:Incremental text-to-speech systems aim at synthesizing a text ’on-the-fly’, while the user is typing a sentence. In this context, this article addresses the problem of the part-of-speech tagging (POS, i.e. Lexical Category) which is a critical step for accu- rate grapheme-to-phoneme conversion and prosody estimation. Here, the main challenge is to estimate the POS of a given word without knowing its ’right context’ (i.e. the following words which are not available yet). To address this issue, we pro- pose a method based on a set of decision trees estimating online whether a given POS tag is likely to be modified when more right-contextual information becomes available. In such a case, the synthesis is delayed until POS stability is guaranteed. This results in delivering the synthetic voice in word chunks of vari- able length. Objective evaluation on French shows that the pro- posed method is able to estimate POS tags with more than a 92% accuracy (compared to a non-incremental system) while minimizing the synthesis latency (between 1 and 4 words). Per- ceptual evaluation (ranking test) is then carried in the context of HMM-based speech synthesis. Experimental results show that the word grouping resulting from the proposed method is rated more acceptable than word-by-word incremental synthesis.
Xiaojie Wang - One of the best experts on this subject based on the ideXlab platform.
-
semi supervised incremental model for Lexical Category acquisition
2013Co-Authors: Bichuan Zhang, Xiaojie WangAbstract:We present a novel semi-supervised incremental approach for discovering word categories, sets of words sharing a significant aspect of distributional context. We utilize high frequency words as seed in order to capture semantic information in form of symmetric similarity of word pair, Lexical Category is then created based on a new clustering algorithm proposed recently: affinity propagation (AP). Furthermore, we assess the performance using a new measure we proposed that meets three criteria: informativeness, diversity and purity. The quantitative and qualitative evaluation show that this semi-supervised incremental approach is plausible for induction of Lexical categories from distributional data.
-
IC-NIDC - Lexical Category based computational model of syntax acquisition
2012 3rd IEEE International Conference on Network Infrastructure and Digital Content, 2012Co-Authors: Bichuan Zhang, Xiaojie WangAbstract:This paper presents a computational model of syntax acquisition based on Lexical Category from a corpus of child-directed utterances. In our proposed Lexical Category based Syntax Acquisition Model (LEXSAM), the implemented algorithm represents words that have an identical backbone and similar context as their associated Lexical Category, and extracts syntactic construction, determined by context-sensitive statistical inference. These Lexical Category representations approximate the semantic input available to the child, and the Lexical categories specify the meanings of clusters of words or syntactic derivations. When tested on utterances from the CHILDES corpus, our model outperforms the one without Lexical Category. The result shows that the children are unlikely to go through a pure syntax acquisition phase, but in a processing on which the Lexical semantic knowledge affects.
-
Graph-based model for Lexical Category acquisition
2012 IEEE Symposium on Robotics and Applications (ISRA), 2012Co-Authors: Bichuan Zhang, Xiaojie Wang, Guannan FangAbstract:We present a novel approach for discovering word categories, sets of words sharing a significant aspect of distributional context. We determine symmetric similarity of word pair, Lexical Category is then created based on graph-partitioning method. We train our model on a corpus of child-directed speech from CHILDES and show that the model successful learns word categories. Furthermore, a number of different measures have been proposed for evaluating computational models of Category acquisition. In this paper, we propose a new measure that meets three criteria: informativeness, diversity and purity.
-
Lexical Category based computational model of syntax acquisition
2012 3rd IEEE International Conference on Network Infrastructure and Digital Content, 2012Co-Authors: Bichuan Zhang, Xiaojie WangAbstract:This paper presents a computational model of syntax acquisition based on Lexical Category from a corpus of child-directed utterances. In our proposed Lexical Category based Syntax Acquisition Model (LEXSAM), the implemented algorithm represents words that have an identical backbone and similar context as their associated Lexical Category, and extracts syntactic construction, determined by context-sensitive statistical inference. These Lexical Category representations approximate the semantic input available to the child, and the Lexical categories specify the meanings of clusters of words or syntactic derivations. When tested on utterances from the CHILDES corpus, our model outperforms the one without Lexical Category. The result shows that the children are unlikely to go through a pure syntax acquisition phase, but in a processing on which the Lexical semantic knowledge affects.
-
CCIS - Dirichlet Process Mixture Models for Lexical Category acquisition
2011 IEEE International Conference on Cloud Computing and Intelligence Systems, 2011Co-Authors: Bichuan Zhang, Xiaojie Wang, Guannan FangAbstract:In this work, we apply Dirichlet Process Mixture Models (DPMMs) to a cognitive computational task in natural language processing (NLP): Lexical Category acquisition. The model takes a corpus of child-directed speech from CHILDES as input. We assess the performance using a new measure we proposed that meets three criteria: informativeness, diversity and purity. The quantitative and qualitative evaluation performed highlights the choice of the feature dimension and inherent parameters can influence the performance of DPMMs towards Lexical Category solutions.