The Experts below are selected from a list of 101244 Experts worldwide ranked by ideXlab platform
Thomas L Griffiths - One of the best experts on this subject based on the ideXlab platform.
-
Learning How to Generalize
Cognitive science, 2019Co-Authors: Joseph L Austerweil, Sophia Sanborn, Thomas L GriffithsAbstract:Generalization is a fundamental problem solved by every cognitive system in essentially every domain. Although it is known that how people generalize varies in complex ways depending on the context or domain, it is an open question how people learn the appropriate way to generalize for a new context. To understand this capability, we cast the problem of learning how to generalize as a problem of learning the appropriate Hypothesis Space for generalization. We propose a normative mathematical framework for learning how to generalize by learning inductive biases for which properties are relevant for generalization in a domain from the statistical structure of features and concepts observed in that domain. More formally, the framework predicts that an ideal learner should learn to generalize by either taking the weighted average of the results of generalizing according to each Hypothesis Space, with weights given by how well each Hypothesis Space fits the previously observed concepts, or by using the most likely Hypothesis Space. We compare the predictions of this framework to human generalization behavior with three experiments in one perceptual (rectangles) and two conceptual (animals and numbers) domains. Across all three studies we find support for the framework's predictions, including individual-level support for averaging in the third study.
-
constructing a Hypothesis Space from the web for large scale bayesian word learning
Cognitive Science, 2012Co-Authors: Joshua T Abbott, Joseph L Austerweil, Thomas L GriffithsAbstract:Constructing a Hypothesis Space from the Web for large-scale Bayesian word learning Joshua T. Abbott (joshua.abbott@berkeley.edu) Joseph L. Austerweil (joseph.austerweil@gmail.com) Thomas L. Griffiths (tom griffiths@berkeley.edu) Department of Psychology, University of California, Berkeley, CA 94720 USA Abstract Bayesian generalization model. In this paper, we use this approach to show how a Hypothesis Space and prior can be constructed automatically from a large online database, mak- ing it possible to apply the Bayesian generalization frame- work to a wide range of naturalistic stimuli. We focus on one specific generalization problem, word learning, where peo- ple learn new words from observing a few objects that can be labeled with that word. Given that the number of possible ex- tensions of a word is essentially infinite, learning the objects referred to by a word is a very difficult inductive problem (Quine, 1975). Xu and Tenenbaum (2007) showed how the Bayesian generalization framework could be used to explain how people learn new words. However, to construct the hy- pothesis Space of their Bayesian model, Xu and Tenenbaum (2007) elicited approximately 400 similarity judgments from their participants. Clearly this is not practical to extend into every domain where people learn words. Thus, word learn- ing is an appropriate setting for exploring novel methods of constructing Hypothesis Spaces and prior distributions. The Bayesian generalization framework has been successful in explaining how people generalize a property from a few observed stimuli to novel stimuli, across several different domains. To create a successful Bayesian generalization model, modelers typically specify a Hypothesis Space and prior probability distribution for each specific domain. How- ever, this raises two problems: the models do not scale beyond the (typically small-scale) domain that they were designed for, and the explanatory power of the models is reduced by their reliance on a hand-coded Hypothesis Space and prior. To solve these two problems, we propose a method for deriving Hypothesis Spaces and priors from large online databases. We evaluate our method by constructing a Hypothesis Space and prior for a Bayesian word learning model from WordNet, a large online database that encodes the semantic relationships between words as a network. After validating our approach by replicating a previous word learning study, we apply the same model to a new experiment featuring three additional taxonomic domains (clothing, containers, and seats). In both experiments, we found that the same automatically constructed Hypothesis Space explains the complex pattern of generalization behavior, producing accurate predictions across a total of six different domains. We propose a method for automatically constructing the Hypothesis Space and prior distribution of a Bayesian word learning model using freely available online resources. In particular, we use WordNet (Fellbaum, 2010; Miller, 1995) as an initial source for automatically creating the Hypothesis Space, and ImageNet (Deng et al., 2009) as a source of natu- ralistic images that can be used as stimuli to test the resulting model in behavioral experiments. WordNet is a popular lexi- cal database of English comprised of over 100,000 relational sets of synonyms. ImageNet is a large ontology of images conforming to the hierarchical structure of WordNet, with the aim of providing over 500 high-quality images per noun in WordNet. These resources allow us to construct Hypothesis Spaces and prior distributions for word learning without elic- iting a single judgment from participants and test the result- ing model on a much larger scale than was previously pos- sible. We demonstrate that the Bayesian model formulated from WordNet captures participant judgments in two behav- ioral experiments, addressing the practical and theoretical is- sues with Bayesian models discussed earlier. Keywords: generalization; concept learning; word learning; Bayesian modeling; online databases Introduction Many problems solved by the mind conform to the same ab- stract computational formulation: How should a property be generalized to novel stimuli from a set of stimuli observed to have the property? As there are many ways to extend the property that are consistent with some observed evidence, these are problems of induction, where the evidence con- strains, but does not determine, the solution to a problem. The Bayesian generalization framework (Shepard, 1987; Tenen- baum & Griffiths, 2001) has been remarkably successful at explaining human generalization behavior in a wide range of domains. However, its success is largely dependent on the choice of a Hypothesis Space and a prior probability distribu- tion on hypotheses, which are usually hand constructed by the researcher for each specific problem. This is unsatisfy- ing practically, because the models do not scale beyond the originally modeled problem, and theoretically, as it is unclear whether their success is due to the cleverness of the modeler and not because of a deep mathematical property of the com- putational problem that people solve. One possible solution is to use existing sources of infor- mation about the organization of a domain as the basis for specifying a Hypothesis Space and prior. This helps address both the practical and the theoretical concerns raised by the The plan of the rest of the paper is as follows. In the next sections we review the Bayesian generalization model and then examine how Xu and Tenenbaum (2007) constructed the Hypothesis Space for their Bayesian word learning model. We then show how to build a Hypothesis Space from WordNet that can be used to evaluate word learning models on a large scale. Afterwards, we present two experiments utilizing this hypoth-
-
CogSci - Constructing a Hypothesis Space from the Web for large-scale Bayesian word learning.
Cognitive Science, 2012Co-Authors: Joshua T Abbott, Joseph L Austerweil, Thomas L GriffithsAbstract:Constructing a Hypothesis Space from the Web for large-scale Bayesian word learning Joshua T. Abbott (joshua.abbott@berkeley.edu) Joseph L. Austerweil (joseph.austerweil@gmail.com) Thomas L. Griffiths (tom griffiths@berkeley.edu) Department of Psychology, University of California, Berkeley, CA 94720 USA Abstract Bayesian generalization model. In this paper, we use this approach to show how a Hypothesis Space and prior can be constructed automatically from a large online database, mak- ing it possible to apply the Bayesian generalization frame- work to a wide range of naturalistic stimuli. We focus on one specific generalization problem, word learning, where peo- ple learn new words from observing a few objects that can be labeled with that word. Given that the number of possible ex- tensions of a word is essentially infinite, learning the objects referred to by a word is a very difficult inductive problem (Quine, 1975). Xu and Tenenbaum (2007) showed how the Bayesian generalization framework could be used to explain how people learn new words. However, to construct the hy- pothesis Space of their Bayesian model, Xu and Tenenbaum (2007) elicited approximately 400 similarity judgments from their participants. Clearly this is not practical to extend into every domain where people learn words. Thus, word learn- ing is an appropriate setting for exploring novel methods of constructing Hypothesis Spaces and prior distributions. The Bayesian generalization framework has been successful in explaining how people generalize a property from a few observed stimuli to novel stimuli, across several different domains. To create a successful Bayesian generalization model, modelers typically specify a Hypothesis Space and prior probability distribution for each specific domain. How- ever, this raises two problems: the models do not scale beyond the (typically small-scale) domain that they were designed for, and the explanatory power of the models is reduced by their reliance on a hand-coded Hypothesis Space and prior. To solve these two problems, we propose a method for deriving Hypothesis Spaces and priors from large online databases. We evaluate our method by constructing a Hypothesis Space and prior for a Bayesian word learning model from WordNet, a large online database that encodes the semantic relationships between words as a network. After validating our approach by replicating a previous word learning study, we apply the same model to a new experiment featuring three additional taxonomic domains (clothing, containers, and seats). In both experiments, we found that the same automatically constructed Hypothesis Space explains the complex pattern of generalization behavior, producing accurate predictions across a total of six different domains. We propose a method for automatically constructing the Hypothesis Space and prior distribution of a Bayesian word learning model using freely available online resources. In particular, we use WordNet (Fellbaum, 2010; Miller, 1995) as an initial source for automatically creating the Hypothesis Space, and ImageNet (Deng et al., 2009) as a source of natu- ralistic images that can be used as stimuli to test the resulting model in behavioral experiments. WordNet is a popular lexi- cal database of English comprised of over 100,000 relational sets of synonyms. ImageNet is a large ontology of images conforming to the hierarchical structure of WordNet, with the aim of providing over 500 high-quality images per noun in WordNet. These resources allow us to construct Hypothesis Spaces and prior distributions for word learning without elic- iting a single judgment from participants and test the result- ing model on a much larger scale than was previously pos- sible. We demonstrate that the Bayesian model formulated from WordNet captures participant judgments in two behav- ioral experiments, addressing the practical and theoretical is- sues with Bayesian models discussed earlier. Keywords: generalization; concept learning; word learning; Bayesian modeling; online databases Introduction Many problems solved by the mind conform to the same ab- stract computational formulation: How should a property be generalized to novel stimuli from a set of stimuli observed to have the property? As there are many ways to extend the property that are consistent with some observed evidence, these are problems of induction, where the evidence con- strains, but does not determine, the solution to a problem. The Bayesian generalization framework (Shepard, 1987; Tenen- baum & Griffiths, 2001) has been remarkably successful at explaining human generalization behavior in a wide range of domains. However, its success is largely dependent on the choice of a Hypothesis Space and a prior probability distribu- tion on hypotheses, which are usually hand constructed by the researcher for each specific problem. This is unsatisfy- ing practically, because the models do not scale beyond the originally modeled problem, and theoretically, as it is unclear whether their success is due to the cleverness of the modeler and not because of a deep mathematical property of the com- putational problem that people solve. One possible solution is to use existing sources of infor- mation about the organization of a domain as the basis for specifying a Hypothesis Space and prior. This helps address both the practical and the theoretical concerns raised by the The plan of the rest of the paper is as follows. In the next sections we review the Bayesian generalization model and then examine how Xu and Tenenbaum (2007) constructed the Hypothesis Space for their Bayesian word learning model. We then show how to build a Hypothesis Space from WordNet that can be used to evaluate word learning models on a large scale. Afterwards, we present two experiments utilizing this hypoth-
Sandro Ridella - One of the best experts on this subject based on the ideXlab platform.
-
a support vector machine classifier from a bit constrained sparse and localized Hypothesis Space
International Joint Conference on Neural Network, 2013Co-Authors: Davide Anguita, Alessandro Ghio, Luca Oneto, Sandro RidellaAbstract:Choosing an appropriate Hypothesis Space in classification applications, according to the Structural Risk Minimization (SRM) principle, is of paramount importance to train effective models: in fact, properly selecting the the Space complexity allows to optimize the learned functions performance. This selection is not straightforward, especially (though not solely) when few samples are available for deriving an effective model (e.g. in bioinformatics applications). In this paper, by exploiting a bit-based definition for Support Vector Machine (SVM) classifiers, selected from an Hypothesis Space described according to sparsity and locality principles, we show how the complexity of the corresponding Space of functions can be effectively tuned through the number of bits used for the function representation. Real world datasets are exploited to show how the number of bits and the degree of sparsity/locality imposed to define the Hypothesis Space affect the complexity of the Space of classifiers and, consequently, the performance of the model, picked up from this set.
-
IJCNN - A support vector machine classifier from a bit-constrained, sparse and localized Hypothesis Space
The 2013 International Joint Conference on Neural Networks (IJCNN), 2013Co-Authors: Davide Anguita, Alessandro Ghio, Luca Oneto, Sandro RidellaAbstract:Choosing an appropriate Hypothesis Space in classification applications, according to the Structural Risk Minimization (SRM) principle, is of paramount importance to train effective models: in fact, properly selecting the the Space complexity allows to optimize the learned functions performance. This selection is not straightforward, especially (though not solely) when few samples are available for deriving an effective model (e.g. in bioinformatics applications). In this paper, by exploiting a bit-based definition for Support Vector Machine (SVM) classifiers, selected from an Hypothesis Space described according to sparsity and locality principles, we show how the complexity of the corresponding Space of functions can be effectively tuned through the number of bits used for the function representation. Real world datasets are exploited to show how the number of bits and the degree of sparsity/locality imposed to define the Hypothesis Space affect the complexity of the Space of classifiers and, consequently, the performance of the model, picked up from this set.
-
a learning machine with a bit based Hypothesis Space
The European Symposium on Artificial Neural Networks, 2013Co-Authors: Davide Anguita, Alessandro Ghio, Luca Oneto, Sandro RidellaAbstract:We propose in this paper a bit-based classifier, picked from an Hypothesis Space described accordingly to sparsity and locality princi- ples: the complexity of the corresponding Space of functions is controlled through the number of bits needed to represent it, so that it will include the classifiers that will be most likely chosen by the learning procedure. Through an introductory example, we show how the number of bits, the sparsity of the representation and the local definition approach affect the complexity of the Space of functions, where the final classifier is selected from.
-
ESANN - A Learning Machine with a Bit-Based Hypothesis Space
2013Co-Authors: Davide Anguita, Alessandro Ghio, Luca Oneto, Sandro RidellaAbstract:We propose in this paper a bit-based classifier, picked from an Hypothesis Space described accordingly to sparsity and locality princi- ples: the complexity of the corresponding Space of functions is controlled through the number of bits needed to represent it, so that it will include the classifiers that will be most likely chosen by the learning procedure. Through an introductory example, we show how the number of bits, the sparsity of the representation and the local definition approach affect the complexity of the Space of functions, where the final classifier is selected from.
-
IJCNN - Selecting the Hypothesis Space for improving the generalization ability of Support Vector Machines
The 2011 International Joint Conference on Neural Networks, 2011Co-Authors: Davide Anguita, Alessandro Ghio, Luca Oneto, Sandro RidellaAbstract:The Structural Risk Minimization framework has been recently proposed as a practical method for model selection in Support Vector Machines (SVMs). The main idea is to effectively measure the complexity of the Hypothesis Space, as defined by the set of possible classifiers, and to use this quantity as a penalty term for guiding the model selection process. Unfortunately, the conventional SVM formulation defines a Hypothesis Space centered at the origin, which can cause undesired effects on the selection of the optimal classifier. We propose here a more flexible SVM formulation, which addresses this drawback, and describe a practical method for selecting more effective Hypothesis Spaces, leading to the improvement of the generalization ability of the final classifier.
Joseph L Austerweil - One of the best experts on this subject based on the ideXlab platform.
-
Learning How to Generalize
Cognitive science, 2019Co-Authors: Joseph L Austerweil, Sophia Sanborn, Thomas L GriffithsAbstract:Generalization is a fundamental problem solved by every cognitive system in essentially every domain. Although it is known that how people generalize varies in complex ways depending on the context or domain, it is an open question how people learn the appropriate way to generalize for a new context. To understand this capability, we cast the problem of learning how to generalize as a problem of learning the appropriate Hypothesis Space for generalization. We propose a normative mathematical framework for learning how to generalize by learning inductive biases for which properties are relevant for generalization in a domain from the statistical structure of features and concepts observed in that domain. More formally, the framework predicts that an ideal learner should learn to generalize by either taking the weighted average of the results of generalizing according to each Hypothesis Space, with weights given by how well each Hypothesis Space fits the previously observed concepts, or by using the most likely Hypothesis Space. We compare the predictions of this framework to human generalization behavior with three experiments in one perceptual (rectangles) and two conceptual (animals and numbers) domains. Across all three studies we find support for the framework's predictions, including individual-level support for averaging in the third study.
-
constructing a Hypothesis Space from the web for large scale bayesian word learning
Cognitive Science, 2012Co-Authors: Joshua T Abbott, Joseph L Austerweil, Thomas L GriffithsAbstract:Constructing a Hypothesis Space from the Web for large-scale Bayesian word learning Joshua T. Abbott (joshua.abbott@berkeley.edu) Joseph L. Austerweil (joseph.austerweil@gmail.com) Thomas L. Griffiths (tom griffiths@berkeley.edu) Department of Psychology, University of California, Berkeley, CA 94720 USA Abstract Bayesian generalization model. In this paper, we use this approach to show how a Hypothesis Space and prior can be constructed automatically from a large online database, mak- ing it possible to apply the Bayesian generalization frame- work to a wide range of naturalistic stimuli. We focus on one specific generalization problem, word learning, where peo- ple learn new words from observing a few objects that can be labeled with that word. Given that the number of possible ex- tensions of a word is essentially infinite, learning the objects referred to by a word is a very difficult inductive problem (Quine, 1975). Xu and Tenenbaum (2007) showed how the Bayesian generalization framework could be used to explain how people learn new words. However, to construct the hy- pothesis Space of their Bayesian model, Xu and Tenenbaum (2007) elicited approximately 400 similarity judgments from their participants. Clearly this is not practical to extend into every domain where people learn words. Thus, word learn- ing is an appropriate setting for exploring novel methods of constructing Hypothesis Spaces and prior distributions. The Bayesian generalization framework has been successful in explaining how people generalize a property from a few observed stimuli to novel stimuli, across several different domains. To create a successful Bayesian generalization model, modelers typically specify a Hypothesis Space and prior probability distribution for each specific domain. How- ever, this raises two problems: the models do not scale beyond the (typically small-scale) domain that they were designed for, and the explanatory power of the models is reduced by their reliance on a hand-coded Hypothesis Space and prior. To solve these two problems, we propose a method for deriving Hypothesis Spaces and priors from large online databases. We evaluate our method by constructing a Hypothesis Space and prior for a Bayesian word learning model from WordNet, a large online database that encodes the semantic relationships between words as a network. After validating our approach by replicating a previous word learning study, we apply the same model to a new experiment featuring three additional taxonomic domains (clothing, containers, and seats). In both experiments, we found that the same automatically constructed Hypothesis Space explains the complex pattern of generalization behavior, producing accurate predictions across a total of six different domains. We propose a method for automatically constructing the Hypothesis Space and prior distribution of a Bayesian word learning model using freely available online resources. In particular, we use WordNet (Fellbaum, 2010; Miller, 1995) as an initial source for automatically creating the Hypothesis Space, and ImageNet (Deng et al., 2009) as a source of natu- ralistic images that can be used as stimuli to test the resulting model in behavioral experiments. WordNet is a popular lexi- cal database of English comprised of over 100,000 relational sets of synonyms. ImageNet is a large ontology of images conforming to the hierarchical structure of WordNet, with the aim of providing over 500 high-quality images per noun in WordNet. These resources allow us to construct Hypothesis Spaces and prior distributions for word learning without elic- iting a single judgment from participants and test the result- ing model on a much larger scale than was previously pos- sible. We demonstrate that the Bayesian model formulated from WordNet captures participant judgments in two behav- ioral experiments, addressing the practical and theoretical is- sues with Bayesian models discussed earlier. Keywords: generalization; concept learning; word learning; Bayesian modeling; online databases Introduction Many problems solved by the mind conform to the same ab- stract computational formulation: How should a property be generalized to novel stimuli from a set of stimuli observed to have the property? As there are many ways to extend the property that are consistent with some observed evidence, these are problems of induction, where the evidence con- strains, but does not determine, the solution to a problem. The Bayesian generalization framework (Shepard, 1987; Tenen- baum & Griffiths, 2001) has been remarkably successful at explaining human generalization behavior in a wide range of domains. However, its success is largely dependent on the choice of a Hypothesis Space and a prior probability distribu- tion on hypotheses, which are usually hand constructed by the researcher for each specific problem. This is unsatisfy- ing practically, because the models do not scale beyond the originally modeled problem, and theoretically, as it is unclear whether their success is due to the cleverness of the modeler and not because of a deep mathematical property of the com- putational problem that people solve. One possible solution is to use existing sources of infor- mation about the organization of a domain as the basis for specifying a Hypothesis Space and prior. This helps address both the practical and the theoretical concerns raised by the The plan of the rest of the paper is as follows. In the next sections we review the Bayesian generalization model and then examine how Xu and Tenenbaum (2007) constructed the Hypothesis Space for their Bayesian word learning model. We then show how to build a Hypothesis Space from WordNet that can be used to evaluate word learning models on a large scale. Afterwards, we present two experiments utilizing this hypoth-
-
CogSci - Constructing a Hypothesis Space from the Web for large-scale Bayesian word learning.
Cognitive Science, 2012Co-Authors: Joshua T Abbott, Joseph L Austerweil, Thomas L GriffithsAbstract:Constructing a Hypothesis Space from the Web for large-scale Bayesian word learning Joshua T. Abbott (joshua.abbott@berkeley.edu) Joseph L. Austerweil (joseph.austerweil@gmail.com) Thomas L. Griffiths (tom griffiths@berkeley.edu) Department of Psychology, University of California, Berkeley, CA 94720 USA Abstract Bayesian generalization model. In this paper, we use this approach to show how a Hypothesis Space and prior can be constructed automatically from a large online database, mak- ing it possible to apply the Bayesian generalization frame- work to a wide range of naturalistic stimuli. We focus on one specific generalization problem, word learning, where peo- ple learn new words from observing a few objects that can be labeled with that word. Given that the number of possible ex- tensions of a word is essentially infinite, learning the objects referred to by a word is a very difficult inductive problem (Quine, 1975). Xu and Tenenbaum (2007) showed how the Bayesian generalization framework could be used to explain how people learn new words. However, to construct the hy- pothesis Space of their Bayesian model, Xu and Tenenbaum (2007) elicited approximately 400 similarity judgments from their participants. Clearly this is not practical to extend into every domain where people learn words. Thus, word learn- ing is an appropriate setting for exploring novel methods of constructing Hypothesis Spaces and prior distributions. The Bayesian generalization framework has been successful in explaining how people generalize a property from a few observed stimuli to novel stimuli, across several different domains. To create a successful Bayesian generalization model, modelers typically specify a Hypothesis Space and prior probability distribution for each specific domain. How- ever, this raises two problems: the models do not scale beyond the (typically small-scale) domain that they were designed for, and the explanatory power of the models is reduced by their reliance on a hand-coded Hypothesis Space and prior. To solve these two problems, we propose a method for deriving Hypothesis Spaces and priors from large online databases. We evaluate our method by constructing a Hypothesis Space and prior for a Bayesian word learning model from WordNet, a large online database that encodes the semantic relationships between words as a network. After validating our approach by replicating a previous word learning study, we apply the same model to a new experiment featuring three additional taxonomic domains (clothing, containers, and seats). In both experiments, we found that the same automatically constructed Hypothesis Space explains the complex pattern of generalization behavior, producing accurate predictions across a total of six different domains. We propose a method for automatically constructing the Hypothesis Space and prior distribution of a Bayesian word learning model using freely available online resources. In particular, we use WordNet (Fellbaum, 2010; Miller, 1995) as an initial source for automatically creating the Hypothesis Space, and ImageNet (Deng et al., 2009) as a source of natu- ralistic images that can be used as stimuli to test the resulting model in behavioral experiments. WordNet is a popular lexi- cal database of English comprised of over 100,000 relational sets of synonyms. ImageNet is a large ontology of images conforming to the hierarchical structure of WordNet, with the aim of providing over 500 high-quality images per noun in WordNet. These resources allow us to construct Hypothesis Spaces and prior distributions for word learning without elic- iting a single judgment from participants and test the result- ing model on a much larger scale than was previously pos- sible. We demonstrate that the Bayesian model formulated from WordNet captures participant judgments in two behav- ioral experiments, addressing the practical and theoretical is- sues with Bayesian models discussed earlier. Keywords: generalization; concept learning; word learning; Bayesian modeling; online databases Introduction Many problems solved by the mind conform to the same ab- stract computational formulation: How should a property be generalized to novel stimuli from a set of stimuli observed to have the property? As there are many ways to extend the property that are consistent with some observed evidence, these are problems of induction, where the evidence con- strains, but does not determine, the solution to a problem. The Bayesian generalization framework (Shepard, 1987; Tenen- baum & Griffiths, 2001) has been remarkably successful at explaining human generalization behavior in a wide range of domains. However, its success is largely dependent on the choice of a Hypothesis Space and a prior probability distribu- tion on hypotheses, which are usually hand constructed by the researcher for each specific problem. This is unsatisfy- ing practically, because the models do not scale beyond the originally modeled problem, and theoretically, as it is unclear whether their success is due to the cleverness of the modeler and not because of a deep mathematical property of the com- putational problem that people solve. One possible solution is to use existing sources of infor- mation about the organization of a domain as the basis for specifying a Hypothesis Space and prior. This helps address both the practical and the theoretical concerns raised by the The plan of the rest of the paper is as follows. In the next sections we review the Bayesian generalization model and then examine how Xu and Tenenbaum (2007) constructed the Hypothesis Space for their Bayesian word learning model. We then show how to build a Hypothesis Space from WordNet that can be used to evaluate word learning models on a large scale. Afterwards, we present two experiments utilizing this hypoth-
Davide Anguita - One of the best experts on this subject based on the ideXlab platform.
-
a support vector machine classifier from a bit constrained sparse and localized Hypothesis Space
International Joint Conference on Neural Network, 2013Co-Authors: Davide Anguita, Alessandro Ghio, Luca Oneto, Sandro RidellaAbstract:Choosing an appropriate Hypothesis Space in classification applications, according to the Structural Risk Minimization (SRM) principle, is of paramount importance to train effective models: in fact, properly selecting the the Space complexity allows to optimize the learned functions performance. This selection is not straightforward, especially (though not solely) when few samples are available for deriving an effective model (e.g. in bioinformatics applications). In this paper, by exploiting a bit-based definition for Support Vector Machine (SVM) classifiers, selected from an Hypothesis Space described according to sparsity and locality principles, we show how the complexity of the corresponding Space of functions can be effectively tuned through the number of bits used for the function representation. Real world datasets are exploited to show how the number of bits and the degree of sparsity/locality imposed to define the Hypothesis Space affect the complexity of the Space of classifiers and, consequently, the performance of the model, picked up from this set.
-
IJCNN - A support vector machine classifier from a bit-constrained, sparse and localized Hypothesis Space
The 2013 International Joint Conference on Neural Networks (IJCNN), 2013Co-Authors: Davide Anguita, Alessandro Ghio, Luca Oneto, Sandro RidellaAbstract:Choosing an appropriate Hypothesis Space in classification applications, according to the Structural Risk Minimization (SRM) principle, is of paramount importance to train effective models: in fact, properly selecting the the Space complexity allows to optimize the learned functions performance. This selection is not straightforward, especially (though not solely) when few samples are available for deriving an effective model (e.g. in bioinformatics applications). In this paper, by exploiting a bit-based definition for Support Vector Machine (SVM) classifiers, selected from an Hypothesis Space described according to sparsity and locality principles, we show how the complexity of the corresponding Space of functions can be effectively tuned through the number of bits used for the function representation. Real world datasets are exploited to show how the number of bits and the degree of sparsity/locality imposed to define the Hypothesis Space affect the complexity of the Space of classifiers and, consequently, the performance of the model, picked up from this set.
-
a learning machine with a bit based Hypothesis Space
The European Symposium on Artificial Neural Networks, 2013Co-Authors: Davide Anguita, Alessandro Ghio, Luca Oneto, Sandro RidellaAbstract:We propose in this paper a bit-based classifier, picked from an Hypothesis Space described accordingly to sparsity and locality princi- ples: the complexity of the corresponding Space of functions is controlled through the number of bits needed to represent it, so that it will include the classifiers that will be most likely chosen by the learning procedure. Through an introductory example, we show how the number of bits, the sparsity of the representation and the local definition approach affect the complexity of the Space of functions, where the final classifier is selected from.
-
ESANN - A Learning Machine with a Bit-Based Hypothesis Space
2013Co-Authors: Davide Anguita, Alessandro Ghio, Luca Oneto, Sandro RidellaAbstract:We propose in this paper a bit-based classifier, picked from an Hypothesis Space described accordingly to sparsity and locality princi- ples: the complexity of the corresponding Space of functions is controlled through the number of bits needed to represent it, so that it will include the classifiers that will be most likely chosen by the learning procedure. Through an introductory example, we show how the number of bits, the sparsity of the representation and the local definition approach affect the complexity of the Space of functions, where the final classifier is selected from.
-
IJCNN - Selecting the Hypothesis Space for improving the generalization ability of Support Vector Machines
The 2011 International Joint Conference on Neural Networks, 2011Co-Authors: Davide Anguita, Alessandro Ghio, Luca Oneto, Sandro RidellaAbstract:The Structural Risk Minimization framework has been recently proposed as a practical method for model selection in Support Vector Machines (SVMs). The main idea is to effectively measure the complexity of the Hypothesis Space, as defined by the set of possible classifiers, and to use this quantity as a penalty term for guiding the model selection process. Unfortunately, the conventional SVM formulation defines a Hypothesis Space centered at the origin, which can cause undesired effects on the selection of the optimal classifier. We propose here a more flexible SVM formulation, which addresses this drawback, and describe a practical method for selecting more effective Hypothesis Spaces, leading to the improvement of the generalization ability of the final classifier.
Joshua T Abbott - One of the best experts on this subject based on the ideXlab platform.
-
constructing a Hypothesis Space from the web for large scale bayesian word learning
Cognitive Science, 2012Co-Authors: Joshua T Abbott, Joseph L Austerweil, Thomas L GriffithsAbstract:Constructing a Hypothesis Space from the Web for large-scale Bayesian word learning Joshua T. Abbott (joshua.abbott@berkeley.edu) Joseph L. Austerweil (joseph.austerweil@gmail.com) Thomas L. Griffiths (tom griffiths@berkeley.edu) Department of Psychology, University of California, Berkeley, CA 94720 USA Abstract Bayesian generalization model. In this paper, we use this approach to show how a Hypothesis Space and prior can be constructed automatically from a large online database, mak- ing it possible to apply the Bayesian generalization frame- work to a wide range of naturalistic stimuli. We focus on one specific generalization problem, word learning, where peo- ple learn new words from observing a few objects that can be labeled with that word. Given that the number of possible ex- tensions of a word is essentially infinite, learning the objects referred to by a word is a very difficult inductive problem (Quine, 1975). Xu and Tenenbaum (2007) showed how the Bayesian generalization framework could be used to explain how people learn new words. However, to construct the hy- pothesis Space of their Bayesian model, Xu and Tenenbaum (2007) elicited approximately 400 similarity judgments from their participants. Clearly this is not practical to extend into every domain where people learn words. Thus, word learn- ing is an appropriate setting for exploring novel methods of constructing Hypothesis Spaces and prior distributions. The Bayesian generalization framework has been successful in explaining how people generalize a property from a few observed stimuli to novel stimuli, across several different domains. To create a successful Bayesian generalization model, modelers typically specify a Hypothesis Space and prior probability distribution for each specific domain. How- ever, this raises two problems: the models do not scale beyond the (typically small-scale) domain that they were designed for, and the explanatory power of the models is reduced by their reliance on a hand-coded Hypothesis Space and prior. To solve these two problems, we propose a method for deriving Hypothesis Spaces and priors from large online databases. We evaluate our method by constructing a Hypothesis Space and prior for a Bayesian word learning model from WordNet, a large online database that encodes the semantic relationships between words as a network. After validating our approach by replicating a previous word learning study, we apply the same model to a new experiment featuring three additional taxonomic domains (clothing, containers, and seats). In both experiments, we found that the same automatically constructed Hypothesis Space explains the complex pattern of generalization behavior, producing accurate predictions across a total of six different domains. We propose a method for automatically constructing the Hypothesis Space and prior distribution of a Bayesian word learning model using freely available online resources. In particular, we use WordNet (Fellbaum, 2010; Miller, 1995) as an initial source for automatically creating the Hypothesis Space, and ImageNet (Deng et al., 2009) as a source of natu- ralistic images that can be used as stimuli to test the resulting model in behavioral experiments. WordNet is a popular lexi- cal database of English comprised of over 100,000 relational sets of synonyms. ImageNet is a large ontology of images conforming to the hierarchical structure of WordNet, with the aim of providing over 500 high-quality images per noun in WordNet. These resources allow us to construct Hypothesis Spaces and prior distributions for word learning without elic- iting a single judgment from participants and test the result- ing model on a much larger scale than was previously pos- sible. We demonstrate that the Bayesian model formulated from WordNet captures participant judgments in two behav- ioral experiments, addressing the practical and theoretical is- sues with Bayesian models discussed earlier. Keywords: generalization; concept learning; word learning; Bayesian modeling; online databases Introduction Many problems solved by the mind conform to the same ab- stract computational formulation: How should a property be generalized to novel stimuli from a set of stimuli observed to have the property? As there are many ways to extend the property that are consistent with some observed evidence, these are problems of induction, where the evidence con- strains, but does not determine, the solution to a problem. The Bayesian generalization framework (Shepard, 1987; Tenen- baum & Griffiths, 2001) has been remarkably successful at explaining human generalization behavior in a wide range of domains. However, its success is largely dependent on the choice of a Hypothesis Space and a prior probability distribu- tion on hypotheses, which are usually hand constructed by the researcher for each specific problem. This is unsatisfy- ing practically, because the models do not scale beyond the originally modeled problem, and theoretically, as it is unclear whether their success is due to the cleverness of the modeler and not because of a deep mathematical property of the com- putational problem that people solve. One possible solution is to use existing sources of infor- mation about the organization of a domain as the basis for specifying a Hypothesis Space and prior. This helps address both the practical and the theoretical concerns raised by the The plan of the rest of the paper is as follows. In the next sections we review the Bayesian generalization model and then examine how Xu and Tenenbaum (2007) constructed the Hypothesis Space for their Bayesian word learning model. We then show how to build a Hypothesis Space from WordNet that can be used to evaluate word learning models on a large scale. Afterwards, we present two experiments utilizing this hypoth-
-
CogSci - Constructing a Hypothesis Space from the Web for large-scale Bayesian word learning.
Cognitive Science, 2012Co-Authors: Joshua T Abbott, Joseph L Austerweil, Thomas L GriffithsAbstract:Constructing a Hypothesis Space from the Web for large-scale Bayesian word learning Joshua T. Abbott (joshua.abbott@berkeley.edu) Joseph L. Austerweil (joseph.austerweil@gmail.com) Thomas L. Griffiths (tom griffiths@berkeley.edu) Department of Psychology, University of California, Berkeley, CA 94720 USA Abstract Bayesian generalization model. In this paper, we use this approach to show how a Hypothesis Space and prior can be constructed automatically from a large online database, mak- ing it possible to apply the Bayesian generalization frame- work to a wide range of naturalistic stimuli. We focus on one specific generalization problem, word learning, where peo- ple learn new words from observing a few objects that can be labeled with that word. Given that the number of possible ex- tensions of a word is essentially infinite, learning the objects referred to by a word is a very difficult inductive problem (Quine, 1975). Xu and Tenenbaum (2007) showed how the Bayesian generalization framework could be used to explain how people learn new words. However, to construct the hy- pothesis Space of their Bayesian model, Xu and Tenenbaum (2007) elicited approximately 400 similarity judgments from their participants. Clearly this is not practical to extend into every domain where people learn words. Thus, word learn- ing is an appropriate setting for exploring novel methods of constructing Hypothesis Spaces and prior distributions. The Bayesian generalization framework has been successful in explaining how people generalize a property from a few observed stimuli to novel stimuli, across several different domains. To create a successful Bayesian generalization model, modelers typically specify a Hypothesis Space and prior probability distribution for each specific domain. How- ever, this raises two problems: the models do not scale beyond the (typically small-scale) domain that they were designed for, and the explanatory power of the models is reduced by their reliance on a hand-coded Hypothesis Space and prior. To solve these two problems, we propose a method for deriving Hypothesis Spaces and priors from large online databases. We evaluate our method by constructing a Hypothesis Space and prior for a Bayesian word learning model from WordNet, a large online database that encodes the semantic relationships between words as a network. After validating our approach by replicating a previous word learning study, we apply the same model to a new experiment featuring three additional taxonomic domains (clothing, containers, and seats). In both experiments, we found that the same automatically constructed Hypothesis Space explains the complex pattern of generalization behavior, producing accurate predictions across a total of six different domains. We propose a method for automatically constructing the Hypothesis Space and prior distribution of a Bayesian word learning model using freely available online resources. In particular, we use WordNet (Fellbaum, 2010; Miller, 1995) as an initial source for automatically creating the Hypothesis Space, and ImageNet (Deng et al., 2009) as a source of natu- ralistic images that can be used as stimuli to test the resulting model in behavioral experiments. WordNet is a popular lexi- cal database of English comprised of over 100,000 relational sets of synonyms. ImageNet is a large ontology of images conforming to the hierarchical structure of WordNet, with the aim of providing over 500 high-quality images per noun in WordNet. These resources allow us to construct Hypothesis Spaces and prior distributions for word learning without elic- iting a single judgment from participants and test the result- ing model on a much larger scale than was previously pos- sible. We demonstrate that the Bayesian model formulated from WordNet captures participant judgments in two behav- ioral experiments, addressing the practical and theoretical is- sues with Bayesian models discussed earlier. Keywords: generalization; concept learning; word learning; Bayesian modeling; online databases Introduction Many problems solved by the mind conform to the same ab- stract computational formulation: How should a property be generalized to novel stimuli from a set of stimuli observed to have the property? As there are many ways to extend the property that are consistent with some observed evidence, these are problems of induction, where the evidence con- strains, but does not determine, the solution to a problem. The Bayesian generalization framework (Shepard, 1987; Tenen- baum & Griffiths, 2001) has been remarkably successful at explaining human generalization behavior in a wide range of domains. However, its success is largely dependent on the choice of a Hypothesis Space and a prior probability distribu- tion on hypotheses, which are usually hand constructed by the researcher for each specific problem. This is unsatisfy- ing practically, because the models do not scale beyond the originally modeled problem, and theoretically, as it is unclear whether their success is due to the cleverness of the modeler and not because of a deep mathematical property of the com- putational problem that people solve. One possible solution is to use existing sources of infor- mation about the organization of a domain as the basis for specifying a Hypothesis Space and prior. This helps address both the practical and the theoretical concerns raised by the The plan of the rest of the paper is as follows. In the next sections we review the Bayesian generalization model and then examine how Xu and Tenenbaum (2007) constructed the Hypothesis Space for their Bayesian word learning model. We then show how to build a Hypothesis Space from WordNet that can be used to evaluate word learning models on a large scale. Afterwards, we present two experiments utilizing this hypoth-