The Experts below are selected from a list of 50193 Experts worldwide ranked by ideXlab platform
Kawin Ethayarajh - One of the best experts on this subject based on the ideXlab platform.
-
rotate king to get queen word relationships as orthogonal transformations in Embedding Space
Empirical Methods in Natural Language Processing, 2019Co-Authors: Kawin EthayarajhAbstract:A notable property of word Embeddings is that word relationships can exist as linear substructures in the Embedding Space. For example, ‘gender’ corresponds to v_woman - v_man and v_queen - v_king. This, in turn, allows word analogies to be solved arithmetically: v_king - v_man + v_woman = v_queen. This property is notable because it suggests that models trained on word Embeddings can easily learn such relationships as geometric translations. However, there is no evidence that models exclusively represent relationships in this manner. We document an alternative way in which downstream models might learn these relationships: orthogonal and linear transformations. For example, given a translation vector for ‘gender’, we can find an orthogonal matrix R, representing a rotation and reflection, such that R(v_king) = v_queen and R(v_man) = v_woman. Analogical reasoning using orthogonal transformations is almost as accurate as using vector arithmetic; using linear transformations is more accurate than both. Our findings suggest that these transformations can be as good a representation of word relationships as translation vectors.
-
rotate king to get queen word relationships as orthogonal transformations in Embedding Space
arXiv: Computation and Language, 2019Co-Authors: Kawin EthayarajhAbstract:A notable property of word Embeddings is that word relationships can exist as linear substructures in the Embedding Space. For example, $\textit{gender}$ corresponds to $\vec{\textit{woman}} - \vec{\textit{man}}$ and $\vec{\textit{queen}} - \vec{\textit{king}}$. This, in turn, allows word analogies to be solved arithmetically: $\vec{\textit{king}} - \vec{\textit{man}} + \vec{\textit{woman}} \approx \vec{\textit{queen}}$. This property is notable because it suggests that models trained on word Embeddings can easily learn such relationships as geometric translations. However, there is no evidence that models $\textit{exclusively}$ represent relationships in this manner. We document an alternative way in which downstream models might learn these relationships: orthogonal and linear transformations. For example, given a translation vector for $\textit{gender}$, we can find an orthogonal matrix $R$, representing a rotation and reflection, such that $R(\vec{\textit{king}}) \approx \vec{\textit{queen}}$ and $R(\vec{\textit{man}}) \approx \vec{\textit{woman}}$. Analogical reasoning using orthogonal transformations is almost as accurate as using vector arithmetic; using linear transformations is more accurate than both. Our findings suggest that these transformations can be as good a representation of word relationships as translation vectors.
Jinghao Zhao - One of the best experts on this subject based on the ideXlab platform.
-
skip the question you don t know an Embedding Space approach
International Joint Conference on Neural Network, 2019Co-Authors: Kaiyuan Chen, Jinghao ZhaoAbstract:Deep neural network gives people power to generalize hidden patterns behind training data. However, due to limitations on available data collection methods, what neural networks learn should never be expected to deal with all the scenarios: predicting on samples that rarely appear in training set will have very low accuracy. Thus, we design an end-to-end neural network. It learns an inherent discriminative Embedding on the training set to perform out-of-distribution(OOD) detection and classification at the same time: both OOD data points and points that resemble those with different labels can be visually observed in this Embedding Space. Based on this model, we also devise a training scheme that trains on only inliers. Experiments on various datasets and metrics validate that our method outperforms the state-of-art OOD detector.
-
IJCNN - Skip The Question You Don’t Know: An Embedding Space Approach
2019 International Joint Conference on Neural Networks (IJCNN), 2019Co-Authors: Kaiyuan Chen, Jinghao ZhaoAbstract:Deep neural network gives people power to generalize hidden patterns behind training data. However, due to limitations on available data collection methods, what neural networks learn should never be expected to deal with all the scenarios: predicting on samples that rarely appear in training set will have very low accuracy. Thus, we design an end-to-end neural network. It learns an inherent discriminative Embedding on the training set to perform out-of-distribution(OOD) detection and classification at the same time: both OOD data points and points that resemble those with different labels can be visually observed in this Embedding Space. Based on this model, we also devise a training scheme that trains on only inliers. Experiments on various datasets and metrics validate that our method outperforms the state-of-art OOD detector.
Hinrich Schütze - One of the best experts on this subject based on the ideXlab platform.
-
LREC - Embedding Space Correlation as a Measure of Domain Similarity
2020Co-Authors: Anne Beyer, Göran Kauermann, Hinrich SchützeAbstract:Prior work has determined domain similarity using text-based features of a corpus. However, when using pre-trained word Embeddings, the underlying text corpus might not be accessible anymore. Therefore, we propose the CCA measure, a new measure of domain similarity based directly on the dimension-wise correlations between corresponding Embedding Spaces. Our results suggest that an inherent notion of domain can be captured this way, as we are able to reproduce our findings for different domain comparisons for English, German, Spanish and Czech as well as in cross-lingual comparisons. We further find a threshold at which the CCA measure indicates that two corpora come from the same domain in a monolingual setting by applying permutation tests. By evaluating the usability of the CCA measure in a domain adaptation application, we also show that it can be used to determine which corpora are more similar to each other in a cross-domain sentiment detection task.
-
a multilingual bpe Embedding Space for universal sentiment lexicon induction
Meeting of the Association for Computational Linguistics, 2019Co-Authors: Mengjie Zhao, Hinrich SchützeAbstract:We present a new method for sentiment lex- icon induction that is designed to be appli- cable to the entire range of typological di- versity of the world’s languages. We eval- uate our method on Parallel Bible Corpus+ (PBC+), a parallel corpus of 1593 languages. The key idea is to use Byte Pair Encodings (BPEs) as basic units for multilingual em- beddings. Through zero-shot transfer from English sentiment, we learn a seed lexicon for each language in the domain of PBC+. Through domain adaptation, we then gener- alize the domain-specific lexicon to a general one. We show – across typologically diverse languages in PBC+ – good quality of seed and general-domain sentiment lexicons by intrin- sic and extrinsic and by automatic and human evaluation. We make freely available our code, seed sentiment lexicons for all 1593 languages and induced general-domain sentiment lexi- cons for 200 languages
Arthur W Toga - One of the best experts on this subject based on the ideXlab platform.
-
Metric Optimization for Surface Analysis in the Laplace-Beltrami Embedding Space
IEEE Transactions on Medical Imaging, 2014Co-Authors: Yonggang Shi, Rongjie Lai, Daniel Pelletier, David C Mohr, Nancy L Sicotte, Danny J.j. Wang, Arthur W TogaAbstract:In this paper, we present a novel approach for the intrinsic mapping of anatomical surfaces and its application in brain mapping research. Using the Laplace-Beltrami eigen-system, we represent each surface with an isometry invariant Embedding in a high dimensional Space. The key idea in our system is that we realize surface deformation in the Embedding Space via the iterative optimization of a conformal metric without explicitly perturbing the surface or its Embedding. By minimizing a distance measure in the Embedding Space with metric optimization, our method generates a conformal map directly between surfaces with highly uniform metric distortion and the ability of aligning salient geometric features. Besides pairwise surface maps, we also extend the metric optimization approach for group-wise atlas construction and multi-atlas cortical label fusion. In experimental results, we demonstrate the robustness and generality of our method by applying it to map both cortical and hippocampal surfaces in population studies. For cortical labeling, our method achieves excellent performance in a cross-validation experiment with 40 manually labeled surfaces, and successfully models localized brain development in a pediatric study of 80 subjects. For hippocampal mapping, our method produces much more significant results than two popular tools on a multiple sclerosis study of 109 subjects.
-
conformal metric optimization on surface cmos for deformation and mapping in laplace beltrami Embedding Space
Medical Image Computing and Computer-Assisted Intervention, 2011Co-Authors: Yonggang Shi, Rongjie Lai, Raja Gill, Daniel Pelletier, David C Mohr, Nancy L Sicotte, Arthur W TogaAbstract:In this paper we develop a novel technique for surface deformation and mapping in the high-dimensional Laplace-Beltrami Embedding Space. The key idea of our work is to realize surface deformation in the Embedding Space via optimization of a conformal metric on the surface. Numerical techniques are developed for computing derivatives of the eigenvalues and eigenfunctions with respect to the conformal metric, which is then applied to compute surface maps in the Embedding Space by minimizing an energy function. In our experiments, we demonstrate the robustness of our method by applying it to map hippocampal atrophy of multiple sclerosis patients with depression on a data set of 109 subjects. Statistically significant results have been obtained that show excellent correlation with clinical variables. A comparison with the popular SPHARM tool has also been performed to demonstrate that our method achieves more significant results.
-
MICCAI (2) - Conformal metric optimization on surface (CMOS) for deformation and mapping in laplace-beltrami Embedding Space
Lecture Notes in Computer Science, 2011Co-Authors: Yonggang Shi, Rongjie Lai, Raja Gill, Daniel Pelletier, David C Mohr, Nancy L Sicotte, Arthur W TogaAbstract:In this paper we develop a novel technique for surface deformation and mapping in the high-dimensional Laplace-Beltrami Embedding Space. The key idea of our work is to realize surface deformation in the Embedding Space via optimization of a conformal metric on the surface. Numerical techniques are developed for computing derivatives of the eigenvalues and eigenfunctions with respect to the conformal metric, which is then applied to compute surface maps in the Embedding Space by minimizing an energy function. In our experiments, we demonstrate the robustness of our method by applying it to map hippocampal atrophy of multiple sclerosis patients with depression on a data set of 109 subjects. Statistically significant results have been obtained that show excellent correlation with clinical variables. A comparison with the popular SPHARM tool has also been performed to demonstrate that our method achieves more significant results.
Yonggang Shi - One of the best experts on this subject based on the ideXlab platform.
-
Riemannian metric optimization on surfaces (RMOS) for intrinsic brain mapping in the Laplace-Beltrami Embedding Space.
Medical Image Analysis, 2018Co-Authors: Jin Kyu Gahm, Yonggang ShiAbstract:Abstract Surface mapping methods play an important role in various brain imaging studies from tracking the maturation of adolescent brains to mapping gray matter atrophy patterns in Alzheimer’s disease. Popular surface mapping approaches based on spherical registration, however, have inherent numerical limitations when severe metric distortions are present during the spherical parameterization step. In this paper, we propose a novel computational framework for intrinsic surface mapping in the Laplace–Beltrami (LB) Embedding Space based on Riemannian metric optimization on surfaces (RMOS). Given a diffeomorphism between two surfaces, an isometry can be defined using the pullback metric, which in turn results in identical LB Embeddings from the two surfaces. The proposed RMOS approach builds upon this mathematical foundation and achieves general feature-driven surface mapping in the LB Embedding Space by iteratively optimizing the Riemannian metric defined on the edges of triangular meshes. At the core of our framework is an optimization engine that converts an energy function for surface mapping into a distance measure in the LB Embedding Space, which can be effectively optimized using gradients of the LB eigen-system with respect to the Riemannian metrics. In the experimental results, we compare the RMOS algorithm with spherical registration using large-scale brain imaging data, and show that RMOS achieves superior performance in the prediction of hippocampal subfields and cortical gyral labels, and the holistic mapping of striatal surfaces for the construction of a striatal connectivity atlas from substantia nigra.
-
MICCAI (1) - Holistic Mapping of Striatum Surfaces in the Laplace-Beltrami Embedding Space.
Medical Image Computing and Computer Assisted Intervention − MICCAI 2017, 2017Co-Authors: Jin Kyu Gahm, Yonggang ShiAbstract:In brain shape analysis, the striatum is typically divided into three parts: the caudate, putamen, and accumbens nuclei for its analysis. Recent connectivity and animal studies, however, indicate striatum-cortical inter-connections do not always follow such subdivisions. For the holistic mapping of striatum surfaces, conventional spherical registration techniques are not suitable due to the large metric distortions in spherical parameterization of striatal surfaces. To overcome this difficulty, we develop a novel striatal surface mapping method using our recently proposed Riemannian metric optimization techniques in the Laplace-Beltrami (LB) Embedding Space. For the robust resolution of sign ambiguities in the LB spectrum, we also devise novel anatomical contextual features to guide the surface mapping in the Embedding Space. In our experimental results, we compare with spherical registration tools from FreeSurfer and FSL to demonstrate that our novel method provides a superior solution to the striatal mapping problem. We also apply our method to map the striatal surfaces from 211 subjects of the Human Connectome Project (HCP), and use the surface maps to construct a cortical connectivity atlas. Our atlas results show that the striato-cortical connectivity is not distinctive according to traditional structural subdivision of the striatum, and further confirms the holistic approach for mapping striatal surfaces.
-
Metric Optimization for Surface Analysis in the Laplace-Beltrami Embedding Space
IEEE Transactions on Medical Imaging, 2014Co-Authors: Yonggang Shi, Rongjie Lai, Daniel Pelletier, David C Mohr, Nancy L Sicotte, Danny J.j. Wang, Arthur W TogaAbstract:In this paper, we present a novel approach for the intrinsic mapping of anatomical surfaces and its application in brain mapping research. Using the Laplace-Beltrami eigen-system, we represent each surface with an isometry invariant Embedding in a high dimensional Space. The key idea in our system is that we realize surface deformation in the Embedding Space via the iterative optimization of a conformal metric without explicitly perturbing the surface or its Embedding. By minimizing a distance measure in the Embedding Space with metric optimization, our method generates a conformal map directly between surfaces with highly uniform metric distortion and the ability of aligning salient geometric features. Besides pairwise surface maps, we also extend the metric optimization approach for group-wise atlas construction and multi-atlas cortical label fusion. In experimental results, we demonstrate the robustness and generality of our method by applying it to map both cortical and hippocampal surfaces in population studies. For cortical labeling, our method achieves excellent performance in a cross-validation experiment with 40 manually labeled surfaces, and successfully models localized brain development in a pediatric study of 80 subjects. For hippocampal mapping, our method produces much more significant results than two popular tools on a multiple sclerosis study of 109 subjects.
-
conformal metric optimization on surface cmos for deformation and mapping in laplace beltrami Embedding Space
Medical Image Computing and Computer-Assisted Intervention, 2011Co-Authors: Yonggang Shi, Rongjie Lai, Raja Gill, Daniel Pelletier, David C Mohr, Nancy L Sicotte, Arthur W TogaAbstract:In this paper we develop a novel technique for surface deformation and mapping in the high-dimensional Laplace-Beltrami Embedding Space. The key idea of our work is to realize surface deformation in the Embedding Space via optimization of a conformal metric on the surface. Numerical techniques are developed for computing derivatives of the eigenvalues and eigenfunctions with respect to the conformal metric, which is then applied to compute surface maps in the Embedding Space by minimizing an energy function. In our experiments, we demonstrate the robustness of our method by applying it to map hippocampal atrophy of multiple sclerosis patients with depression on a data set of 109 subjects. Statistically significant results have been obtained that show excellent correlation with clinical variables. A comparison with the popular SPHARM tool has also been performed to demonstrate that our method achieves more significant results.
-
MICCAI (2) - Conformal metric optimization on surface (CMOS) for deformation and mapping in laplace-beltrami Embedding Space
Lecture Notes in Computer Science, 2011Co-Authors: Yonggang Shi, Rongjie Lai, Raja Gill, Daniel Pelletier, David C Mohr, Nancy L Sicotte, Arthur W TogaAbstract:In this paper we develop a novel technique for surface deformation and mapping in the high-dimensional Laplace-Beltrami Embedding Space. The key idea of our work is to realize surface deformation in the Embedding Space via optimization of a conformal metric on the surface. Numerical techniques are developed for computing derivatives of the eigenvalues and eigenfunctions with respect to the conformal metric, which is then applied to compute surface maps in the Embedding Space by minimizing an energy function. In our experiments, we demonstrate the robustness of our method by applying it to map hippocampal atrophy of multiple sclerosis patients with depression on a data set of 109 subjects. Statistically significant results have been obtained that show excellent correlation with clinical variables. A comparison with the popular SPHARM tool has also been performed to demonstrate that our method achieves more significant results.