The Experts below are selected from a list of 39 Experts worldwide ranked by ideXlab platform
Ambuj K. Singh - One of the best experts on this subject based on the ideXlab platform.
-
A Latent Parameter Node-Centric Model for Spatial Networks
PloS one, 2013Co-Authors: Nicholas D. Larusso, Brian E. Ruttenberg, Ambuj K. SinghAbstract:Spatial networks, in which Nodes and edges are embedded in space, play a vital role in the study of complex systems. For example, many social networks attach geo-location information to each user, allowing the study of not only topological interactions between users, but spatial interactions as well. The defining property of spatial networks is that edge distances are associated with a cost, which may subtly influence the topology of the network. However, the cost function over distance is rarely known, thus developing a model of connections in spatial networks is a difficult task. In this paper, we introduce a novel model for capturing the interaction between spatial effects and network structure. Our approach represents a unique combination of ideas from latent variable statistical models and spatial network modeling. In contrast to previous work, we view the ability to form long/short-distance connections to be dependent on the individual Nodes involved. For example, a Node's specific surroundings (e.g. network structure and Node density) may make it more likely to form a long distance link than other Nodes with the same degree. To capture this information, we attach a latent variable to each Node which represents a Node's spatial reach. These variables are inferred from the network structure using a Markov Chain Monte Carlo algorithm. We experimentally evaluate our proposed model on 4 different types of real-world spatial networks (e.g. transportation, biological, infrastructure, and social). We apply our model to the task of link prediction and achieve up to a 35% improvement over previous approaches in terms of the area under the ROC curve. Additionally, we show that our model is particularly helpful for predicting links between Nodes with low degrees. In these cases, we see much larger improvements over previous models.
Hao Peng - One of the best experts on this subject based on the ideXlab platform.
-
incrementally learning the hierarchical softmax function for neural language models
National Conference on Artificial Intelligence, 2017Co-Authors: Hao Peng, Yangqiu Song, Yaopeng LiuAbstract:Neural network language models (NNLMs) have attracted a lot of attention recently. In this paper, we present a training method that can incrementally train the hierarchical softmax function for NNMLs. We split the cost function to model old and update corpora separately, and factorize the objective function for the hierarchical softmax. Then we provide a new stochastic gradient based method to update all the word vectors and Parameters, by comparing the old tree generated based on the old corpus and the new tree generated based on the combined (old and update) corpus. Theoretical analysis shows that the mean square error of the Parameter vectors can be bounded by a function of the number of changed words related to the Parameter Node. Experimental results show that incremental training can save a lot of time. The smaller the update corpus is, the faster the update training process is, where an up to 30 times speedup has been achieved. We also use both word similarity/relatedness tasks and dependency parsing task as our benchmarks to evaluate the correctness of the updated word vectors.
-
AAAI - Incrementally learning the hierarchical softmax function for neural language models
2017Co-Authors: Hao Peng, Yangqiu Song, Liu YaopengAbstract:Neural network language models (NNLMs) have attracted a lot of attention recently. In this paper, we present a training method that can incrementally train the hierarchical softmax function for NNMLs. We split the cost function to model old and update corpora separately, and factorize the objective function for the hierarchical softmax. Then we provide a new stochastic gradient based method to update all the word vectors and Parameters, by comparing the old tree generated based on the old corpus and the new tree generated based on the combined (old and update) corpus. Theoretical analysis shows that the mean square error of the Parameter vectors can be bounded by a function of the number of changed words related to the Parameter Node. Experimental results show that incremental training can save a lot of time. The smaller the update corpus is, the faster the update training process is, where an up to 30 times speedup has been achieved. We also use both word similarity/relatedness tasks and dependency parsing task as our benchmarks to evaluate the correctness of the updated word vectors.
Yaopeng Liu - One of the best experts on this subject based on the ideXlab platform.
-
incrementally learning the hierarchical softmax function for neural language models
National Conference on Artificial Intelligence, 2017Co-Authors: Hao Peng, Yangqiu Song, Yaopeng LiuAbstract:Neural network language models (NNLMs) have attracted a lot of attention recently. In this paper, we present a training method that can incrementally train the hierarchical softmax function for NNMLs. We split the cost function to model old and update corpora separately, and factorize the objective function for the hierarchical softmax. Then we provide a new stochastic gradient based method to update all the word vectors and Parameters, by comparing the old tree generated based on the old corpus and the new tree generated based on the combined (old and update) corpus. Theoretical analysis shows that the mean square error of the Parameter vectors can be bounded by a function of the number of changed words related to the Parameter Node. Experimental results show that incremental training can save a lot of time. The smaller the update corpus is, the faster the update training process is, where an up to 30 times speedup has been achieved. We also use both word similarity/relatedness tasks and dependency parsing task as our benchmarks to evaluate the correctness of the updated word vectors.
Liu Yaopeng - One of the best experts on this subject based on the ideXlab platform.
-
AAAI - Incrementally learning the hierarchical softmax function for neural language models
2017Co-Authors: Hao Peng, Yangqiu Song, Liu YaopengAbstract:Neural network language models (NNLMs) have attracted a lot of attention recently. In this paper, we present a training method that can incrementally train the hierarchical softmax function for NNMLs. We split the cost function to model old and update corpora separately, and factorize the objective function for the hierarchical softmax. Then we provide a new stochastic gradient based method to update all the word vectors and Parameters, by comparing the old tree generated based on the old corpus and the new tree generated based on the combined (old and update) corpus. Theoretical analysis shows that the mean square error of the Parameter vectors can be bounded by a function of the number of changed words related to the Parameter Node. Experimental results show that incremental training can save a lot of time. The smaller the update corpus is, the faster the update training process is, where an up to 30 times speedup has been achieved. We also use both word similarity/relatedness tasks and dependency parsing task as our benchmarks to evaluate the correctness of the updated word vectors.
Nicholas D. Larusso - One of the best experts on this subject based on the ideXlab platform.
-
A Latent Parameter Node-Centric Model for Spatial Networks
PloS one, 2013Co-Authors: Nicholas D. Larusso, Brian E. Ruttenberg, Ambuj K. SinghAbstract:Spatial networks, in which Nodes and edges are embedded in space, play a vital role in the study of complex systems. For example, many social networks attach geo-location information to each user, allowing the study of not only topological interactions between users, but spatial interactions as well. The defining property of spatial networks is that edge distances are associated with a cost, which may subtly influence the topology of the network. However, the cost function over distance is rarely known, thus developing a model of connections in spatial networks is a difficult task. In this paper, we introduce a novel model for capturing the interaction between spatial effects and network structure. Our approach represents a unique combination of ideas from latent variable statistical models and spatial network modeling. In contrast to previous work, we view the ability to form long/short-distance connections to be dependent on the individual Nodes involved. For example, a Node's specific surroundings (e.g. network structure and Node density) may make it more likely to form a long distance link than other Nodes with the same degree. To capture this information, we attach a latent variable to each Node which represents a Node's spatial reach. These variables are inferred from the network structure using a Markov Chain Monte Carlo algorithm. We experimentally evaluate our proposed model on 4 different types of real-world spatial networks (e.g. transportation, biological, infrastructure, and social). We apply our model to the task of link prediction and achieve up to a 35% improvement over previous approaches in terms of the area under the ROC curve. Additionally, we show that our model is particularly helpful for predicting links between Nodes with low degrees. In these cases, we see much larger improvements over previous models.