The Experts below are selected from a list of 216 Experts worldwide ranked by ideXlab platform
Yoshua Bengio - One of the best experts on this subject based on the ideXlab platform.
-
end to end online writer identification with recurrent neural network
IEEE Transactions on Human-Machine Systems, 2017Co-Authors: Xuyao Zhang, Yoshua BengioAbstract:Writer identification is an important topic for pattern recognition and artificial intelligence. Traditional methods rely heavily on sophisticated hand-crafted features to represent the characteristics of different writers. In this paper, we propose an end-to-end framework for online text-independent writer identification by using a recurrent neural network (RNN). Specifically, the handwriting data of a particular writer are represented by a set of random hybrid strokes (RHSs). Each RHS is a randomly sampled short sequence representing pen tip movements ( $xy$ -coordinates) and pen-down or pen-up states. RHS is independent of the content and language involved in handwriting; therefore, writer identification at the RHS level is more general and convenient than the character level or the word level, which also requires character/word segmentation. The RNN model with bidirectional long short-term memory is used to encode each RHS into a fixed-Length Vector for final classification. All the RHSs of a writer are classified independently, and then, the posterior probabilities are averaged to make the final decision. The proposed framework is end-to-end and does not require any domain knowledge for handwriting data analysis. Experiments on both English (133 writers) and Chinese (186 writers) databases verify the advantages of our method compared with other state-of-the-art approaches.
-
neural machine translation by jointly learning to align and translate
International Conference on Learning Representations, 2015Co-Authors: Dzmitry Bahdanau, Kyunghyun Cho, Yoshua BengioAbstract:Abstract: Neural machine translation is a recently proposed approach to machine translation. Unlike the traditional statistical machine translation, the neural machine translation aims at building a single neural network that can be jointly tuned to maximize the translation performance. The models proposed recently for neural machine translation often belong to a family of encoder-decoders and consists of an encoder that encodes a source sentence into a fixed-Length Vector from which a decoder generates a translation. In this paper, we conjecture that the use of a fixed-Length Vector is a bottleneck in improving the performance of this basic encoder-decoder architecture, and propose to extend this by allowing a model to automatically (soft-)search for parts of a source sentence that are relevant to predicting a target word, without having to form these parts as a hard segment explicitly. With this new approach, we achieve a translation performance comparable to the existing state-of-the-art phrase-based system on the task of English-to-French translation. Furthermore, qualitative analysis reveals that the (soft-)alignments found by the model agree well with our intuition.
-
learning phrase representations using rnn encoder decoder for statistical machine translation
arXiv: Computation and Language, 2014Co-Authors: Bart Van Merrienboer, Caglar Gulcehre, Fethi Bougares, Holger Schwenk, Kyunghyun Cho, Dzmitry Bahdanau, Yoshua BengioAbstract:In this paper, we propose a novel neural network model called RNN Encoder-Decoder that consists of two recurrent neural networks (RNN). One RNN encodes a sequence of symbols into a fixed-Length Vector representation, and the other decodes the representation into another sequence of symbols. The encoder and decoder of the proposed model are jointly trained to maximize the conditional probability of a target sequence given a source sequence. The performance of a statistical machine translation system is empirically found to improve by using the conditional probabilities of phrase pairs computed by the RNN Encoder-Decoder as an additional feature in the existing log-linear model. Qualitatively, we show that the proposed model learns a semantically and syntactically meaningful representation of linguistic phrases.
Xueqi Cheng - One of the best experts on this subject based on the ideXlab platform.
-
aggregating neural word embeddings for document representation
European Conference on Information Retrieval, 2018Co-Authors: Ruqing Zhang, Jun Xu, Xueqi ChengAbstract:Recent advances in natural language processing (NLP) have shown that semantically meaningful representations of words can be efficiently acquired by distributed models. In such a case, a text document can be viewed as a bag-of-word-embeddings (BoWE), and the remaining question is how to obtain a fixed-Length Vector representation of the document for efficient document process. Beyond those heuristic aggregation methods, recent work has shown that one can leverage the Fisher kernel (FK) framework to generate document representations based on BoWE in a principled way. In this work, words are embedded into a Euclidean space by latent semantic indexing (LSI), and a Gaussian Mixture Model (GMM) is employed as the generative model for nonlinear FK-based aggregation. In this work, we propose an alternate FK-based aggregation method for document representation based on neural word embeddings. As we know, neural embedding models have been proven significantly better performance in word representations than LSI, where semantic relations between neural word embeddings are typically measured by cosine similarity rather than Euclidean distance. Therefore, we introduce a mixture of Von Mises-Fisher distributions (moVMF) as the generative model of neural word embeddings, and derive a new FK-based aggregation method for document representation based on BoWE. We report document classification, clustering and retrieval experiments and demonstrate that our model can produce state-of-the-art performance as compared with existing baseline methods.
Adam Podhorski - One of the best experts on this subject based on the ideXlab platform.
-
fast communication geometric mmse for one sided and two sided Vector linear predictors from the finite Length case to the infinite Length case
Signal Processing, 2011Co-Authors: Jesus Gutierrezgutierrez, Inaki Iglesias, Adam PodhorskiAbstract:In the present paper we derive a formula for the geometric minimum mean square error (GMMSE) for one-sided and two-sided finite-Length Vector linear predictors. This formula is written only in terms of the autocorrelation matrix of the Vector process being predicted. We also obtain a formula for the GMMSE for one-sided and two-sided infinite-Length Vector linear predictors of any wide sense stationary (WSS) Vector process. This GMMSE expression for the infinite-Length case is derived from the GMMSE expression obtained for the finite-Length case.
Ruqing Zhang - One of the best experts on this subject based on the ideXlab platform.
-
aggregating neural word embeddings for document representation
European Conference on Information Retrieval, 2018Co-Authors: Ruqing Zhang, Jun Xu, Xueqi ChengAbstract:Recent advances in natural language processing (NLP) have shown that semantically meaningful representations of words can be efficiently acquired by distributed models. In such a case, a text document can be viewed as a bag-of-word-embeddings (BoWE), and the remaining question is how to obtain a fixed-Length Vector representation of the document for efficient document process. Beyond those heuristic aggregation methods, recent work has shown that one can leverage the Fisher kernel (FK) framework to generate document representations based on BoWE in a principled way. In this work, words are embedded into a Euclidean space by latent semantic indexing (LSI), and a Gaussian Mixture Model (GMM) is employed as the generative model for nonlinear FK-based aggregation. In this work, we propose an alternate FK-based aggregation method for document representation based on neural word embeddings. As we know, neural embedding models have been proven significantly better performance in word representations than LSI, where semantic relations between neural word embeddings are typically measured by cosine similarity rather than Euclidean distance. Therefore, we introduce a mixture of Von Mises-Fisher distributions (moVMF) as the generative model of neural word embeddings, and derive a new FK-based aggregation method for document representation based on BoWE. We report document classification, clustering and retrieval experiments and demonstrate that our model can produce state-of-the-art performance as compared with existing baseline methods.
Lei Ying - One of the best experts on this subject based on the ideXlab platform.
-
Secure Communications Over Wireless Broadcast Networks: Stability and Utility Maximization
IEEE Transactions on Information Forensics and Security, 2011Co-Authors: Yingbin Liang, H.v. Poor, Lei YingAbstract:A wireless broadcast network model with secrecy constraints is investigated, in which a source node broadcasts K confidential message flows to K user nodes, with each message intended to be decoded accurately by one user and to be kept secret from all other users (who are thus considered to be eavesdroppers with regard to all other messages but their own). The source maintains a queue for each message flow if it is not served immediately. The channel from the source to the K users is modeled as a fading broadcast channel, and the channel state information is assumed to be known to the source and the corresponding receivers. Two eavesdropping models are considered. For a collaborative eavesdropping model, in which the eavesdroppers exchange their outputs, the secrecy capacity region is obtained, within which each rate Vector is achieved by using a time-division scheme and a source power control policy over channel states. A throughput optimal queue-Length-based rate scheduling algorithm is further derived that stabilizes all arrival rate Vectors contained in the secrecy capacity region. Moreover, the network utility function is maximized via joint design of rate control, rate scheduling, power control, and secure coding. More precisely, a source controls the message arrival rate according to its message queue, the rate scheduling selects a transmission rate based the queue Length Vector, and the rate Vector is achieved by power control and secure coding. These components work jointly to solve the network utility maximization problem. For a noncollaborative eavesdropping model, in which eavesdroppers do not exchange their outputs, an achievable secrecy rate region is derived based on a time-division scheme, and the queue-Length-based rate scheduling algorithm and the corresponding power control policy are obtained that stabilize all arrival rate Vectors in this region. The network utility maximizing rate control Vector is also obtained.