The Experts below are selected from a list of 5976 Experts worldwide ranked by ideXlab platform

Bowen Zhou - One of the best experts on this subject based on the ideXlab platform.

  • conversational query rewriting with self supervised learning
    International Conference on Acoustics Speech and Signal Processing, 2021
    Co-Authors: Hang Liu, Meng Chen, Bowen Zhou
    Abstract:

    Context Modeling plays a critical role in building multi-turn Dialogue systems. Conversational Query Rewriting (CQR) aims to simplify the multi-turn Dialogue Modeling into a single-turn problem by explicitly rewriting the conversational query into a self-contained utterance. However, existing approaches rely on massive supervised training data, which is labor-intensive to annotate. And the detection of the omitted important information from context can be further improved. Besides, intent consistency constraint between contextual query and rewritten query is also ignored. To tackle these issues, we first propose to construct a large-scale CQR dataset automatically via self-supervised learning, which does not need human annotation. Then we introduce a novel CQR model Teresa based on Transformer, which is enhanced by self-attentive keywords detection and intent consistency constraint. Finally, we conduct extensive experiments on two public datasets. Experimental results demonstrate that our proposed model outperforms existing CQR baselines significantly, and also prove the effectiveness of self-supervised learning on improving the CQR performance.

Zhao Hai - One of the best experts on this subject based on the ideXlab platform.

  • Advances in Multi-turn Dialogue Comprehension: A Survey
    2021
    Co-Authors: Zhang Zhuosheng, Zhao Hai
    Abstract:

    Training machines to understand natural language and interact with humans is an elusive and essential task of artificial intelligence. A diversity of Dialogue systems has been designed with the rapid development of deep learning techniques, especially the recent pre-trained language models (PrLMs). Among these studies, the fundamental yet challenging type of task is Dialogue comprehension whose role is to teach the machines to read and comprehend the Dialogue context before responding. In this paper, we review the previous methods from the technical perspective of Dialogue Modeling for the Dialogue comprehension task. We summarize the characteristics and challenges of Dialogue comprehension in contrast to plain-text reading comprehension. Then, we discuss three typical patterns of Dialogue Modeling. In addition, we categorize Dialogue-related pre-training techniques which are employed to enhance PrLMs in Dialogue scenarios. Finally, we highlight the technical advances in recent years and point out the lessons from the empirical analysis and the prospects towards a new frontier of researches

  • Advances in Multi-turn Dialogue Comprehension: A Survey
    2021
    Co-Authors: Zhang Zhuosheng, Zhao Hai
    Abstract:

    Training machines to understand natural language and interact with humans is an elusive and essential task in the field of artificial intelligence. In recent years, a diversity of Dialogue systems has been designed with the rapid development of deep learning researches, especially the recent pre-trained language models. Among these studies, the fundamental yet challenging part is Dialogue comprehension whose role is to teach the machines to read and comprehend the Dialogue context before responding. In this paper, we review the previous methods from the perspective of Dialogue Modeling. We summarize the characteristics and challenges of Dialogue comprehension in contrast to plain-text reading comprehension. Then, we discuss three typical patterns of Dialogue Modeling that are widely-used in Dialogue comprehension tasks such as response selection and conversation question-answering, as well as Dialogue-related language Modeling techniques to enhance PrLMs in Dialogue scenarios. Finally, we highlight the technical advances in recent years and point out the lessons we can learn from the empirical analysis and the prospects towards a new frontier of researches

  • Advances in Multi-turn Dialogue Comprehension: A Survey
    2021
    Co-Authors: Zhang Zhuosheng, Zhao Hai
    Abstract:

    Training machines to understand natural language and interact with humans is an elusive and essential task of artificial intelligence. A diversity of Dialogue systems has been designed with the rapid development of deep learning techniques, especially the recent pre-trained language models (PrLMs). Among these studies, the fundamental yet challenging type of task is Dialogue comprehension whose role is to teach the machines to read and comprehend the Dialogue context before responding. In this paper, we review the previous methods from the technical perspective of Dialogue Modeling for the Dialogue comprehension task. We summarize the characteristics and challenges of Dialogue comprehension in contrast to plain-text reading comprehension. Then, we discuss three typical patterns of Dialogue Modeling. In addition, we categorize Dialogue-related pre-training techniques which are employed to enhance PrLMs in Dialogue scenarios. Finally, we highlight the technical advances in recent years and point out the lessons from the empirical analysis and the prospects towards a new frontier of researches.Comment: Upload as new by mistake. Please see the updated version in the existing one: arXiv:2103.0312

  • Topic-Aware Multi-turn Dialogue Modeling
    2020
    Co-Authors: Zhao Hai, Zhang Zhuosheng
    Abstract:

    In the retrieval-based multi-turn Dialogue Modeling, it remains a challenge to select the most appropriate response according to extracting salient features in context utterances. As a conversation goes on, topic shift at discourse-level naturally happens through the continuous multi-turn Dialogue context. However, all known retrieval-based systems are satisfied with exploiting local topic words for context utterance representation but fail to capture such essential global topic-aware clues at discourse-level. Instead of taking topic-agnostic n-gram utterance as processing unit for matching purpose in existing systems, this paper presents a novel topic-aware solution for multi-turn Dialogue Modeling, which segments and extracts topic-aware utterances in an unsupervised way, so that the resulted model is capable of capturing salient topic shift at discourse-level in need and thus effectively track topic flow during multi-turn conversation. Our topic-aware Modeling is implemented by a newly proposed unsupervised topic-aware segmentation algorithm and Topic-Aware Dual-attention Matching (TADAM) Network, which matches each topic segment with the response in a dual cross-attention way. Experimental results on three public datasets show TADAM can outperform the state-of-the-art method, especially by 3.3% on E-commerce dataset that has an obvious topic shift

  • Filling the Gap of Utterance-aware and Speaker-aware Representation for Multi-turn Dialogue
    2020
    Co-Authors: Liu Longxiang, Zhao Hai, Zhang Zhuosheng, Xi Zhou, Zhou Xiang
    Abstract:

    A multi-turn Dialogue is composed of multiple utterances from two or more different speaker roles. Thus utterance- and speaker-aware clues are supposed to be well captured in models. However, in the existing retrieval-based multi-turn Dialogue Modeling, the pre-trained language models (PrLMs) as encoder represent the Dialogues coarsely by taking the pairwise Dialogue history and candidate response as a whole, the hierarchical information on either utterance interrelation or speaker roles coupled in such representations is not well addressed. In this work, we propose a novel model to fill such a gap by Modeling the effective utterance-aware and speaker-aware representations entailed in a Dialogue history. In detail, we decouple the contextualized word representations by masking mechanisms in Transformer-based PrLM, making each word only focus on the words in current utterance, other utterances, two speaker roles (i.e., utterances of sender and utterances of receiver), respectively. Experimental results show that our method boosts the strong ELECTRA baseline substantially in four public benchmark datasets, and achieves various new state-of-the-art performance over previous methods. A series of ablation studies are conducted to demonstrate the effectiveness of our method.Comment: 9 pages, 2 figures, 9 table

Zhang Zhuosheng - One of the best experts on this subject based on the ideXlab platform.

  • Advances in Multi-turn Dialogue Comprehension: A Survey
    2021
    Co-Authors: Zhang Zhuosheng, Zhao Hai
    Abstract:

    Training machines to understand natural language and interact with humans is an elusive and essential task of artificial intelligence. A diversity of Dialogue systems has been designed with the rapid development of deep learning techniques, especially the recent pre-trained language models (PrLMs). Among these studies, the fundamental yet challenging type of task is Dialogue comprehension whose role is to teach the machines to read and comprehend the Dialogue context before responding. In this paper, we review the previous methods from the technical perspective of Dialogue Modeling for the Dialogue comprehension task. We summarize the characteristics and challenges of Dialogue comprehension in contrast to plain-text reading comprehension. Then, we discuss three typical patterns of Dialogue Modeling. In addition, we categorize Dialogue-related pre-training techniques which are employed to enhance PrLMs in Dialogue scenarios. Finally, we highlight the technical advances in recent years and point out the lessons from the empirical analysis and the prospects towards a new frontier of researches

  • Advances in Multi-turn Dialogue Comprehension: A Survey
    2021
    Co-Authors: Zhang Zhuosheng, Zhao Hai
    Abstract:

    Training machines to understand natural language and interact with humans is an elusive and essential task in the field of artificial intelligence. In recent years, a diversity of Dialogue systems has been designed with the rapid development of deep learning researches, especially the recent pre-trained language models. Among these studies, the fundamental yet challenging part is Dialogue comprehension whose role is to teach the machines to read and comprehend the Dialogue context before responding. In this paper, we review the previous methods from the perspective of Dialogue Modeling. We summarize the characteristics and challenges of Dialogue comprehension in contrast to plain-text reading comprehension. Then, we discuss three typical patterns of Dialogue Modeling that are widely-used in Dialogue comprehension tasks such as response selection and conversation question-answering, as well as Dialogue-related language Modeling techniques to enhance PrLMs in Dialogue scenarios. Finally, we highlight the technical advances in recent years and point out the lessons we can learn from the empirical analysis and the prospects towards a new frontier of researches

  • Advances in Multi-turn Dialogue Comprehension: A Survey
    2021
    Co-Authors: Zhang Zhuosheng, Zhao Hai
    Abstract:

    Training machines to understand natural language and interact with humans is an elusive and essential task of artificial intelligence. A diversity of Dialogue systems has been designed with the rapid development of deep learning techniques, especially the recent pre-trained language models (PrLMs). Among these studies, the fundamental yet challenging type of task is Dialogue comprehension whose role is to teach the machines to read and comprehend the Dialogue context before responding. In this paper, we review the previous methods from the technical perspective of Dialogue Modeling for the Dialogue comprehension task. We summarize the characteristics and challenges of Dialogue comprehension in contrast to plain-text reading comprehension. Then, we discuss three typical patterns of Dialogue Modeling. In addition, we categorize Dialogue-related pre-training techniques which are employed to enhance PrLMs in Dialogue scenarios. Finally, we highlight the technical advances in recent years and point out the lessons from the empirical analysis and the prospects towards a new frontier of researches.Comment: Upload as new by mistake. Please see the updated version in the existing one: arXiv:2103.0312

  • Topic-Aware Multi-turn Dialogue Modeling
    2020
    Co-Authors: Zhao Hai, Zhang Zhuosheng
    Abstract:

    In the retrieval-based multi-turn Dialogue Modeling, it remains a challenge to select the most appropriate response according to extracting salient features in context utterances. As a conversation goes on, topic shift at discourse-level naturally happens through the continuous multi-turn Dialogue context. However, all known retrieval-based systems are satisfied with exploiting local topic words for context utterance representation but fail to capture such essential global topic-aware clues at discourse-level. Instead of taking topic-agnostic n-gram utterance as processing unit for matching purpose in existing systems, this paper presents a novel topic-aware solution for multi-turn Dialogue Modeling, which segments and extracts topic-aware utterances in an unsupervised way, so that the resulted model is capable of capturing salient topic shift at discourse-level in need and thus effectively track topic flow during multi-turn conversation. Our topic-aware Modeling is implemented by a newly proposed unsupervised topic-aware segmentation algorithm and Topic-Aware Dual-attention Matching (TADAM) Network, which matches each topic segment with the response in a dual cross-attention way. Experimental results on three public datasets show TADAM can outperform the state-of-the-art method, especially by 3.3% on E-commerce dataset that has an obvious topic shift

  • Filling the Gap of Utterance-aware and Speaker-aware Representation for Multi-turn Dialogue
    2020
    Co-Authors: Liu Longxiang, Zhao Hai, Zhang Zhuosheng, Xi Zhou, Zhou Xiang
    Abstract:

    A multi-turn Dialogue is composed of multiple utterances from two or more different speaker roles. Thus utterance- and speaker-aware clues are supposed to be well captured in models. However, in the existing retrieval-based multi-turn Dialogue Modeling, the pre-trained language models (PrLMs) as encoder represent the Dialogues coarsely by taking the pairwise Dialogue history and candidate response as a whole, the hierarchical information on either utterance interrelation or speaker roles coupled in such representations is not well addressed. In this work, we propose a novel model to fill such a gap by Modeling the effective utterance-aware and speaker-aware representations entailed in a Dialogue history. In detail, we decouple the contextualized word representations by masking mechanisms in Transformer-based PrLM, making each word only focus on the words in current utterance, other utterances, two speaker roles (i.e., utterances of sender and utterances of receiver), respectively. Experimental results show that our method boosts the strong ELECTRA baseline substantially in four public benchmark datasets, and achieves various new state-of-the-art performance over previous methods. A series of ablation studies are conducted to demonstrate the effectiveness of our method.Comment: 9 pages, 2 figures, 9 table

Hang Liu - One of the best experts on this subject based on the ideXlab platform.

  • conversational query rewriting with self supervised learning
    International Conference on Acoustics Speech and Signal Processing, 2021
    Co-Authors: Hang Liu, Meng Chen, Bowen Zhou
    Abstract:

    Context Modeling plays a critical role in building multi-turn Dialogue systems. Conversational Query Rewriting (CQR) aims to simplify the multi-turn Dialogue Modeling into a single-turn problem by explicitly rewriting the conversational query into a self-contained utterance. However, existing approaches rely on massive supervised training data, which is labor-intensive to annotate. And the detection of the omitted important information from context can be further improved. Besides, intent consistency constraint between contextual query and rewritten query is also ignored. To tackle these issues, we first propose to construct a large-scale CQR dataset automatically via self-supervised learning, which does not need human annotation. Then we introduce a novel CQR model Teresa based on Transformer, which is enhanced by self-attentive keywords detection and intent consistency constraint. Finally, we conduct extensive experiments on two public datasets. Experimental results demonstrate that our proposed model outperforms existing CQR baselines significantly, and also prove the effectiveness of self-supervised learning on improving the CQR performance.

Meng Chen - One of the best experts on this subject based on the ideXlab platform.

  • conversational query rewriting with self supervised learning
    International Conference on Acoustics Speech and Signal Processing, 2021
    Co-Authors: Hang Liu, Meng Chen, Bowen Zhou
    Abstract:

    Context Modeling plays a critical role in building multi-turn Dialogue systems. Conversational Query Rewriting (CQR) aims to simplify the multi-turn Dialogue Modeling into a single-turn problem by explicitly rewriting the conversational query into a self-contained utterance. However, existing approaches rely on massive supervised training data, which is labor-intensive to annotate. And the detection of the omitted important information from context can be further improved. Besides, intent consistency constraint between contextual query and rewritten query is also ignored. To tackle these issues, we first propose to construct a large-scale CQR dataset automatically via self-supervised learning, which does not need human annotation. Then we introduce a novel CQR model Teresa based on Transformer, which is enhanced by self-attentive keywords detection and intent consistency constraint. Finally, we conduct extensive experiments on two public datasets. Experimental results demonstrate that our proposed model outperforms existing CQR baselines significantly, and also prove the effectiveness of self-supervised learning on improving the CQR performance.