The Experts below are selected from a list of 360 Experts worldwide ranked by ideXlab platform
Min Zhang - One of the best experts on this subject based on the ideXlab platform.
-
investigating Reading Behavior in fine grained relevance judgment
International ACM SIGIR Conference on Research and Development in Information Retrieval, 2020Co-Authors: Jiaxin Mao, Yiqun Liu, Min ZhangAbstract:A better understanding of users' Reading Behavior helps improve many information retrieval (IR) tasks, such as relevance estimation and document ranking. Existing research has already leveraged eye movement information to investigate user's Reading process during document-level relevance judgments and the findings were adopted to build more effective ranking models. Recently, fine-grained (e.g., passage or sentence level) relevance judgments have been paid much attention to with the requirements in conversational search and QA systems. However, there is still a lack of thorough investigation on user's Reading Behavior during these kinds of interaction processes. To shed light on this research question, we investigate how users allocate their attention to passages of a document during the relevance judgment process. With the eye-tracking data collected in a laboratory study, we show that users pay more attention to the "key" passages which contain key useful information. Users tend to revisit these key passages several times to accumulate and verify the gathered information. With both content and user Behavior features, we find that key passages can be predicted with supervised learning. We believe that this work contributes to better understanding users' Reading Behavior and may provide more explainability for relevance estimation.
-
human Behavior inspired machine Reading comprehension
International ACM SIGIR Conference on Research and Development in Information Retrieval, 2019Co-Authors: Yukun Zheng, Jiaxin Mao, Yiqun Liu, Min ZhangAbstract:Machine Reading Comprehension (MRC) is one of the most challenging tasks in both NLP and IR researches. Recently, a number of deep neural models have been successfully adopted to some simplified MRC task settings, whose performances were close to or even better than human beings. However, these models still have large performance gaps with human beings in more practical settings, such as MS MARCO and DuReader datasets. Although there are many works studying human Reading Behavior, the Behavior patterns in complex Reading comprehension scenarios remain under-investigated. We believe that a better understanding of how human reads and allocates their attention during Reading comprehension processes can help improve the performance of MRC tasks. In this paper, we conduct a lab study to investigate human's Reading Behavior patterns during Reading comprehension tasks, where 32 users are recruited to take 60 distinct tasks. By analyzing the collected eye-tracking data and answers from participants, we propose a two-stage Reading Behavior model, in which the first stage is to search for possible answer candidates and the second stage is to generate the final answer through a comparison and verification process. We also find that human's attention distribution is affected by both question-dependent factors (e.g., answer and soft matching signal with questions) and question-independent factors (e.g., position, IDF and Part-of-Speech tags of words). We extract features derived from the two-stage Reading Behavior model to predict human's attention signals during Reading comprehension, which significantly improves performance in the MRC task. Findings in our work may bring insight into the understanding of human Reading and information seeking processes, and help the machine to better meet users' information needs.
-
teach machine how to read Reading Behavior inspired relevance estimation
International ACM SIGIR Conference on Research and Development in Information Retrieval, 2019Co-Authors: Jiaxin Mao, Yiqun Liu, Chao Wang, Min ZhangAbstract:Retrieval models aim to estimate the relevance of a document to a certain query. Although existing retrieval models have gained much success in both deepening our understanding of information seeking Behavior and constructing practical retrieval systems (e.g. Web search engines), we have to admit that the models work in a rather different manner than how humans make relevance judgments. In this paper, we aim to reexamine the existing models as well as to propose new ones based on the findings in how human read documents during relevance judgment. First, we summarize a number of Reading heuristics from practical user Behavior patterns, which are categorized into implicit and explicit heuristics. By reviewing a variety of existing retrieval models, we find that most of them only satisfy a part of these Reading heuristics. To evaluate the effectiveness of each heuristic, we conduct an ablation study and find that most heuristics have positive impacts on retrieval performance. We further integrate all the effective heuristics into a new retrieval model named Reading Inspired Model (RIM). Specifically, implicit Reading heuristics are incorporated into the model framework and explicit Reading heuristics are modeled as a Markov Decision Process and learned by reinforcement learning. Experimental results on a large-scale public available benchmark dataset and two test sets from NTCIR WWW tasks show that RIM outperforms most existing models, which illustrates the effectiveness of the Reading heuristics. We believe that this work contributes to constructing retrieval models with both higher retrieval performance and better explainability.
Jiaxin Mao - One of the best experts on this subject based on the ideXlab platform.
-
investigating Reading Behavior in fine grained relevance judgment
International ACM SIGIR Conference on Research and Development in Information Retrieval, 2020Co-Authors: Jiaxin Mao, Yiqun Liu, Min ZhangAbstract:A better understanding of users' Reading Behavior helps improve many information retrieval (IR) tasks, such as relevance estimation and document ranking. Existing research has already leveraged eye movement information to investigate user's Reading process during document-level relevance judgments and the findings were adopted to build more effective ranking models. Recently, fine-grained (e.g., passage or sentence level) relevance judgments have been paid much attention to with the requirements in conversational search and QA systems. However, there is still a lack of thorough investigation on user's Reading Behavior during these kinds of interaction processes. To shed light on this research question, we investigate how users allocate their attention to passages of a document during the relevance judgment process. With the eye-tracking data collected in a laboratory study, we show that users pay more attention to the "key" passages which contain key useful information. Users tend to revisit these key passages several times to accumulate and verify the gathered information. With both content and user Behavior features, we find that key passages can be predicted with supervised learning. We believe that this work contributes to better understanding users' Reading Behavior and may provide more explainability for relevance estimation.
-
human Behavior inspired machine Reading comprehension
International ACM SIGIR Conference on Research and Development in Information Retrieval, 2019Co-Authors: Yukun Zheng, Jiaxin Mao, Yiqun Liu, Min ZhangAbstract:Machine Reading Comprehension (MRC) is one of the most challenging tasks in both NLP and IR researches. Recently, a number of deep neural models have been successfully adopted to some simplified MRC task settings, whose performances were close to or even better than human beings. However, these models still have large performance gaps with human beings in more practical settings, such as MS MARCO and DuReader datasets. Although there are many works studying human Reading Behavior, the Behavior patterns in complex Reading comprehension scenarios remain under-investigated. We believe that a better understanding of how human reads and allocates their attention during Reading comprehension processes can help improve the performance of MRC tasks. In this paper, we conduct a lab study to investigate human's Reading Behavior patterns during Reading comprehension tasks, where 32 users are recruited to take 60 distinct tasks. By analyzing the collected eye-tracking data and answers from participants, we propose a two-stage Reading Behavior model, in which the first stage is to search for possible answer candidates and the second stage is to generate the final answer through a comparison and verification process. We also find that human's attention distribution is affected by both question-dependent factors (e.g., answer and soft matching signal with questions) and question-independent factors (e.g., position, IDF and Part-of-Speech tags of words). We extract features derived from the two-stage Reading Behavior model to predict human's attention signals during Reading comprehension, which significantly improves performance in the MRC task. Findings in our work may bring insight into the understanding of human Reading and information seeking processes, and help the machine to better meet users' information needs.
-
teach machine how to read Reading Behavior inspired relevance estimation
International ACM SIGIR Conference on Research and Development in Information Retrieval, 2019Co-Authors: Jiaxin Mao, Yiqun Liu, Chao Wang, Min ZhangAbstract:Retrieval models aim to estimate the relevance of a document to a certain query. Although existing retrieval models have gained much success in both deepening our understanding of information seeking Behavior and constructing practical retrieval systems (e.g. Web search engines), we have to admit that the models work in a rather different manner than how humans make relevance judgments. In this paper, we aim to reexamine the existing models as well as to propose new ones based on the findings in how human read documents during relevance judgment. First, we summarize a number of Reading heuristics from practical user Behavior patterns, which are categorized into implicit and explicit heuristics. By reviewing a variety of existing retrieval models, we find that most of them only satisfy a part of these Reading heuristics. To evaluate the effectiveness of each heuristic, we conduct an ablation study and find that most heuristics have positive impacts on retrieval performance. We further integrate all the effective heuristics into a new retrieval model named Reading Inspired Model (RIM). Specifically, implicit Reading heuristics are incorporated into the model framework and explicit Reading heuristics are modeled as a Markov Decision Process and learned by reinforcement learning. Experimental results on a large-scale public available benchmark dataset and two test sets from NTCIR WWW tasks show that RIM outperforms most existing models, which illustrates the effectiveness of the Reading heuristics. We believe that this work contributes to constructing retrieval models with both higher retrieval performance and better explainability.
Yiqun Liu - One of the best experts on this subject based on the ideXlab platform.
-
investigating Reading Behavior in fine grained relevance judgment
International ACM SIGIR Conference on Research and Development in Information Retrieval, 2020Co-Authors: Jiaxin Mao, Yiqun Liu, Min ZhangAbstract:A better understanding of users' Reading Behavior helps improve many information retrieval (IR) tasks, such as relevance estimation and document ranking. Existing research has already leveraged eye movement information to investigate user's Reading process during document-level relevance judgments and the findings were adopted to build more effective ranking models. Recently, fine-grained (e.g., passage or sentence level) relevance judgments have been paid much attention to with the requirements in conversational search and QA systems. However, there is still a lack of thorough investigation on user's Reading Behavior during these kinds of interaction processes. To shed light on this research question, we investigate how users allocate their attention to passages of a document during the relevance judgment process. With the eye-tracking data collected in a laboratory study, we show that users pay more attention to the "key" passages which contain key useful information. Users tend to revisit these key passages several times to accumulate and verify the gathered information. With both content and user Behavior features, we find that key passages can be predicted with supervised learning. We believe that this work contributes to better understanding users' Reading Behavior and may provide more explainability for relevance estimation.
-
human Behavior inspired machine Reading comprehension
International ACM SIGIR Conference on Research and Development in Information Retrieval, 2019Co-Authors: Yukun Zheng, Jiaxin Mao, Yiqun Liu, Min ZhangAbstract:Machine Reading Comprehension (MRC) is one of the most challenging tasks in both NLP and IR researches. Recently, a number of deep neural models have been successfully adopted to some simplified MRC task settings, whose performances were close to or even better than human beings. However, these models still have large performance gaps with human beings in more practical settings, such as MS MARCO and DuReader datasets. Although there are many works studying human Reading Behavior, the Behavior patterns in complex Reading comprehension scenarios remain under-investigated. We believe that a better understanding of how human reads and allocates their attention during Reading comprehension processes can help improve the performance of MRC tasks. In this paper, we conduct a lab study to investigate human's Reading Behavior patterns during Reading comprehension tasks, where 32 users are recruited to take 60 distinct tasks. By analyzing the collected eye-tracking data and answers from participants, we propose a two-stage Reading Behavior model, in which the first stage is to search for possible answer candidates and the second stage is to generate the final answer through a comparison and verification process. We also find that human's attention distribution is affected by both question-dependent factors (e.g., answer and soft matching signal with questions) and question-independent factors (e.g., position, IDF and Part-of-Speech tags of words). We extract features derived from the two-stage Reading Behavior model to predict human's attention signals during Reading comprehension, which significantly improves performance in the MRC task. Findings in our work may bring insight into the understanding of human Reading and information seeking processes, and help the machine to better meet users' information needs.
-
teach machine how to read Reading Behavior inspired relevance estimation
International ACM SIGIR Conference on Research and Development in Information Retrieval, 2019Co-Authors: Jiaxin Mao, Yiqun Liu, Chao Wang, Min ZhangAbstract:Retrieval models aim to estimate the relevance of a document to a certain query. Although existing retrieval models have gained much success in both deepening our understanding of information seeking Behavior and constructing practical retrieval systems (e.g. Web search engines), we have to admit that the models work in a rather different manner than how humans make relevance judgments. In this paper, we aim to reexamine the existing models as well as to propose new ones based on the findings in how human read documents during relevance judgment. First, we summarize a number of Reading heuristics from practical user Behavior patterns, which are categorized into implicit and explicit heuristics. By reviewing a variety of existing retrieval models, we find that most of them only satisfy a part of these Reading heuristics. To evaluate the effectiveness of each heuristic, we conduct an ablation study and find that most heuristics have positive impacts on retrieval performance. We further integrate all the effective heuristics into a new retrieval model named Reading Inspired Model (RIM). Specifically, implicit Reading heuristics are incorporated into the model framework and explicit Reading heuristics are modeled as a Markov Decision Process and learned by reinforcement learning. Experimental results on a large-scale public available benchmark dataset and two test sets from NTCIR WWW tasks show that RIM outperforms most existing models, which illustrates the effectiveness of the Reading heuristics. We believe that this work contributes to constructing retrieval models with both higher retrieval performance and better explainability.
Weidong Huang - One of the best experts on this subject based on the ideXlab platform.
-
establishing aesthetics based on human graph Reading Behavior two eye tracking studies
Ubiquitous Computing, 2013Co-Authors: Weidong HuangAbstract:A great deal of real-world data have graph structures, and such structures are often visualized into node-link diagrams for a better understanding of the data. Aesthetic criteria have been used as quality measures to evaluate the effectiveness of graph visualizations in conveying the embedded information to end users. However, commonly applied aesthetics are originally proposed based on common senses and personal intuitions; thus, their relevance to effectiveness is not guaranteed. It has been agreed that aesthetics should be established based on empirical evidence and derived from theories of how people read graphs. As the first step to this end, we have conducted two eye tracking studies in an attempt to understand the underlying mechanism of edge crossings, the most discussed aesthetic, affecting human graph Reading performance. These studies lead to the findings of an important aesthetic of crossing angles and a graph Reading Behavior of geodesic path tendency. We demonstrate that eye tracking is an effective method for gaining insights into how people read graphs and that how aesthetics can be established based on human graph Reading Behavior.
-
a graph Reading Behavior geodesic path tendency
IEEE Pacific Visualization Symposium, 2009Co-Authors: Weidong Huang, Peter Eades, Seokhee HongAbstract:The end result of graph visualization is that people read the graph and understand the data. To make this effective, it is essential to construct visualizations based on how people read graphs. Despite the popularity and importance of graph usage in a variety of application domains, little is known about how people read graphs. The lack of this knowledge has severely limited the effectiveness of graph visualizations. In attempts to understand how people read graphs, we previously observed that people have geodesic-path tendency based on subjective eye tracking data. This paper presents two controlled experiments. One is to approve the existence of the geodesic-path tendency. The other is to examine the effects of this tendency on people in Reading graphs. The results show that in performing path search tasks, when eyes encounter a node that has more than one link, links that go toward the target node are more likely to be searched first. The results also indicate that when graphs are drawn with branch links on the path leading away from the target node, graph Reading performance can be significantly improved.
Reinhold Kliegl - One of the best experts on this subject based on the ideXlab platform.
-
perceptual span depends on font size during the Reading of chinese sentences
Journal of Experimental Psychology: Learning Memory and Cognition, 2015Co-Authors: Ming Yan, Wei Zhou, Hua Shu, Reinhold KlieglAbstract:The present study explored the perceptual span (i.e., the physical extent of an area from which useful visual information is extracted during a single fixation) during the Reading of Chinese sentences in 2 experiments. In Experiment 1, we tested whether the rightward span can go beyond 3 characters when visually similar masks were used. Results showed that Chinese readers needed at least 4 characters to the right of fixation to maintain a normal Reading Behavior when visually similar masks were used and when characters were displayed in small fonts, indicating that the span is dynamically influenced by masking materials. In Experiments 2 and 3, we asked whether the perceptual span varies as a function of font size in spaced (German) and unspaced (Chinese) scripts. Results clearly suggest perceptual span depends on font size in Chinese, but we failed to find such evidence for German. We propose that the perceptual span in Chinese is flexible; it is strongly constrained by its language-specific properties such as high information density and lack of word spacing. Implications for saccade-target selection during the Reading of Chinese sentences are discussed.
