The Experts below are selected from a list of 360 Experts worldwide ranked by ideXlab platform
Amr Magdy - One of the best experts on this subject based on the ideXlab platform.
-
Microblogs data management: a survey
The VLDB Journal, 2019Co-Authors: Amr Magdy, Laila Abdelhafeez, Yunfan Kang, Eric Ong, Mohamed F MokbelAbstract:Microblogs data is the microlength user-generated data that is posted on the web, e.g., tweets, online reviews, comments on news and social media. It has gained considerable attention in recent years due to its widespread popularity, rich content, and value in several societal applications. Nowadays, Microblogs applications span a wide spectrum of interests including targeted advertising, market reports, news delivery, political campaigns, rescue services, and public health. Consequently, major research efforts have been spent to manage, analyze, and visualize Microblogs to support different applications. This paper gives a comprehensive review of major research and system work in Microblogs data management. The paper reviews core components that enable large-scale querying and indexing for Microblogs data. A dedicated part gives particular focus for discussing system-level issues and on-going effort on supporting Microblogs through the rising wave of big data systems. In addition, we review the major research topics that exploit these core data management components to provide innovative and effective analysis and visualization for Microblogs, such as event detection, recommendations, automatic geotagging, and user queries. Throughout the different parts, we highlight the challenges, innovations, and future opportunities in Microblogs data research.
-
geotrend spatial trending queries on real time Microblogs
Advances in Geographic Information Systems, 2016Co-Authors: Amr Magdy, Mohamed F Mokbel, Ahmed M Aly, Sameh Elnikety, Suman Nath, Walid G ArefAbstract:This paper presents GeoTrend; a system for scalable support of spatial trend discovery on recent Microblogs, e.g., tweets and online reviews, that come in real time. GeoTrend is distinguished from existing techniques in three aspects: (1) It discovers trends in arbitrary spatial regions, e.g., city blocks. (2) It supports trending measures that effectively capture trending items under a variety of definitions that suit different applications. (3) It promotes recent Microblogs as first-class citizens and optimizes its system components to digest a continuous flow of fast data in main-memory while removing old data efficiently. GeoTrend queries are top-k queries that discover the most trending k keywords that are posted within an arbitrary spatial region and during the last T time units. To support its queries efficiently, GeoTrend employs an in-memory spatial index that is able to efficiently digest incoming data and expire data that is beyond the last T time units. The index also materializes top-k keywords in different spatial regions so that incoming queries can be processed with low latency. In case of peak times, a main-memory optimization technique is employed to shed less important data, so that the system still sustains high query accuracy with limited memory resources. Experimental results based on real Twitter feed and Bing Mobile spatial search queries show the scalability of GeoTrend to support arrival rates of up to 50,000 microblog/second, average query latency of 3 milli-seconds, and at least 90+% query accuracy even under limited memory resources.
-
On main-memory flushing in Microblogs data management systems
2016 IEEE 32nd International Conference on Data Engineering (ICDE), 2016Co-Authors: Amr Magdy, Rami Alghamdi, Mohamed F MokbelAbstract:Searching Microblogs, e.g., tweets and comments, is practically supported through main-memory indexing for scalable data digestion and efficient query evaluation. With continuity and excessive numbers of Microblogs, it is infeasible to keep data in main-memory for long periods. Thus, once allocated memory budget is filled, a portion of data is flushed from memory to disk to continuously accommodate newly incoming data. Existing techniques come with either low memory hit ratio due to flushing items regardless of their relevance to incoming queries or significant overhead of tracking individual data items, which limit scalability of Microblogs systems in either cases. In this paper, we propose kFlushing policy that exploits popularity of top-k queries in Microblogs to smartly select a subset of Microblogs to flush. kFlushing is mainly designed to increase memory hit ratio. To this end, it identifies and flushes in-memory data that does not contribute to incoming queries. The freed memory space is utilized to accumulate more useful data that is used to answer more queries from memory contents. When all memory is utilized for useful data, kFlushing flushes data that is less likely to degrade memory hit ratio. In addition, kFlushing comes with a little overhead that keeps high system scalability in terms of high digestion rates of incoming fast data. Extensive experimental evaluation shows the effectiveness and scalability of kFlushing to improve main-memory hit by 26–330% while coping up with fast microblog streams of up to 100K microblog/second.
-
garnet a holistic system approach for trending queries in Microblogs
International Conference on Data Engineering, 2016Co-Authors: Christopher Jonathan, Amr Magdy, Mohamed F Mokbel, Albert JonathanAbstract:The recent wide popularity of Microblogs (e.g., tweets, online comments) has empowered various important applications, including, news delivery, event detection, market analysis, and target advertising. A core module in all these applications is a frequent/trending query processor that aims to find out those topics that are highly frequent or trending in the social media through posted Microblogs. Unfortunately current attempts for such core module suffer from several drawbacks. Most importantly, their narrow scope, as they focus only on solving trending queries for a very special case of localized and very recent Microblogs. This paper presents GARNET; a holistic system equipped with one-stop efficient and scalable solution for supporting a generic form of context-aware frequent and trending queries on Microblogs. GARNET supports both frequent and trending queries, any arbitrary time interval either current, recent, or past, of fixed granularity, and having a set of arbitrary filters over contextual attributes. From a system point of view, GARNET is very appealing and industry-friendly, as one needs to realize it once in the system. Then, a myriad of various forms of trending and frequent queries are immediately supported. Experimental evidence based on a real system prototype of GARNET and billions of real Twitter data show the scalability and efficiency of GARNET for various query types.
-
taghreed a system for querying analyzing and visualizing geotagged Microblogs
Advances in Geographic Information Systems, 2014Co-Authors: Amr Magdy, Louai Alarabi, Saif Alharthi, Mashaal Musleh, Thanaa M Ghanem, Sohaib Ghani, Mohamed F MokbelAbstract:This paper presents Taghreed; a full-fledged system for efficient and scalable querying, analyzing, and visualizing geotagged Microblogs, e.g., tweets. Taghreed supports arbitrary queries on a large number (Billions) of Microblogs that go up to several months in the past. Taghreed consists of four main components: (f) Indexer, (2) query engine, (3) recovery manager, and (4) visualizer. Taghreed indexer efficiently digests incoming Microblogs with high arrival rates in light memory-resident indexes. When the memory becomes full, a flushing policy manager transfers the memory contents to disk indexes which are managing Billions of Microblogs for several months. On memory failure, the recovery manager restores the system status from replicated copies for the main-memory content. Taghreed query engine consists of two modules: a query optimizer and a query processor. The query optimizer generates an optimal query plan to be executed by the query processor through efficient retrieval techniques to provide low query response, i.e., order of milli-seconds. Taghreed visualizer allows end users to issue a wide variety of spatio-temporal queries. Then, it graphically presents the answers and allows interactive exploration through them. Taghreed is the first system that addresses all these challenges collectively for Microblogs data. In the paper, each system component is described in detail.
Mohamed F Mokbel - One of the best experts on this subject based on the ideXlab platform.
-
Microblogs data management: a survey
The VLDB Journal, 2019Co-Authors: Amr Magdy, Laila Abdelhafeez, Yunfan Kang, Eric Ong, Mohamed F MokbelAbstract:Microblogs data is the microlength user-generated data that is posted on the web, e.g., tweets, online reviews, comments on news and social media. It has gained considerable attention in recent years due to its widespread popularity, rich content, and value in several societal applications. Nowadays, Microblogs applications span a wide spectrum of interests including targeted advertising, market reports, news delivery, political campaigns, rescue services, and public health. Consequently, major research efforts have been spent to manage, analyze, and visualize Microblogs to support different applications. This paper gives a comprehensive review of major research and system work in Microblogs data management. The paper reviews core components that enable large-scale querying and indexing for Microblogs data. A dedicated part gives particular focus for discussing system-level issues and on-going effort on supporting Microblogs through the rising wave of big data systems. In addition, we review the major research topics that exploit these core data management components to provide innovative and effective analysis and visualization for Microblogs, such as event detection, recommendations, automatic geotagging, and user queries. Throughout the different parts, we highlight the challenges, innovations, and future opportunities in Microblogs data research.
-
geotrend spatial trending queries on real time Microblogs
Advances in Geographic Information Systems, 2016Co-Authors: Amr Magdy, Mohamed F Mokbel, Ahmed M Aly, Sameh Elnikety, Suman Nath, Walid G ArefAbstract:This paper presents GeoTrend; a system for scalable support of spatial trend discovery on recent Microblogs, e.g., tweets and online reviews, that come in real time. GeoTrend is distinguished from existing techniques in three aspects: (1) It discovers trends in arbitrary spatial regions, e.g., city blocks. (2) It supports trending measures that effectively capture trending items under a variety of definitions that suit different applications. (3) It promotes recent Microblogs as first-class citizens and optimizes its system components to digest a continuous flow of fast data in main-memory while removing old data efficiently. GeoTrend queries are top-k queries that discover the most trending k keywords that are posted within an arbitrary spatial region and during the last T time units. To support its queries efficiently, GeoTrend employs an in-memory spatial index that is able to efficiently digest incoming data and expire data that is beyond the last T time units. The index also materializes top-k keywords in different spatial regions so that incoming queries can be processed with low latency. In case of peak times, a main-memory optimization technique is employed to shed less important data, so that the system still sustains high query accuracy with limited memory resources. Experimental results based on real Twitter feed and Bing Mobile spatial search queries show the scalability of GeoTrend to support arrival rates of up to 50,000 microblog/second, average query latency of 3 milli-seconds, and at least 90+% query accuracy even under limited memory resources.
-
On main-memory flushing in Microblogs data management systems
2016 IEEE 32nd International Conference on Data Engineering (ICDE), 2016Co-Authors: Amr Magdy, Rami Alghamdi, Mohamed F MokbelAbstract:Searching Microblogs, e.g., tweets and comments, is practically supported through main-memory indexing for scalable data digestion and efficient query evaluation. With continuity and excessive numbers of Microblogs, it is infeasible to keep data in main-memory for long periods. Thus, once allocated memory budget is filled, a portion of data is flushed from memory to disk to continuously accommodate newly incoming data. Existing techniques come with either low memory hit ratio due to flushing items regardless of their relevance to incoming queries or significant overhead of tracking individual data items, which limit scalability of Microblogs systems in either cases. In this paper, we propose kFlushing policy that exploits popularity of top-k queries in Microblogs to smartly select a subset of Microblogs to flush. kFlushing is mainly designed to increase memory hit ratio. To this end, it identifies and flushes in-memory data that does not contribute to incoming queries. The freed memory space is utilized to accumulate more useful data that is used to answer more queries from memory contents. When all memory is utilized for useful data, kFlushing flushes data that is less likely to degrade memory hit ratio. In addition, kFlushing comes with a little overhead that keeps high system scalability in terms of high digestion rates of incoming fast data. Extensive experimental evaluation shows the effectiveness and scalability of kFlushing to improve main-memory hit by 26–330% while coping up with fast microblog streams of up to 100K microblog/second.
-
garnet a holistic system approach for trending queries in Microblogs
International Conference on Data Engineering, 2016Co-Authors: Christopher Jonathan, Amr Magdy, Mohamed F Mokbel, Albert JonathanAbstract:The recent wide popularity of Microblogs (e.g., tweets, online comments) has empowered various important applications, including, news delivery, event detection, market analysis, and target advertising. A core module in all these applications is a frequent/trending query processor that aims to find out those topics that are highly frequent or trending in the social media through posted Microblogs. Unfortunately current attempts for such core module suffer from several drawbacks. Most importantly, their narrow scope, as they focus only on solving trending queries for a very special case of localized and very recent Microblogs. This paper presents GARNET; a holistic system equipped with one-stop efficient and scalable solution for supporting a generic form of context-aware frequent and trending queries on Microblogs. GARNET supports both frequent and trending queries, any arbitrary time interval either current, recent, or past, of fixed granularity, and having a set of arbitrary filters over contextual attributes. From a system point of view, GARNET is very appealing and industry-friendly, as one needs to realize it once in the system. Then, a myriad of various forms of trending and frequent queries are immediately supported. Experimental evidence based on a real system prototype of GARNET and billions of real Twitter data show the scalability and efficiency of GARNET for various query types.
-
taghreed a system for querying analyzing and visualizing geotagged Microblogs
Advances in Geographic Information Systems, 2014Co-Authors: Amr Magdy, Louai Alarabi, Saif Alharthi, Mashaal Musleh, Thanaa M Ghanem, Sohaib Ghani, Mohamed F MokbelAbstract:This paper presents Taghreed; a full-fledged system for efficient and scalable querying, analyzing, and visualizing geotagged Microblogs, e.g., tweets. Taghreed supports arbitrary queries on a large number (Billions) of Microblogs that go up to several months in the past. Taghreed consists of four main components: (f) Indexer, (2) query engine, (3) recovery manager, and (4) visualizer. Taghreed indexer efficiently digests incoming Microblogs with high arrival rates in light memory-resident indexes. When the memory becomes full, a flushing policy manager transfers the memory contents to disk indexes which are managing Billions of Microblogs for several months. On memory failure, the recovery manager restores the system status from replicated copies for the main-memory content. Taghreed query engine consists of two modules: a query optimizer and a query processor. The query optimizer generates an optimal query plan to be executed by the query processor through efficient retrieval techniques to provide low query response, i.e., order of milli-seconds. Taghreed visualizer allows end users to issue a wide variety of spatio-temporal queries. Then, it graphically presents the answers and allows interactive exploration through them. Taghreed is the first system that addresses all these challenges collectively for Microblogs data. In the paper, each system component is described in detail.
Jianguo Xiao - One of the best experts on this subject based on the ideXlab platform.
-
CMiner: Opinion Extraction and Summarization for Chinese Microblogs
IEEE Transactions on Knowledge and Data Engineering, 2016Co-Authors: Xinjie Zhou, Xiaojun Wan, Jianguo XiaoAbstract:Sentiment analysis of microblog texts has drawn lots of attention in both the academic and industrial fields. However, most of the current work only focuses on polarity classification. In this paper, we present an opinion mining system for Chinese Microblogs called CMiner. Instead of polarity classification, CMiner focuses on more complicated opinion mining tasks - opinion target extraction and opinion summarization. Novel algorithms are developed for the two tasks and integrated into the end-to-end system. CMiner can help to effectively understand the users' opinion towards different opinion targets in a microblog topic. Specially, we develop an unsupervised label propagation algorithm for opinion target extraction. The opinion targets of all messages in a topic are collectively extracted based on the assumption that similar messages may focus on similar opinion targets. In addition, we build an aspect-based opinion summarization framework for microblog topics. After getting the opinion targets of all the microblog messages in a topic, we cluster the opinion targets into several groups and extract representative targets and summaries for each group. A co-ranking algorithm is proposed to rank both the opinion targets and microblog sentences simultaneously. Experimental results on a benchmark dataset show the effectiveness of our system and the algorithms.
-
collective opinion target extraction in chinese Microblogs
Empirical Methods in Natural Language Processing, 2013Co-Authors: Xinjie Zhou, Xiaojun Wan, Jianguo XiaoAbstract:Microblog messages pose severe challenges for current sentiment analysis techniques due to some inherent characteristics such as the length limit and informal writing style. In this paper, we study the problem of extracting opinion targets of Chinese microblog messages. Such fine-grained word-level task has not been well investigated in Microblogs yet. We propose an unsupervised label propagation algorithm to address the problem. The opinion targets of all messages in a topic are collectively extracted based on the assumption that similar messages may focus on similar opinion targets. Topics in Microblogs are identified by hashtags or using clustering algorithms. Experimental results on Chinese Microblogs show the effectiveness of our framework and algorithms.
Suman Nath - One of the best experts on this subject based on the ideXlab platform.
-
geotrend spatial trending queries on real time Microblogs
Advances in Geographic Information Systems, 2016Co-Authors: Amr Magdy, Mohamed F Mokbel, Ahmed M Aly, Sameh Elnikety, Suman Nath, Walid G ArefAbstract:This paper presents GeoTrend; a system for scalable support of spatial trend discovery on recent Microblogs, e.g., tweets and online reviews, that come in real time. GeoTrend is distinguished from existing techniques in three aspects: (1) It discovers trends in arbitrary spatial regions, e.g., city blocks. (2) It supports trending measures that effectively capture trending items under a variety of definitions that suit different applications. (3) It promotes recent Microblogs as first-class citizens and optimizes its system components to digest a continuous flow of fast data in main-memory while removing old data efficiently. GeoTrend queries are top-k queries that discover the most trending k keywords that are posted within an arbitrary spatial region and during the last T time units. To support its queries efficiently, GeoTrend employs an in-memory spatial index that is able to efficiently digest incoming data and expire data that is beyond the last T time units. The index also materializes top-k keywords in different spatial regions so that incoming queries can be processed with low latency. In case of peak times, a main-memory optimization technique is employed to shed less important data, so that the system still sustains high query accuracy with limited memory resources. Experimental results based on real Twitter feed and Bing Mobile spatial search queries show the scalability of GeoTrend to support arrival rates of up to 50,000 microblog/second, average query latency of 3 milli-seconds, and at least 90+% query accuracy even under limited memory resources.
-
mercury a memory constrained spatio temporal real time search on Microblogs
International Conference on Data Engineering, 2014Co-Authors: Amr Magdy, Mohamed F Mokbel, Sameh Elnikety, Suman NathAbstract:This paper presents Mercury; a system for real-time support of top-k spatio-temporal queries on Microblogs, where users are able to browse recent Microblogs near their locations. With high arrival rates of Microblogs, Mercury ensures real-time query response within a tight memory-constrained environment. Mercury bounds its search space to include only those Microblogs that have arrived within certain spatial and temporal boundaries, in which only the top-k Microblogs, according to a spatio-temporal ranking function, are returned in the search results. Mercury employs: (a) a scalable dynamic in-memory index structure that is capable of digesting all incoming Microblogs, (b) an efficient query processor that exploits the in-memory index through spatio-temporal pruning techniques that reduce the number of visited Microblogs to return the final answer, (c) an index size tuning module that dynamically finds and adjusts the minimum index size to ensure that incoming queries will be answered accurately, and (d) a load shedding technique that trades slight decrease in query accuracy for significant storage savings. Extensive experimental results based on a real-time Twitter Firehose feed and actual locations of Bing search queries show that Mercury supports high arrival rates of up to 64K Microblogs/second and average query latency of 4 msec.
Jugal Kalita - One of the best experts on this subject based on the ideXlab platform.
-
summarization of twitter Microblogs
The Computer Journal, 2014Co-Authors: Beaux Sharifi, David I Inouye, Jugal KalitaAbstract:Owing to the sheer volume of text generated by a microblog site like Twitter, it is often difficult to fully understand what is being said about various topics. This paper presents algorithms for summarizing microblog documents. Initially, we present algorithms that produce single-document summaries but later extend them to produce summaries containing multiple documents. We evaluate the generated summaries by comparing them to both manually produced summaries and, for the multiple-post summaries, to the summarization results of some of the leading traditional summarization systems.
-
summarizing Microblogs automatically
North American Chapter of the Association for Computational Linguistics, 2010Co-Authors: Beaux Sharifi, Markanthony Hutton, Jugal KalitaAbstract:In this paper, we focus on a recent Web trend called microblogging, and in particular a site called Twitter. The content of such a site is an extraordinarily large number of small textual messages, posted by millions of users, at random or in response to perceived events or situations. We have developed an algorithm that takes a trending phrase or any phrase specified by a user, collects a large number of posts containing the phrase, and provides an automatically created summary of the posts related to the term. We present examples of summaries we produce along with initial evaluation.