The Experts below are selected from a list of 7371 Experts worldwide ranked by ideXlab platform

Mohamed F Mokbel - One of the best experts on this subject based on the ideXlab platform.

  • ST-Hadoop: a Mapreduce Framework for spatio-temporal data
    GeoInformatica, 2018
    Co-Authors: Louai Alarabi, Mohamed F Mokbel, Mashaal Musleh
    Abstract:

    This paper presents ST-Hadoop; the first full-fledged open-source Mapreduce Framework with a native support for spatio-temporal data. ST-Hadoop is a comprehensive extension to Hadoop and SpatialHadoop that injects spatio-temporal data awareness inside each of their layers, mainly, language, indexing, and operations layers. In the language layer, ST-Hadoop provides built in spatio-temporal data types and operations. In the indexing layer, ST-Hadoop spatiotemporally loads and divides data across computation nodes in Hadoop Distributed File System in a way that mimics spatio-temporal index structures, which result in achieving orders of magnitude better performance than Hadoop and SpatialHadoop when dealing with spatio-temporal data and queries. In the operations layer, ST-Hadoop shipped with support for three fundamental spatio-temporal queries, namely, spatio-temporal range, top-k nearest neighbor, and join queries. Extensibility of ST-Hadoop allows others to extend features and operations easily using similar approaches described in the paper. Extensive experiments conducted on large-scale dataset of size 10 TB that contains over 1 Billion spatio-temporal records, to show that ST-Hadoop achieves orders of magnitude better performance than Hadoop and SpaitalHadoop when dealing with spatio-temporal data and operations. The key idea behind the performance gained in ST-Hadoop is its ability in indexing spatio-temporal data within Hadoop Distributed File System.

  • st hadoop a Mapreduce Framework for spatio temporal data
    Symposium on Large Spatial Databases, 2017
    Co-Authors: Louai Alarabi, Mohamed F Mokbel, Mashaal Musleh
    Abstract:

    This paper presents ST-Hadoop; the first full-fledged open-source Mapreduce Framework with a native support for spatio-temporal data. ST-Hadoop is a comprehensive extension to Hadoop and SpatialHadoop that injects spatio-temporal data awareness inside each of their layers, mainly, language, indexing, and operations layers. In the language layer, ST-Hadoop provides built in spatio-temporal data types and operations. In the indexing layer, ST-Hadoop spatiotemporally loads and divides data across computation nodes in Hadoop Distributed File System in a way that mimics spatio-temporal index structures, which result in achieving orders of magnitude better performance than Hadoop and SpatialHadoop when dealing with spatio-temporal data and queries. In the operations layer, ST-Hadoop shipped with support for two fundamental spatio-temporal queries, namely, spatio-temporal range and join queries. Extensibility of ST-Hadoop allows others to expand features and operations easily using similar approach described in the paper. Extensive experiments conducted on large-scale dataset of size 10 TB that contains over 1 Billion spatio-temporal records, to show that ST-Hadoop achieves orders of magnitude better performance than Hadoop and SpaitalHadoop when dealing with spatio-temporal data and operations. The key idea behind the performance gained in ST-Hadoop is its ability in indexing spatio-temporal data within Hadoop Distributed File System.

  • a demonstration of st hadoop a Mapreduce Framework for big spatio temporal data
    Very Large Data Bases, 2017
    Co-Authors: Louai Alarabi, Mohamed F Mokbel
    Abstract:

    This demo presents ST-Hadoop; the first full-fledged open-source Mapreduce Framework with a native support for spatio-temporal data. ST-Hadoop injects spatio-temporal awareness in the Hadoop base code, which results in achieving order(s) of magnitude better performance than Hadoop and SpatialHadoop when dealing with spatio-temporal data and queries. The key idea behind ST-Hadoop is its ability in indexing spatio-temporal data within Hadoop Distributed File System (HDFS). A real system prototype of ST-Hadoop, running on a local cluster of 24 machines, is demonstrated with two big-spatio-temporal datasets of Twitter and NYC Taxi data, each of around one billion records.

  • hadoopviz a Mapreduce Framework for extensible visualization of big spatial data
    International Conference on Data Engineering, 2016
    Co-Authors: Ahmed Eldawy, Mohamed F Mokbel, Christopher Jonathan
    Abstract:

    This paper introduces HadoopViz; a Mapreduce-based Framework for visualizing big spatial data. HadoopViz has three unique features that distinguish it from other techniques. (1) It exposes an extensible interface which allows users to define a new visualization types, e.g., scatter plot, road network, or heat map, by defining five abstract functions, without delving into the implementation details of the Mapreduce algorithms. As it is open source, HadoopViz allows algorithm designers to focus on how the data should be visualized rather than performance or scalability issues. (2) HadoopViz is capable of generating big images with giga-pixel resolution by employing a three-phase technique, partition-plot-merge. (3) HadoopViz provides a smoothing functionality which can fuse nearby records together as the image is plotted. This makes it capable of generating more types of images with high quality as compared to existing work. Experimental results on real datasets of up to 14 Billion points show the extensibility, scalability, and efficiency of HadoopViz to handle different visualization types of spatial big data.

  • spatialhadoop a Mapreduce Framework for spatial data
    International Conference on Data Engineering, 2015
    Co-Authors: Ahmed Eldawy, Mohamed F Mokbel
    Abstract:

    This paper describes SpatialHadoop; a full-fledged Mapreduce Framework with native support for spatial data. SpatialHadoop is a comprehensive extension to Hadoop that injects spatial data awareness in each Hadoop layer, namely, the language, storage, Mapreduce, and operations layers. In the language layer, SpatialHadoop adds a simple and expressive high level language for spatial data types and operations. In the storage layer, SpatialHadoop adapts traditional spatial index structures, Grid, R-tree and R+-tree, to form a two-level spatial index. SpatialHadoop enriches the Mapreduce layer by two new components, SpatialFileSplitter and SpatialRecordReader, for efficient and scalable spatial data processing. In the operations layer, SpatialHadoop is already equipped with a dozen of operations, including range query, kNN, and spatial join. Other spatial operations are also implemented following a similar approach. Extensive experiments on real system prototype and real datasets show that SpatialHadoop achieves orders of magnitude better performance than Hadoop for spatial data processing.

Louai Alarabi - One of the best experts on this subject based on the ideXlab platform.

  • ST-Hadoop: a Mapreduce Framework for spatio-temporal data
    GeoInformatica, 2018
    Co-Authors: Louai Alarabi, Mohamed F Mokbel, Mashaal Musleh
    Abstract:

    This paper presents ST-Hadoop; the first full-fledged open-source Mapreduce Framework with a native support for spatio-temporal data. ST-Hadoop is a comprehensive extension to Hadoop and SpatialHadoop that injects spatio-temporal data awareness inside each of their layers, mainly, language, indexing, and operations layers. In the language layer, ST-Hadoop provides built in spatio-temporal data types and operations. In the indexing layer, ST-Hadoop spatiotemporally loads and divides data across computation nodes in Hadoop Distributed File System in a way that mimics spatio-temporal index structures, which result in achieving orders of magnitude better performance than Hadoop and SpatialHadoop when dealing with spatio-temporal data and queries. In the operations layer, ST-Hadoop shipped with support for three fundamental spatio-temporal queries, namely, spatio-temporal range, top-k nearest neighbor, and join queries. Extensibility of ST-Hadoop allows others to extend features and operations easily using similar approaches described in the paper. Extensive experiments conducted on large-scale dataset of size 10 TB that contains over 1 Billion spatio-temporal records, to show that ST-Hadoop achieves orders of magnitude better performance than Hadoop and SpaitalHadoop when dealing with spatio-temporal data and operations. The key idea behind the performance gained in ST-Hadoop is its ability in indexing spatio-temporal data within Hadoop Distributed File System.

  • st hadoop a Mapreduce Framework for spatio temporal data
    Symposium on Large Spatial Databases, 2017
    Co-Authors: Louai Alarabi, Mohamed F Mokbel, Mashaal Musleh
    Abstract:

    This paper presents ST-Hadoop; the first full-fledged open-source Mapreduce Framework with a native support for spatio-temporal data. ST-Hadoop is a comprehensive extension to Hadoop and SpatialHadoop that injects spatio-temporal data awareness inside each of their layers, mainly, language, indexing, and operations layers. In the language layer, ST-Hadoop provides built in spatio-temporal data types and operations. In the indexing layer, ST-Hadoop spatiotemporally loads and divides data across computation nodes in Hadoop Distributed File System in a way that mimics spatio-temporal index structures, which result in achieving orders of magnitude better performance than Hadoop and SpatialHadoop when dealing with spatio-temporal data and queries. In the operations layer, ST-Hadoop shipped with support for two fundamental spatio-temporal queries, namely, spatio-temporal range and join queries. Extensibility of ST-Hadoop allows others to expand features and operations easily using similar approach described in the paper. Extensive experiments conducted on large-scale dataset of size 10 TB that contains over 1 Billion spatio-temporal records, to show that ST-Hadoop achieves orders of magnitude better performance than Hadoop and SpaitalHadoop when dealing with spatio-temporal data and operations. The key idea behind the performance gained in ST-Hadoop is its ability in indexing spatio-temporal data within Hadoop Distributed File System.

  • a demonstration of st hadoop a Mapreduce Framework for big spatio temporal data
    Very Large Data Bases, 2017
    Co-Authors: Louai Alarabi, Mohamed F Mokbel
    Abstract:

    This demo presents ST-Hadoop; the first full-fledged open-source Mapreduce Framework with a native support for spatio-temporal data. ST-Hadoop injects spatio-temporal awareness in the Hadoop base code, which results in achieving order(s) of magnitude better performance than Hadoop and SpatialHadoop when dealing with spatio-temporal data and queries. The key idea behind ST-Hadoop is its ability in indexing spatio-temporal data within Hadoop Distributed File System (HDFS). A real system prototype of ST-Hadoop, running on a local cluster of 24 machines, is demonstrated with two big-spatio-temporal datasets of Twitter and NYC Taxi data, each of around one billion records.

  • ST-Hadoop: A Mapreduce Framework for Big Spatio-temporal Data
    Proceedings of the 2017 ACM International Conference on Management of Data - SIGMOD '17, 2017
    Co-Authors: Louai Alarabi
    Abstract:

    This paper presents ST-Hadoop; the first full-fledged open-source Mapreduce Framework with a native support for spatio-temporal data. ST-Hadoop is a comprehensive extension to Hadoop and SpatialHadoop that injects spatio-temporal data awareness inside each of their layers, mainly, language, indexing, and operations layers. In the language layer, ST-Hadoop provides built in spatio-temporal data types and operations. In the indexing layer, ST-Hadoop spatiotemporally loads and divides data across computation nodes in Hadoop Distributed File System in a way that mimics spatio-temporal index structures, which result in achieving orders of magnitude better performance than Hadoop and SpatialHadoop when dealing with spatio-temporal data and queries. In the operations layer, ST-Hadoop shipped with support for two fundamental spatio-temporal queries, namely, spatio-temporal range and join queries. Extensibility of ST-Hadoop allows others to expand features and operations easily using similar approach described in the paper. Extensive experiments conducted on large-scale dataset of size 10 TB that contains over 1 Billion spatio-temporal records, to show that ST-Hadoop achieves orders of magnitude better performance than Hadoop and SpaitalHadoop when dealing with spatio-temporal data and operations. The key idea behind the performance gained in ST-Hadoop is its ability in indexing spatio-temporal data within Hadoop Distributed File System.

Ahmed Eldawy - One of the best experts on this subject based on the ideXlab platform.

  • hadoopviz a Mapreduce Framework for extensible visualization of big spatial data
    International Conference on Data Engineering, 2016
    Co-Authors: Ahmed Eldawy, Mohamed F Mokbel, Christopher Jonathan
    Abstract:

    This paper introduces HadoopViz; a Mapreduce-based Framework for visualizing big spatial data. HadoopViz has three unique features that distinguish it from other techniques. (1) It exposes an extensible interface which allows users to define a new visualization types, e.g., scatter plot, road network, or heat map, by defining five abstract functions, without delving into the implementation details of the Mapreduce algorithms. As it is open source, HadoopViz allows algorithm designers to focus on how the data should be visualized rather than performance or scalability issues. (2) HadoopViz is capable of generating big images with giga-pixel resolution by employing a three-phase technique, partition-plot-merge. (3) HadoopViz provides a smoothing functionality which can fuse nearby records together as the image is plotted. This makes it capable of generating more types of images with high quality as compared to existing work. Experimental results on real datasets of up to 14 Billion points show the extensibility, scalability, and efficiency of HadoopViz to handle different visualization types of spatial big data.

  • spatialhadoop a Mapreduce Framework for spatial data
    International Conference on Data Engineering, 2015
    Co-Authors: Ahmed Eldawy, Mohamed F Mokbel
    Abstract:

    This paper describes SpatialHadoop; a full-fledged Mapreduce Framework with native support for spatial data. SpatialHadoop is a comprehensive extension to Hadoop that injects spatial data awareness in each Hadoop layer, namely, the language, storage, Mapreduce, and operations layers. In the language layer, SpatialHadoop adds a simple and expressive high level language for spatial data types and operations. In the storage layer, SpatialHadoop adapts traditional spatial index structures, Grid, R-tree and R+-tree, to form a two-level spatial index. SpatialHadoop enriches the Mapreduce layer by two new components, SpatialFileSplitter and SpatialRecordReader, for efficient and scalable spatial data processing. In the operations layer, SpatialHadoop is already equipped with a dozen of operations, including range query, kNN, and spatial join. Other spatial operations are also implemented following a similar approach. Extensive experiments on real system prototype and real datasets show that SpatialHadoop achieves orders of magnitude better performance than Hadoop for spatial data processing.

  • spatialhadoop a Mapreduce Framework for spatial data
    International Conference on Data Engineering, 2015
    Co-Authors: Ahmed Eldawy, Mohamed F Mokbel
    Abstract:

    This paper describes SpatialHadoop; a full-fledged Mapreduce Framework with native support for spatial data. SpatialHadoop is a comprehensive extension to Hadoop that injects spatial data awareness in each Hadoop layer, namely, the language, storage, Mapreduce, and operations layers. In the language layer, SpatialHadoop adds a simple and expressive high level language for spatial data types and operations. In the storage layer, SpatialHadoop adapts traditional spatial index structures, Grid, R-tree and R+-tree, to form a two-level spatial index. SpatialHadoop enriches the Mapreduce layer by two new components, SpatialFileSplitter and SpatialRecordReader, for efficient and scalable spatial data processing. In the operations layer, SpatialHadoop is already equipped with a dozen of operations, including range query, kNN, and spatial join. Other spatial operations are also implemented following a similar approach. Extensive experiments on real system prototype and real datasets show that SpatialHadoop achieves orders of magnitude better performance than Hadoop for spatial data processing.

  • spatialhadoop towards flexible and scalable spatial processing using Mapreduce
    International Conference on Management of Data, 2014
    Co-Authors: Ahmed Eldawy
    Abstract:

    Recently, Mapreduce Frameworks, e.g., Hadoop, have been used extensively in different applications that include tera-byte sorting, machine learning, and graph processing. With the huge volumes of spatial data coming from different sources, there is an increasing demand to exploit the efficiency of Hadoop, coupled with the flexibility of the Mapreduce Framework, in spatial data processing. However, Hadoop falls short in supporting spatial data efficiently as the core is unaware of spatial data properties. This paper describes SpatialHadoop; a full-edged Mapreduce Framework with native support for spatial data. SpatialHadoop is a comprehensive extension to Hadoop that injects spatial data awareness in each Hadoop layer, namely, the language, storage, Mapreduce, and operations layers. In the language layer, SpatialHadoop adds a simple and ex- pressive high level language for spatial data types and operations. In the storage layer, SpatialHadoop adapts traditional spatial index structures, Grid, R-tree and R+-tree, to form a two-level spatial index. SpatialHadoop enriches the Mapreduce layer by two new components, SpatialFileSplitter and SpatialRecordReader, for efficient and scalable spatial data processing. In the operations layer, SpatialHadoop is already equipped with a dozen of operations, including range query, kNN, and spatial join. The flexibility and open source nature of SpatialHadoop allows more spatial operations to be implemented efficiently using Mapreduce. Extensive experiments on a real system prototype and real datasets show that SpatialHadoop achieves orders of magnitude better performance than Hadoop for spatial data processing.

  • a demonstration of spatialhadoop an efficient Mapreduce Framework for spatial data
    Very Large Data Bases, 2013
    Co-Authors: Ahmed Eldawy, Mohamed F Mokbel
    Abstract:

    This demo presents SpatialHadoop as the first full-fledged Mapreduce Framework with native support for spatial data. SpatialHadoop is a comprehensive extension to Hadoop that pushes spatial data inside the core functionality of Hadoop. SpatialHadoop runs existing Hadoop programs as is, yet, it achieves order(s) of magnitude better performance than Hadoop when dealing with spatial data. SpatialHadoop employs a simple spatial high level language, a two-level spatial index structure, basic spatial components built inside the Mapreduce layer, and three basic spatial operations: range queries, k-NN queries, and spatial join. Other spatial operations can be similarly deployed in SpatialHadoop. We demonstrate a real system prototype of SpatialHadoop running on an Amazon EC2 cluster against two sets of real spatial data obtained from Tiger Files and OpenStreetMap with sizes 60GB and 300GB, respectively.

Francisco Herrera - One of the best experts on this subject based on the ideXlab platform.

  • a Mapreduce approach to address big data classification problems based on the fusion of linguistic fuzzy rules
    International Journal of Computational Intelligence Systems, 2015
    Co-Authors: Sara Del Rio, Victoria Lopez, Jose Manuel Benitez, Francisco Herrera
    Abstract:

    AbstractThe big data term is used to describe the exponential data growth that has recently occurred and represents an immense challenge for traditional learning techniques. To deal with big data classification problems we propose the Chi-FRBCS-BigData algorithm, a linguistic fuzzy rule-based classification system that uses the Mapreduce Framework to learn and fuse rule bases. It has been developed in two versions with different fusion processes. An experimental study is carried out and the results obtained show that the proposal is able to handle these problems providing competitive results.

  • cost sensitive linguistic fuzzy rule based classification systems under the Mapreduce Framework for imbalanced big data
    Fuzzy Sets and Systems, 2015
    Co-Authors: Victoria Lopez, Sara Del Rio, Jose Manuel Benitez, Francisco Herrera
    Abstract:

    Classification with big data has become one of the latest trends when talking about learning from the available information. The data growth in the last years has rocketed the interest in effectively acquiring knowledge to analyze and predict trends. The variety and veracity that are related to big data introduce a degree of uncertainty that has to be handled in addition to the volume and velocity requirements. This data usually also presents what is known as the problem of classification with imbalanced datasets, a class distribution where the most important concepts to be learned are presented by a negligible number of examples in relation to the number of examples from the other classes. In order to adequately deal with imbalanced big data we propose the Chi-FRBCS-BigDataCS algorithm, a fuzzy rule based classification system that is able to deal with the uncertainly that is introduced in large volumes of data without disregarding the learning in the underrepresented class. The method uses the Mapreduce Framework to distribute the computational operations of the fuzzy model while it includes cost-sensitive learning techniques in its design to address the imbalance that is present in the data. The good performance of this approach is supported by the experimental analysis that is carried out over twenty-four imbalanced big data cases of study. The results obtained show that the proposal is able to handle these problems obtaining competitive results both in the classification performance of the model and the time needed for the computation.

Bingsheng He - One of the best experts on this subject based on the ideXlab platform.

  • MrPhi: An Optimized Mapreduce Framework on Intel Xeon Phi Coprocessors
    IEEE Transactions on Parallel and Distributed Systems, 2015
    Co-Authors: Mian Lu, Huynh Phung Huynh, Yun Liang, Bingsheng He
    Abstract:

    In this work, we develop MrPhi , an optimized Mapreduce Framework on a heterogeneous computing platform, particularly equipped with multiple Intel Xeon Phi coprocessors. To the best of our knowledge, this is the first work to optimize the Mapreduce Framework on the Xeon Phi. We first focus on employing advanced features of the Xeon Phi to achieve high performance on a single coprocessor. We propose a vectorization friendly technique and SIMD hash computation algorithms to utilize the SIMD vectors. Then we pipeline the map and reduce phases to improve the resource utilization. Furthermore, we eliminate multiple local arrays but use low cost atomic operations on the global array to improve the thread scalability. For a given application, our Framework is able to automatically detect suitable techniques to apply. Moreover, we extend our Framework to a heterogeneous platform to utilize all hardware resource effectively. We adopt non-blocking data transfer to hide the communication overhead. We also adopt aligned memory transfer in order to fully utilize the PCIe bandwidth between the host and coprocessor. We conduct comprehensive experiments to benchmark the Xeon Phi and compare our optimized Mapreduce Framework with a state-of-the-art multi-core based Mapreduce Framework (Phoenix++). By evaluating six real-world applications, the experimental results show that our optimized Framework is 1.2 to 38 $\times$ faster than Phoenix++ for various applications on a single Xeon Phi. Additionally, the performance of four applications is able to achieve linear scalability on a platform equipped with up to four Xeon Phi coprocessors.