The Experts below are selected from a list of 2517 Experts worldwide ranked by ideXlab platform

Lofstead Jay - One of the best experts on this subject based on the ideXlab platform.

  • TMaR: a two‑stage Mapreduce scheduler for heterogeneous environments
    2020
    Co-Authors: Maleki Neda, Faragardi, Hamid Reza, Rahmani, Amir Masoud, Conti Mauro, Lofstead Jay
    Abstract:

    In the context of Mapreduce Task scheduling, many algorithms mainly focus on the scheduling of Reduce Tasks with the assumption that scheduling of Map Tasks is already done. However, in the cloud deployments of Mapreduce, the input data is located on remote storage which indicates the importance of the scheduling of Map Tasks as well. In this paper, we propose a two-stage Map and Reduce Task scheduler for heterogeneous environments, called TMaR. TMaR schedules Map and Reduce Tasks on the servers that minimize the Task fnish time in each stage, respectively. We employ a dynamic partition binder for Reduce Tasks in the Reduce stage to lighten the shufing trafc. Indeed, TMaR minimizes the makespan of a batch of Tasks in heterogeneous environments while considering the network trafc. The simulation results demonstrate that TMaR outperforms Hadoop-stock and Hadoop-A in terms of makespan and network trafc and achieves by an average of 29%, 36%, and 14% performance using Wordcount, Sort, and Grep benchmarks. Besides, the power reduction of TMaR is up to 12%

Lofstead J. - One of the best experts on this subject based on the ideXlab platform.

  • TMaR: a two-stage Mapreduce scheduler for heterogeneous environments
    'Springer Science and Business Media LLC', 2020
    Co-Authors: Maleki N., Faragardi H. R., Rahmani A. M., Conti M., Lofstead J.
    Abstract:

    In the context of Mapreduce Task scheduling, many algorithms mainly focus on the scheduling of Reduce Tasks with the assumption that scheduling of Map Tasks is already done. However, in the cloud deployments of Mapreduce, the input data is located on remote storage which indicates the importance of the scheduling of Map Tasks as well. In this paper, we propose a two-stage Map and Reduce Task scheduler for heterogeneous environments, called TMaR. TMaR schedules Map and Reduce Tasks on the servers that minimize the Task finish time in each stage, respectively. We employ a dynamic partition binder for Reduce Tasks in the Reduce stage to lighten the shuffling traffic. Indeed, TMaR minimizes the makespan of a batch of Tasks in heterogeneous environments while considering the network traffic. The simulation results demonstrate that TMaR outperforms Hadoop-stock and Hadoop-A in terms of makespan and network traffic and achieves by an average of 29%, 36%, and 14% performance using Wordcount, Sort, and Grep benchmarks. Besides, the power reduction of TMaR is up to 12%

K. K. Phang - One of the best experts on this subject based on the ideXlab platform.

  • An enhanced hadoop heartbeat mechanism for Mapreduce Task scheduler using dynamic calibration
    China Communications, 2018
    Co-Authors: K. K. Phang
    Abstract:

    Mapreduce is a popular programming model for processing large-scale datasets in a distributed environment and is a fundamental component of current cloud computing and big data applications. In this paper, a heartbeat mechanism for Mapreduce Task Scheduler using Dynamic Calibration (HMTS-DC) is proposed to address the unbalanced node computation capacity problem in a heterogeneous Mapreduce environment. HMTS-DC uses two mechanisms to dynamically adapt and balance Tasks assigned to each compute node: 1) using heartbeat to dynamically estimate the capacity of the compute nodes, and 2) using data locality of replicated data blocks to reduce data transfer between nodes. With the first mechanism, based on the heartbeats received during the early state of the job, the Task scheduler can dynamically estimate the computational capacity of each node. Using the second mechanism, unprocessed Tasks local to each compute node are reassigned and reserved to allow nodes with greater capacities to reserve more local Tasks than their weaker counterparts. Experimental results show that HMTS-DC performs better than Hadoop and Dynamic Data Placement Strategy (DDP) in a dynamic environment. Furthermore, an enhanced HMTS-DC (EHMTS-DC) is proposed by incorporating historical data. In contrast to the "slow start" property of HMTS-DC, EHMTS-DC relies on the historical computation capacity of the slave machines. The experimental results show that EHMTS-DC outperforms HMTS-DC in a dynamic environment.

Maleki Neda - One of the best experts on this subject based on the ideXlab platform.

  • TMaR: a two‑stage Mapreduce scheduler for heterogeneous environments
    2020
    Co-Authors: Maleki Neda, Faragardi, Hamid Reza, Rahmani, Amir Masoud, Conti Mauro, Lofstead Jay
    Abstract:

    In the context of Mapreduce Task scheduling, many algorithms mainly focus on the scheduling of Reduce Tasks with the assumption that scheduling of Map Tasks is already done. However, in the cloud deployments of Mapreduce, the input data is located on remote storage which indicates the importance of the scheduling of Map Tasks as well. In this paper, we propose a two-stage Map and Reduce Task scheduler for heterogeneous environments, called TMaR. TMaR schedules Map and Reduce Tasks on the servers that minimize the Task fnish time in each stage, respectively. We employ a dynamic partition binder for Reduce Tasks in the Reduce stage to lighten the shufing trafc. Indeed, TMaR minimizes the makespan of a batch of Tasks in heterogeneous environments while considering the network trafc. The simulation results demonstrate that TMaR outperforms Hadoop-stock and Hadoop-A in terms of makespan and network trafc and achieves by an average of 29%, 36%, and 14% performance using Wordcount, Sort, and Grep benchmarks. Besides, the power reduction of TMaR is up to 12%

Maleki N. - One of the best experts on this subject based on the ideXlab platform.

  • TMaR: a two-stage Mapreduce scheduler for heterogeneous environments
    'Springer Science and Business Media LLC', 2020
    Co-Authors: Maleki N., Faragardi H. R., Rahmani A. M., Conti M., Lofstead J.
    Abstract:

    In the context of Mapreduce Task scheduling, many algorithms mainly focus on the scheduling of Reduce Tasks with the assumption that scheduling of Map Tasks is already done. However, in the cloud deployments of Mapreduce, the input data is located on remote storage which indicates the importance of the scheduling of Map Tasks as well. In this paper, we propose a two-stage Map and Reduce Task scheduler for heterogeneous environments, called TMaR. TMaR schedules Map and Reduce Tasks on the servers that minimize the Task finish time in each stage, respectively. We employ a dynamic partition binder for Reduce Tasks in the Reduce stage to lighten the shuffling traffic. Indeed, TMaR minimizes the makespan of a batch of Tasks in heterogeneous environments while considering the network traffic. The simulation results demonstrate that TMaR outperforms Hadoop-stock and Hadoop-A in terms of makespan and network traffic and achieves by an average of 29%, 36%, and 14% performance using Wordcount, Sort, and Grep benchmarks. Besides, the power reduction of TMaR is up to 12%