The Experts below are selected from a list of 351498 Experts worldwide ranked by ideXlab platform
Amol Deshpande - One of the best experts on this subject based on the ideXlab platform.
-
sword scalable Workload aware data placement for transactional Workloads
Extending Database Technology, 2013Co-Authors: Abdul Quamar, Ashwin K Kumar, Amol DeshpandeAbstract:In this paper, we address the problem of transparently scaling out transactional (OLTP) Workloads on relational databases, to support database-as-a-service in cloud computing environment. The primary challenges in supporting such Workloads include choosing how to partition the data across a large number of machines, minimizing the number of distributed transactions, providing high data availability, and tolerating failures gracefully. Capturing and modeling the transactional Workload over a period of time, and then exploiting that information for data placement and replication has been shown to provide significant benefits in performance, both in terms of transaction latencies and overall throughput. However, such Workload-aware data placement approaches can incur very high overheads, and further, may perform worse than naive approaches if the Workload changes. In this work, we propose SWORD, a s calable wor kload-aware d ata partitioning and placement approach for OLTP Workloads, that incorporates a suite of novel techniques to significantly reduce the overheads incurred both during the initial placement, and during query execution at runtime. We model the Workload as a hypergraph over the data items, and propose using a hypergraph compression technique to reduce the overheads of partitioning. To deal with Workload changes, we propose an incremental data repartitioning technique that modifies data placement in small steps without resorting to complete Workload repartitioning. We have built a Workload-aware active replication mechanism in SWORD to increase availability and enable load balancing. We propose the use of fine-grained quorums defined at the level of groups of tuples to control the cost of distributed updates, improve throughput, and provide adaptability to different Workloads. To our knowledge, SWORD is the first system that uses fine-grained quorums in this context. The results of our experimental evaluation on SWORD deployed on an Amazon EC2 cluster show that our techniques result in orders-of-magnitude reductions in the partitioning and book-keeping overheads, and improve tolerance to failures and Workload changes; we also show that choosing quorums based on the query access patterns enables us to better handle query Workloads with different read and write access patterns.
-
EDBT - SWORD: scalable Workload-aware data placement for transactional Workloads
Proceedings of the 16th International Conference on Extending Database Technology - EDBT '13, 2013Co-Authors: Abdul Quamar, K. Ashwin Kumar, Amol DeshpandeAbstract:In this paper, we address the problem of transparently scaling out transactional (OLTP) Workloads on relational databases, to support database-as-a-service in cloud computing environment. The primary challenges in supporting such Workloads include choosing how to partition the data across a large number of machines, minimizing the number of distributed transactions, providing high data availability, and tolerating failures gracefully. Capturing and modeling the transactional Workload over a period of time, and then exploiting that information for data placement and replication has been shown to provide significant benefits in performance, both in terms of transaction latencies and overall throughput. However, such Workload-aware data placement approaches can incur very high overheads, and further, may perform worse than naive approaches if the Workload changes. In this work, we propose SWORD, a s calable wor kload-aware d ata partitioning and placement approach for OLTP Workloads, that incorporates a suite of novel techniques to significantly reduce the overheads incurred both during the initial placement, and during query execution at runtime. We model the Workload as a hypergraph over the data items, and propose using a hypergraph compression technique to reduce the overheads of partitioning. To deal with Workload changes, we propose an incremental data repartitioning technique that modifies data placement in small steps without resorting to complete Workload repartitioning. We have built a Workload-aware active replication mechanism in SWORD to increase availability and enable load balancing. We propose the use of fine-grained quorums defined at the level of groups of tuples to control the cost of distributed updates, improve throughput, and provide adaptability to different Workloads. To our knowledge, SWORD is the first system that uses fine-grained quorums in this context. The results of our experimental evaluation on SWORD deployed on an Amazon EC2 cluster show that our techniques result in orders-of-magnitude reductions in the partitioning and book-keeping overheads, and improve tolerance to failures and Workload changes; we also show that choosing quorums based on the query access patterns enables us to better handle query Workloads with different read and write access patterns.
Abdul Quamar - One of the best experts on this subject based on the ideXlab platform.
-
sword scalable Workload aware data placement for transactional Workloads
Extending Database Technology, 2013Co-Authors: Abdul Quamar, Ashwin K Kumar, Amol DeshpandeAbstract:In this paper, we address the problem of transparently scaling out transactional (OLTP) Workloads on relational databases, to support database-as-a-service in cloud computing environment. The primary challenges in supporting such Workloads include choosing how to partition the data across a large number of machines, minimizing the number of distributed transactions, providing high data availability, and tolerating failures gracefully. Capturing and modeling the transactional Workload over a period of time, and then exploiting that information for data placement and replication has been shown to provide significant benefits in performance, both in terms of transaction latencies and overall throughput. However, such Workload-aware data placement approaches can incur very high overheads, and further, may perform worse than naive approaches if the Workload changes. In this work, we propose SWORD, a s calable wor kload-aware d ata partitioning and placement approach for OLTP Workloads, that incorporates a suite of novel techniques to significantly reduce the overheads incurred both during the initial placement, and during query execution at runtime. We model the Workload as a hypergraph over the data items, and propose using a hypergraph compression technique to reduce the overheads of partitioning. To deal with Workload changes, we propose an incremental data repartitioning technique that modifies data placement in small steps without resorting to complete Workload repartitioning. We have built a Workload-aware active replication mechanism in SWORD to increase availability and enable load balancing. We propose the use of fine-grained quorums defined at the level of groups of tuples to control the cost of distributed updates, improve throughput, and provide adaptability to different Workloads. To our knowledge, SWORD is the first system that uses fine-grained quorums in this context. The results of our experimental evaluation on SWORD deployed on an Amazon EC2 cluster show that our techniques result in orders-of-magnitude reductions in the partitioning and book-keeping overheads, and improve tolerance to failures and Workload changes; we also show that choosing quorums based on the query access patterns enables us to better handle query Workloads with different read and write access patterns.
-
EDBT - SWORD: scalable Workload-aware data placement for transactional Workloads
Proceedings of the 16th International Conference on Extending Database Technology - EDBT '13, 2013Co-Authors: Abdul Quamar, K. Ashwin Kumar, Amol DeshpandeAbstract:In this paper, we address the problem of transparently scaling out transactional (OLTP) Workloads on relational databases, to support database-as-a-service in cloud computing environment. The primary challenges in supporting such Workloads include choosing how to partition the data across a large number of machines, minimizing the number of distributed transactions, providing high data availability, and tolerating failures gracefully. Capturing and modeling the transactional Workload over a period of time, and then exploiting that information for data placement and replication has been shown to provide significant benefits in performance, both in terms of transaction latencies and overall throughput. However, such Workload-aware data placement approaches can incur very high overheads, and further, may perform worse than naive approaches if the Workload changes. In this work, we propose SWORD, a s calable wor kload-aware d ata partitioning and placement approach for OLTP Workloads, that incorporates a suite of novel techniques to significantly reduce the overheads incurred both during the initial placement, and during query execution at runtime. We model the Workload as a hypergraph over the data items, and propose using a hypergraph compression technique to reduce the overheads of partitioning. To deal with Workload changes, we propose an incremental data repartitioning technique that modifies data placement in small steps without resorting to complete Workload repartitioning. We have built a Workload-aware active replication mechanism in SWORD to increase availability and enable load balancing. We propose the use of fine-grained quorums defined at the level of groups of tuples to control the cost of distributed updates, improve throughput, and provide adaptability to different Workloads. To our knowledge, SWORD is the first system that uses fine-grained quorums in this context. The results of our experimental evaluation on SWORD deployed on an Amazon EC2 cluster show that our techniques result in orders-of-magnitude reductions in the partitioning and book-keeping overheads, and improve tolerance to failures and Workload changes; we also show that choosing quorums based on the query access patterns enables us to better handle query Workloads with different read and write access patterns.
Ashwin K Kumar - One of the best experts on this subject based on the ideXlab platform.
-
sword scalable Workload aware data placement for transactional Workloads
Extending Database Technology, 2013Co-Authors: Abdul Quamar, Ashwin K Kumar, Amol DeshpandeAbstract:In this paper, we address the problem of transparently scaling out transactional (OLTP) Workloads on relational databases, to support database-as-a-service in cloud computing environment. The primary challenges in supporting such Workloads include choosing how to partition the data across a large number of machines, minimizing the number of distributed transactions, providing high data availability, and tolerating failures gracefully. Capturing and modeling the transactional Workload over a period of time, and then exploiting that information for data placement and replication has been shown to provide significant benefits in performance, both in terms of transaction latencies and overall throughput. However, such Workload-aware data placement approaches can incur very high overheads, and further, may perform worse than naive approaches if the Workload changes. In this work, we propose SWORD, a s calable wor kload-aware d ata partitioning and placement approach for OLTP Workloads, that incorporates a suite of novel techniques to significantly reduce the overheads incurred both during the initial placement, and during query execution at runtime. We model the Workload as a hypergraph over the data items, and propose using a hypergraph compression technique to reduce the overheads of partitioning. To deal with Workload changes, we propose an incremental data repartitioning technique that modifies data placement in small steps without resorting to complete Workload repartitioning. We have built a Workload-aware active replication mechanism in SWORD to increase availability and enable load balancing. We propose the use of fine-grained quorums defined at the level of groups of tuples to control the cost of distributed updates, improve throughput, and provide adaptability to different Workloads. To our knowledge, SWORD is the first system that uses fine-grained quorums in this context. The results of our experimental evaluation on SWORD deployed on an Amazon EC2 cluster show that our techniques result in orders-of-magnitude reductions in the partitioning and book-keeping overheads, and improve tolerance to failures and Workload changes; we also show that choosing quorums based on the query access patterns enables us to better handle query Workloads with different read and write access patterns.
Dror G Feitelson - One of the best experts on this subject based on the ideXlab platform.
-
Resampling with Feedback — A New Paradigm of Using Workload Data for Performance Evaluation
Euro-Par 2016: Parallel Processing, 2016Co-Authors: Dror G FeitelsonAbstract:Reliable performance evaluations require representative Workloads. This has led to the use of accounting logs from production systems as a source for Workload data in simulations. But using such logs directly suffers from various deficiencies, such as providing data about only one specific situation, and lack of flexibility, namely the inability to adjust the Workload as needed. Creating Workload models solves some of these problems but creates others, most notably the danger of missing out on important details that were not recognized in advance, and therefore not included in the model. Resampling solves many of these deficiencies by combining the best of both worlds. It is based on partitioning real Workloads into basic components (e.g. the jobs contributed by different users), and then generating new Workloads by sampling from this pool of basic components. The generated Workloads are adjusted dynamically to the conditions of the simulated system using a feedback loop, which may adjust the throughput. Using this methodology analysts can create multiple varied (but related) Workloads from the same original log, all the time retaining much of the structure that exists in the original Workload. Resampling with feedback thus provides a new way to use Workload logs which benefits from the realism of logs while eliminating many of their drawbacks. In addition, it enables evaluations of throughput effects that are impossible with static Workloads.This paper was written to accompany a keynote address at EuroPar 2016. It summarizes my and my students’ work and reflects a personal view. The goal is to show the big picture and the building and interplay of ideas, at the possible expense of not providing a full overview of and comparison with related work.
-
Workload Modeling for Computer Systems Performance Evaluation
2015Co-Authors: Dror G FeitelsonAbstract:Reliable performance evaluations require the use of representative Workloads. This is no easy task since modern computer systems and their Workloads are complex, with many interrelated attributes and complicated structures. Experts often use sophisticated mathematics to analyze and describe Workload models, making these models difficult for practitioners to grasp. This book aims to close this gap by emphasizing the intuition and the reasoning behind the definitions and derivations related to the Workload models. It provides numerous examples from real production systems, with hundreds of graphs. Using this book, readers will be able to analyze collected Workload data and clean it if necessary, derive statistical models that include skewed marginal distributions and correlations, and consider the need for generative models and feedback from the system. The descriptive statistics techniques covered are also useful for other domains.
-
Workload resampling for performance evaluation of parallel job schedulers
Concurrency and Computation: Practice and Experience, 2014Co-Authors: Netanel Zakay, Dror G FeitelsonAbstract:Evaluating the performance of a computer system is based on using representative Workloads. Common practice is to either use real Workload traces to drive simulations or use statistical Workload models that are based on such traces. Such models allow various Workload attributes to be manipulated, thus providing desirable flexibility, but may lose details of the Workload's internal structure. To overcome this, we suggest to combine the benefits of real traces and flexible modeling. Focusing on the problem of evaluating the performance of parallel job schedulers, we partition the trace of submitted jobs into independent subtraces representing different users and then recombine them in various ways, while maintaining features such as long-range dependence and the daily and weekly cycles of activity. This facilitates the creation of longer Workload traces that enable longer simulations, the creation of multiple statistically similar Workloads that can be used to gauge confidence intervals, the creation of Workloads with different load levels, and increasing the frequency of specific events like large surges of activity. Copyright © 2014 John Wiley & Sons, Ltd.
-
ICPE - Workload resampling for performance evaluation of parallel job schedulers
Proceedings of the ACM SPEC international conference on International conference on performance engineering - ICPE '13, 2013Co-Authors: Netanel Zakay, Dror G FeitelsonAbstract:Evaluating the performance of a computer system is based on using representative Workloads. Common practice is to either use real Workload traces to drive simulations, or else to use statistical Workload models that are based on such traces. Such models allow various Workload attributes to be manipulated, thus providing desirable flexibility, but may lose details of the Workload's internal structure. To overcome this, we suggest to combine the benefits of real traces and flexible modeling. Focusing on the problem of evaluating the performance of parallel job schedulers, we partition each trace into independent subtraces representing different users, and then re-combine them in various ways, while maintaining features like the daily and weekly cycles of activity. This facilitates the creation of longer Workload traces that enable longer simulations, the creation of multiple statistically similar Workloads that can be used to gauge confidence intervals, and the creation of Workloads with different load levels.
-
ISPASS - Workload sanitation for performance evaluation
2006 IEEE International Symposium on Performance Analysis of Systems and Software, 2006Co-Authors: Dror G Feitelson, Dan TsafrirAbstract:The performance of computer systems depends, among other things, on the Workload. Performance evaluations are therefore often done using logs of Workloads on current productions systems, under the assumption that such real Workloads are representative and reliable; likewise, Workload modeling is typically based on real Workloads. We show, however, that real Workloads may also contain anomalies that make them non-representative and unreliable. This is a special case of multi-class Workloads, where one class is the "real" Workload which we wish to use in the evaluation, and the other class contaminates the log with "bogus" data. We provide several examples of this situation, including a previously unrecognized type of anomaly we call "Workload flurries": surges of activity with a repetitive nature, caused by a single user, that dominate the Workload for a relatively short period. Using a Workload with such anomalies in effect emphasizes rare and unique events (e.g. occurring for a few days out of two years of logged data), and risks optimizing the design decision for the anomalous Workload at the expense of the normal Workload. Thus we claim that such anomalies should be removed from the Workload before it is used in evaluations, and that ignoring them is actually an unjustifiable approach.
Hyeonsang Eom - One of the best experts on this subject based on the ideXlab platform.
-
Workload-aware resource management for software-defined compute
Cluster Computing, 2016Co-Authors: Yoonsung Nam, Minkyu Kang, Jincheol Kim, Hanul Sung, Hyeonsang EomAbstract:With advance of cloud computing technologies, there have been more diverse and heterogeneous Workloads running on cloud datacenters. As more and more Workloads run on the datacenters, the contention for the limited shared resources may increase, which can make the management of the resources difficult, often leading to low resource utilization. For effective resource management, the management software for the datacenters should be redesigned and used in a software-defined way to dynamically allocate “right” resources to Workloads based on different characteristics of Workloads so that they can decrease the cost of their operation while meeting the service level objectives such as satisfying the latency requirement. However, recent datacenter resource management frameworks do not operate in such software-defined ways, thus leading to not only the waste of resources, but also the performance degradation. To address this problem, we have designed and developed a Workload-aware resource management framework for software-defined compute. The framework consists mainly of the Workload profiler and Workload-aware schedulers. To demonstrate the effectiveness of the framework, we have prototyped the schedulers that minimize the interferences on the shared computing and memory resources. We have compared them with the existing schedulers in the OpenStack and VMWare vSphere testbeds, and evaluated its effectiveness in high contention scenarios. Our experimental study suggests that the use of our proposed approach can lead to up to 100 % improvements in throughput and up to 95 % reductions in tail latency for latency critical Workloads compared to the existing ones.
