The Experts below are selected from a list of 12684 Experts worldwide ranked by ideXlab platform

Schahram Dustdar - One of the best experts on this subject based on the ideXlab platform.

  • CBSE - Principles and methods for Elastic Computing
    Proceedings of the 17th international ACM Sigsoft symposium on Component-based software engineering - CBSE '14, 2014
    Co-Authors: Schahram Dustdar
    Abstract:

    In this talk I will address one of the most relevant challenges for a decade to come: How to design, model, and execute distributed systems utilizing the Internet of Things, software services, as well as human based services, considering modern Cloud Computing and Elasticity principles. Elasticity is seen as one of the main characteristics of Cloud Computing today. Is Elasticity simply scalability on steroids? In this talk I will discuss the main principles of Elasticity, present a fresh look at this problem, and examine how to integrate people, software services, and things into one composite system, which can be modeled, programmed, and deployed on a large scale in an Elastic way.

  • iterative test suites refinement for Elastic Computing systems
    Foundations of Software Engineering, 2013
    Co-Authors: Alessio Gambi, Antonio Filieri, Schahram Dustdar
    Abstract:

    Elastic Computing systems can dynamically scale to continuously and cost-effectively provide their required Quality of Service in face of time-varying workloads, and they are usually implemented in the cloud. Despite their wide-spread adoption by industry, a formal definition of Elasticity and suitable procedures for its assessment and verification are still missing. Both academia and industry are trying to adapt established testing procedures for functional and non-functional properties, with limited effectiveness with respect to Elasticity. In this paper we propose a new methodology to automatically generate test-suites for testing the Elastic properties of systems. Elasticity, plasticity, and oscillations are first formalized through a convenient behavioral abstraction of the Elastic system and then used to drive an iterative test suite refinement process. The outcomes of our approach are a test suite tailored to the violation of Elasticity properties and a human-readable abstraction of the system behavior to further support diagnosis and fix.

  • Advanced Web Services - Virtualizing Software and Human for Elastic Hybrid Services
    Advanced Web Services, 2013
    Co-Authors: Muhammad Zuhri Catur Candra, Hong-linh Truong, Rostyslav Zabolotnyi, Schahram Dustdar
    Abstract:

    Human capabilities have been incorporated into IT systems for solving complex problems since several years. Still, it is very challenging to program human capabilities due to the lack of techniques and tools. In this paper, we will discuss techniques and frameworks for conceptualizing and virtualizing human capabilities under programmable units and for provisioning them using cloud service models. We will discuss how Elastic composite applications can be built by combining programmable units of software-based and human-based services in the Vienna Elastic Computing Model.

  • modeling Elasticity trade offs in adaptive mixed systems
    Workshops on Enabling Technologies: Infrastracture for Collaborative Enterprises, 2013
    Co-Authors: Muhammad Zuhri Catur Candra, Hong-linh Truong, Schahram Dustdar
    Abstract:

    In the past decade, Elastic Computing has emerged as a popular solution approach for on-demand Computing resources' provisioning. Today, such dynamic resource provisioning is used not only in machine-based Computing systems but also in mixed systems containing machine- and human-based Computing elements. Many efforts have been undertaken to achieve resource Elasticity. However, to provide a truly Elastic computation, system designers also need to foresee trade-offs between cost, quality, and resources. Unfortunately, the lack of modeling tools leads to difficulty in designing on-demand provisioning strategy in mixed systems. In this paper we present a modeling tool named Elasticity Profile (EP) that specifies constructs for modeling the Elastic behavior of mixed systems with respect to trade-offs between cost, quality, and resources. We also present a conceptual framework where our EP can be deployed, utilized, and bound to runtime systems. Furthermore, we demonstrate the suitability of EP model using several use cases.

  • on estimating actuation delays in Elastic Computing systems
    Software Engineering for Adaptive and Self-Managing Systems, 2013
    Co-Authors: Alessio Gambi, Hong-linh Truong, Daniel Moldovan, Georgiana Copil, Schahram Dustdar
    Abstract:

    Elastic controllers autonomically adjust the allocation of resources in cloud Computing systems. Usually such controllers assume that control actions will take immediate effect. In clouds, however, actuation times may be long, and the controllers can hardly guarantee acceptable levels of service if they neglect these actuation delays. Therefore, the ability to correctly estimate the time that control actions take effect on the systems is crucial. However, detecting actuation delays in Elastic Computing systems is challenging because cloud systems provide only inaccurate and incomplete data about reconfigurations timing. In this paper, we tackle the problem of estimating the delay of control actions in Elastic systems. We identify recurring types of changes in the monitored metrics and requirements to properly carry out the estimation. Based on that, we develop a novel framework for the actuation delays estimation that utilizes change point detection techniques. We conduct several experiments with real-world systems to illustrate the feasibility and applicability of our framework.

Zahir Tari - One of the best experts on this subject based on the ideXlab platform.

  • optimizing the transition waste in coded Elastic Computing
    International Symposium on Information Theory, 2020
    Co-Authors: Hoang Dau, Ryan Gabrys, Yu-chih Huang, Chen Feng, Quang-hung Luu, Eidah J. Alzahrani, Zahir Tari
    Abstract:

    Motivated by recently available services in the cloud Computing industry, e.g., EC2 Spot or Azure Batch, where spare/low-priority virtual machines are offered at a fraction of the price of the on-demand instances but can be preempted on short notice, we investigate coded Computing solutions over Elastic resources, where the set of available machines may change in the middle of the computation. Our contributions are two-fold: We first propose an efficient method to minimize the transition waste, a newly introduced concept quantifying the total number of tasks that existing machines have to abandon or take on anew when a machine joins or leaves, for the cyclic Elastic task allocation scheme recently proposed in the literature (Yang et al. ISIT’19). We then proceed to generalize such a scheme and introduce new task allocation schemes based on finite geometry that achieve zero transition wastes as long as the number of active machines varies within a fixed range. The proposed solutions can be applied on top of existing coded Computing schemes tolerating stragglers.

  • ISIT - Optimizing the Transition Waste in Coded Elastic Computing
    2020 IEEE International Symposium on Information Theory (ISIT), 2020
    Co-Authors: Hoang Dau, Ryan Gabrys, Yu-chih Huang, Chen Feng, Quang-hung Luu, Eidah J. Alzahrani, Zahir Tari
    Abstract:

    Motivated by recently available services in the cloud Computing industry, e.g., EC2 Spot or Azure Batch, where spare/low-priority virtual machines are offered at a fraction of the price of the on-demand instances but can be preempted on short notice, we investigate coded Computing solutions over Elastic resources, where the set of available machines may change in the middle of the computation. Our contributions are two-fold: We first propose an efficient method to minimize the transition waste, a newly introduced concept quantifying the total number of tasks that existing machines have to abandon or take on anew when a machine joins or leaves, for the cyclic Elastic task allocation scheme recently proposed in the literature (Yang et al. ISIT’19). We then proceed to generalize such a scheme and introduce new task allocation schemes based on finite geometry that achieve zero transition wastes as long as the number of active machines varies within a fixed range. The proposed solutions can be applied on top of existing coded Computing schemes tolerating stragglers.

  • Optimizing the Transition Waste in Coded Elastic Computing.
    arXiv: Information Theory, 2019
    Co-Authors: Hoang Dau, Ryan Gabrys, Yu-chih Huang, Chen Feng, Quang-hung Luu, Eidah J. Alzahrani, Zahir Tari
    Abstract:

    Distributed Computing, in which a resource-intensive task is divided into subtasks and distributed among different machines, plays a key role in solving large-scale problems, e.g., machine learning for large datasets or massive computational problems arising in genomic research. Coded Computing is a recently emerging paradigm where redundancy for distributed Computing is introduced to alleviate the impact of slow machines, or stragglers, on the completion time. Motivated by recently available services in the cloud Computing industry, e.g., EC2 Spot or Azure Batch, where spare/low-priority virtual machines are offered at a fraction of the price of the on-demand instances but can be preempted in a short notice, we investigate coded Computing solutions over Elastic resources, where the set of available machines may change in the middle of the computation. Our contributions are two-fold: We first propose an efficient method to minimize the transition waste, a newly introduced concept quantifying the total number of tasks that existing machines have to abandon or take on anew when a machine joins or leaves, for the cyclic Elastic task allocation scheme recently proposed in the literature (Yang et al. ISIT'19). We then proceed to generalize such a scheme and introduce new task allocation schemes based on finite geometry that achieve zero transition wastes as long as the number of active machines varies within a fixed range. The proposed solutions can be applied on top of every existing coded Computing scheme tolerating stragglers.

Yongyao Jiang - One of the best experts on this subject based on the ideXlab platform.

  • A High Performance, Spatiotemporal Statistical Analysis System Based on a Spatiotemporal Cloud Platform
    ISPRS International Journal of Geo-Information, 2017
    Co-Authors: Baoxuan Jin, Weiwei Song, Kang Zhao, Xiaoyan Wei, Yongyao Jiang
    Abstract:

    With the increase in size and complexity of spatiotemporal data, traditional methods for performing statistical analysis are insufficient for meeting real-time requirements for mining information from Big Data, due to both data- and Computing-intensive factors. To solve the Big Data challenges in geostatistics and to support decision-making, a high performance, spatiotemporal statistical analysis system (Geostatistics-Hadoop) is proposed in this paper. The proposed system has several features: (1) Hadoop is enhanced to handle spatial data in a native format and execute a number of parallelized spatial analysis algorithms to solve practical geospatial analysis problems; (2) the Oozie-based workflow system is utilized to ease the operation and sharing of spatial analysis services; and (3) a private cloud platform based on Eucalyptus is leveraged to provide on-the-fly and Elastic Computing resources. Experimental results show that Geostatistics-Hadoop efficiently conducts rapid information mining and analysis of big spatiotemporal data sets, with the support of Elastic Computing resources from a cloud platform. The adoption of cloud Computing and the Hadoop cluster to parallelize statistical calculations significantly improves the performance of Big Data analyses.

Rong-rong Chen - One of the best experts on this subject based on the ideXlab platform.

  • Coded Elastic Computing on Machines with Heterogeneous Storage and Computation Speed
    IEEE Transactions on Communications, 2021
    Co-Authors: Nicholas Woolsey, Rong-rong Chen
    Abstract:

    We study the optimal design of heterogeneous Coded Elastic Computing (CEC) where machines have varying computation speeds and storage. CEC introduced by Yang et al. in 2018 is a framework that mitigates the impact of Elastic events, where machines can join and leave at arbitrary times. In CEC, data is distributed among machines using a Maximum Distance Separable (MDS) code such that subsets of machines can perform the desired computations. However, state-of-the-art CEC designs only operate on homogeneous networks where machines have the same speeds and storage. This may not be practical. In this work, based on an MDS storage assignment, we develop a novel computation assignment approach for heterogeneous CEC networks to minimize the overall computation time. We first consider the scenario where machines have heterogeneous Computing speeds but same storage and then the scenario where both heterogeneities are present. We propose a novel combinatorial optimization formulation and solve it exactly by decomposing it into a convex optimization problem to find the optimal computation load and a filling problem to find the exact computation assignment. A low-complexity filling algorithm is adapted and can be completed within a number of iterations equal to at most the number of available machines.

  • Heterogeneous Computation Assignments in Coded Elastic Computing.
    arXiv: Information Theory, 2020
    Co-Authors: Nicholas Woolsey, Rong-rong Chen
    Abstract:

    We study the optimal design of a heterogeneous coded Elastic Computing (CEC) network where machines have varying relative computation speeds. CEC introduced by Yang {\it et al.} is a framework which mitigates the impact of Elastic events, where machines join and leave the network. A set of data is distributed among storage constrained machines using a Maximum Distance Separable (MDS) code such that any subset of machines of a specific size can perform the desired computations. This design eliminates the need to re-distribute the data after each Elastic event. In this work, we develop a process for an arbitrary heterogeneous Computing network to minimize the overall computation time by defining an optimal computation load, or number of computations assigned to each machine. We then present an algorithm to define a specific computation assignment among the machines that makes use of the MDS code and meets the optimal computation load.

  • ISIT - Heterogeneous Computation Assignments in Coded Elastic Computing
    2020 IEEE International Symposium on Information Theory (ISIT), 2020
    Co-Authors: Nicholas Woolsey, Rong-rong Chen
    Abstract:

    We study the optimal design of a heterogeneous coded Elastic Computing (CEC) network where machines have varying relative computation speeds. CEC introduced by Yang et al. is a framework which mitigates the impact of Elastic events, where machines join and leave the network. A set of data is distributed among storage constrained machines using a Maximum Distance Separable (MDS) code such that any subset of machines of a specific size can perform the desired computations. This design eliminates the need to re-distribute the data after each Elastic event. In this work, we develop a process for an arbitrary heterogeneous Computing network to minimize the overall computation time by defining an optimal computation load, or number of computations assigned to each machine. We then present an algorithm to define a specific computation assignment among the machines that makes use of the MDS code and meets the optimal computation load.

Germán Moltó - One of the best experts on this subject based on the ideXlab platform.

  • Deployment of Elastic Virtual Hybrid Clusters Across Cloud Sites
    Journal of Grid Computing, 2021
    Co-Authors: Miguel Caballer, Marica Antonacci, Zdeněk Šustr, Michele Perniola, Germán Moltó
    Abstract:

    Virtual clusters are widely used Computing platforms than can be deployed in multiple cloud platforms. The ability to dynamically grow and shrink the number of nodes has paved the way for customised Elastic Computing both for High Performance Computing and High Throughput Computing workloads. However, Elasticity is typically restricted to a single cloud site, thus hindering the ability to provision computational resources from multiple geographically distributed cloud sites. To this aim, this paper introduces an architecture of open-source components that coherently deploy a virtual Elastic cluster across multiple cloud sites to perform large-scale Computing. These hybrid virtual Elastic clusters are automatically deployed and configured using an Infrastructure as Code (IaC) approach on a distributed hybrid testbed that spans different organizations, including on-premises and public clouds, supporting automated tunneling of communications across the cluster nodes with advanced VPN topologies. The results indicate that cluster-based Computing of embarrassingly parallel jobs can benefit from hybrid virtual clusters that aggregate Computing resources from multiple cloud back-ends and bring them together into a dedicated, albeit virtual network.

  • TOSCA-based orchestration of complex clusters at the IaaS level
    Journal of Physics: Conference Series, 2017
    Co-Authors: Miguel Caballer, Giuseppe Donvito, R. Rocha, Germán Moltó, Mathieu Velten
    Abstract:

    This paper describes the adoption and extension of the TOSCA standard by the INDIGO-DataCloud project for the definition and deployment of complex Computing clusters together with the required support in both OpenStack and OpenNebula, carried out in close collaboration with industry partners such as IBM. Two examples of these clusters are described in this paper, the definition of an Elastic Computing cluster to support the Galaxy bioinformatics application where the nodes are dynamically added and removed from the cluster to adapt to the workload, and the definition of an scalable Apache Mesos cluster for the execution of batch jobs and support for long-running services. The coupling of TOSCA with Ansible Roles to perform automated installation has resulted in the definition of high-level, deterministic templates to provision complex Computing clusters across different Cloud sites.