The Experts below are selected from a list of 96 Experts worldwide ranked by ideXlab platform

Le Goff Fabrice - One of the best experts on this subject based on the ideXlab platform.

  • Multi-threaded checksum computation for ATLAS high-performance storage software
    2019
    Co-Authors: Le Goff Fabrice, Avolio Giuseppe
    Abstract:

    ATLAS is one of the generic-purpose experiments observing hadron collisions at the LHC at CERN. Its trigger and data acquisition system (TDAQ) is responsible for selecting and transporting interesting physics events from the detector to permanent storage where the data are used for physics analysis. The transient storage of ATLAS TDAQ is the last component of the online system in the data flow. It records selected events at several GB/s to non-volatile storage before transfer to offline permanent storage where physics analysis are undertaken. The transient storage is a distributed system consisting of high-performance direct-attached storage servers accounting for 480 hard drives. A distributed multi-threaded C++ application operates the hardware. The transient storage is also responsible for computing a checksum for the data, which is used to ensure data integrity up to the physics analysis. Reliability and efficiency of this system are critical for the operations of TDAQ as well as the validity of the analysis. This paper presents the existing multi-threading strategy of the software and how the available hardware resources are used. We then introduce how multi-threaded checksum computation was introduced to increase significantly the maximum throughput of the system. We discuss the key concepts of the implementation with a focus on the importance of overhead minimization. Finally the paper reports on the tests done on the production system to demonstrate the validity of the implementation and measurements of the performance improvement in the view of future LHC and ATLAS upgrades

  • Automated load balancing in the ATLAS high-performance storage software
    'Springer Science and Business Media LLC', 2017
    Co-Authors: Le Goff Fabrice, Vandelli Waine
    Abstract:

    The ATLAS experiment collects proton-proton collision events delivered by the LHC accelerator at CERN. The ATLAS Trigger and Data Acquisition (TDAQ) system selects, transports and eventually records event data from the detector at several gigabytes per second. The data are recorded on transient storage before being delivered to permanent storage. The transient storage consists of high-performance direct-attached storage servers accounting for about 500 hard drives. The transient storage operates dedicated software in the form of a distributed multi-threaded application. The workload includes both CPU-demanding and IO-oriented tasks. This paper presents the original application threading model for this particular workload, discussing the load-sharing strategy among the available CPU cores. The limitations of this strategy were reached in 2016 due to changes in the trigger configuration involving a new data distribution pattern. We then describe a novel data-driven load-sharing strategy, designed to automatically adapt to evolving operational conditions, as driven by the detector configuration or the physics research goals. The improved efficiency and adaptability of the solution were measured with dedicated studies on both test and production systems. This paper reports on the results of those tests which demonstrate the capability of operating in a large variety of conditions with minimal user intervention.ATLAS [1] is one of the general purpose detectors observing proton-proton collisions provided by the LHC [2] at CERN. The ATLAS Trigger and Data Acquisition (TDAQ) system [3] is responsible for conveying the event data from the detector up to a permanent mass-storage system provided by CERN. This work focuses on the Data Logger system which lies at the end of the data flow path in the TDAQ system. The Data Logger is a transient storage system recording the selected event data on hard drives before transferring them to permanent storage where they are available for offline analysis

  • Automated load balancing in the ATLAS high-performance storage software
    2017
    Co-Authors: Le Goff Fabrice, Vandelli Waine
    Abstract:

    The ATLAS experiment collects proton-proton collision events delivered by the LHC accelerator at CERN. The ATLAS Trigger and Data Acquisition (TDAQ) system selects, transports and eventually records event data from the detector at several gigabytes per second. The data are recorded on transient storage before being delivered to permanent storage. The transient storage consists of high-performance direct-attached storage servers accounting for about 500 hard drives. The transient storage operates dedicated software in the form of a distributed multi-threaded application. The workload includes both CPU-demanding and IO-oriented tasks. This paper presents the original application threading model for this particular workload, discussing the load-sharing strategy among the available CPU cores. The limitations of this strategy were reached in 2016 due to changes in the trigger configuration involving a new data distribution pattern. We then describe a novel data-driven load-sharing strategy, designed to automatically adapt to evolving operational conditions, as driven by the detector configuration or the physics research goals. The improved efficiency and adaptability of the solution were measured with dedicated studies on both test and production systems. This paper reports on the results of those tests which demonstrate the capability of operating in a large variety of conditions with minimal user intervention

Luis Pabon - One of the best experts on this subject based on the ideXlab platform.

  • mercury host side flash caching for the data center
    IEEE Conference on Mass Storage Systems and Technologies, 2012
    Co-Authors: Steve Byan, James Lentini, Anshul Madan, Luis Pabon
    Abstract:

    The adoption of flash memory in high volume consumer products such as cell phones, tablet computers, digital cameras, and portable music players has driven down flash costs and increased flash quality. This trend is pushing flash memory into new applications, including enterprise computing. In enterprise data centers, servers containing flash-based SolidState Drives (SSDs) are becoming common. However, data center architects prefer to deploy shared storage over direct-attached storage (DAS). Shared storage offers superior manageability, availability, and scalability compared to DAS. For these reasons, system designers want to reap the benefits of direct attached flash memory without decreasing the value of shared storage systems. Our solution is Mercury, a persistent, write-through host-side cache for flash memory. By designing Mercury as a hypervisor cache, we simplify integration and deployment into host environments. This paper presents our experience building a host-side flash cache, an architectural analysis of possible cache attachment points, and a performance evaluation using enterprise workloads. Our results show a 26% improvement in the bandwidth observed by the Jetstress benchmark and a 500% improvement in the I/O rate of an enterprise workload.

Vandelli Waine - One of the best experts on this subject based on the ideXlab platform.

  • Automated load balancing in the ATLAS high-performance storage software
    'Springer Science and Business Media LLC', 2017
    Co-Authors: Le Goff Fabrice, Vandelli Waine
    Abstract:

    The ATLAS experiment collects proton-proton collision events delivered by the LHC accelerator at CERN. The ATLAS Trigger and Data Acquisition (TDAQ) system selects, transports and eventually records event data from the detector at several gigabytes per second. The data are recorded on transient storage before being delivered to permanent storage. The transient storage consists of high-performance direct-attached storage servers accounting for about 500 hard drives. The transient storage operates dedicated software in the form of a distributed multi-threaded application. The workload includes both CPU-demanding and IO-oriented tasks. This paper presents the original application threading model for this particular workload, discussing the load-sharing strategy among the available CPU cores. The limitations of this strategy were reached in 2016 due to changes in the trigger configuration involving a new data distribution pattern. We then describe a novel data-driven load-sharing strategy, designed to automatically adapt to evolving operational conditions, as driven by the detector configuration or the physics research goals. The improved efficiency and adaptability of the solution were measured with dedicated studies on both test and production systems. This paper reports on the results of those tests which demonstrate the capability of operating in a large variety of conditions with minimal user intervention.ATLAS [1] is one of the general purpose detectors observing proton-proton collisions provided by the LHC [2] at CERN. The ATLAS Trigger and Data Acquisition (TDAQ) system [3] is responsible for conveying the event data from the detector up to a permanent mass-storage system provided by CERN. This work focuses on the Data Logger system which lies at the end of the data flow path in the TDAQ system. The Data Logger is a transient storage system recording the selected event data on hard drives before transferring them to permanent storage where they are available for offline analysis

  • Automated load balancing in the ATLAS high-performance storage software
    2017
    Co-Authors: Le Goff Fabrice, Vandelli Waine
    Abstract:

    The ATLAS experiment collects proton-proton collision events delivered by the LHC accelerator at CERN. The ATLAS Trigger and Data Acquisition (TDAQ) system selects, transports and eventually records event data from the detector at several gigabytes per second. The data are recorded on transient storage before being delivered to permanent storage. The transient storage consists of high-performance direct-attached storage servers accounting for about 500 hard drives. The transient storage operates dedicated software in the form of a distributed multi-threaded application. The workload includes both CPU-demanding and IO-oriented tasks. This paper presents the original application threading model for this particular workload, discussing the load-sharing strategy among the available CPU cores. The limitations of this strategy were reached in 2016 due to changes in the trigger configuration involving a new data distribution pattern. We then describe a novel data-driven load-sharing strategy, designed to automatically adapt to evolving operational conditions, as driven by the detector configuration or the physics research goals. The improved efficiency and adaptability of the solution were measured with dedicated studies on both test and production systems. This paper reports on the results of those tests which demonstrate the capability of operating in a large variety of conditions with minimal user intervention

Jerry Hamann - One of the best experts on this subject based on the ideXlab platform.

  • microsoft exchange implementation on a distributed storage area network
    International Journal of Computers and Applications, 2008
    Co-Authors: Nagapramod Mandagere, Jerry Hamann, Suresh Muknahallipatna, Howard Johnson
    Abstract:

    AbstractThe ever-increasing demand for fast and reliable data storage has caused dramatic changes in the storage model. direct attached storage (DAS) architecture, also known as server-centric architecture, is being replaced by the storage area network (SAN), leading to storage-centric architecture. In SAN storage devices exist completely independent of any computer (servers in particular). Multiple servers can access the same storage device without involvement of other servers. Recently, companies have been able to implement storage integration/sharing, remote backup/restoral, near-real-time disk mirroring/replication, and the like, using a new SAN architecture known as distributed storage area network (DSAN). The DSAN is multiple SANs interconnected using existing Internet Protocol (IP) networks to distribute storage over wide geographical areas. Even though DSANs provide additional capabilities to a SAN, the end-to-end latency (ETEL) due to large physical cable lengths (IP networks) imposes limitations...

  • the effect of end to end latency in a distributed storage area network on microsoft exchange server 2003 performance part i
    Local Computer Networks, 2004
    Co-Authors: Suresh Muknahallipatna, Nagapramod Mandagere, P. Patil, Jerry Hamann
    Abstract:

    The ever increasing demand for fast and reliable data storage has caused dramatic changes in the storage model. The direct attached storage (DAS) model is being replaced by storage area networks (SANs). SANs provide the capability to distribute the storage across multiple servers leading to decentralized and externalized storage. Recently, the accessibility of SANs to small companies has been improved by using existing IP networks to distribute storage over long geographical distances, leading to the distributed storage area network (DSAN). Although DSANs have provided additional capabilities to SANs, they have created a new set of problems for enterprise class applications, like Microsoft Exchange Server 2003, with increasing end to end latency (ETEL), due to large physical cable lengths. We present an attempt to analyze the effect of ETEL in DSAN on the performance of Microsoft Exchange Server 2003 by monitoring the Exchange behavior using LoadSim 2003 benchmark.

Andrew S. Tanenbaum - One of the best experts on this subject based on the ideXlab platform.

  • File-Level, Host-Side Flash Caching with Loris
    2014
    Co-Authors: David C. Van Moolenbroek, Shara Santhanam, Andrew S. Tanenbaum
    Abstract:

    Abstract—As enterprises shift from using direct-attached storage to network-based storage for housing primary data, flashbased, host-side caching has gained momentum as the primary latency reduction technique. In this paper, we make the case for integration of flash caching algorithms at the file level, as opposed to the conventional block-level integration. In doing so, we will show how our extensions to Loris, a reliable, file-oriented storage stack, transform it into a framework for designing layout-independent, file-level caching systems. Using our Loris prototype, we demonstrate the effectiveness of Loris-based, filelevel flash caching systems over their block-level counterparts, and investigate the effect of various write and allocation policies on the overall performance