The Experts below are selected from a list of 585 Experts worldwide ranked by ideXlab platform
Pan Zeng - One of the best experts on this subject based on the ideXlab platform.
-
ICDCS - RoLo: A Rotated Logging Storage Architecture for Enterprise Data Centers
2010 IEEE 30th International Conference on Distributed Computing Systems, 2010Co-Authors: Yinliang Yue, Lei Tian, Hong Jiang, Fang Wang, Dan Feng, Quan Zhang, Pan ZengAbstract:We propose RoLo (Rotated Logging), a new Logging architecture for RAID10 systems for enhanced energy efficiency, performance and reliability. By spreading destaging I/O activities among short idle time slots and proactively reclaiming the stale Logging space, RoLo rotates loggers among a logical Logging space pool formed collectively from the free storage space available among mirrored disks. Therefore, without the extra dedicated log disks and the corresponding Centralized Logging, RoLo eliminates the additional hardware and energy costs, potential single point of failure and performance bottleneck. Furthermore, RoLo prolongs the lifecycle of the disks and improves the system’s energy efficiency by reducing the disk spin up/down frequency. We develop three flavors of RoLo, that is, RoLo-E/R/P, to emphasize energy efficiency, reliability, and performance respectively. Extensive trace-driven evaluations demonstrate the advantages of the three RoLo schemes over both a RAID10 system with Centralized Logging architecture and a typical RAID10 system.
Yinliang Yue - One of the best experts on this subject based on the ideXlab platform.
-
Rotated Logging Storage Architectures for Data Centers: Models and Optimizations
IEEE Transactions on Computers, 2016Co-Authors: Yinliang Yue, Lei Tian, Hong Jiang, Fang Wang, Dan FengAbstract:We propose Ro tated Lo gging (RoLo), a new Logging architecture for parallel disk-based mirrored storage systems for enhanced energy efficiency, which is one of the key concerns in modern data centers. By spreading destaging I/O activities among short idle time slots and proactively reclaiming the stale Logging space, RoLo rotates loggers among a logical Logging space pool formed collectively from the free storage space available among mirrored disks. We develop three flavors of RoLo, that is, RoLo-P/R/E, to emphasize performance, reliability, and energy efficiency respectively. Without the extra dedicated log disks and the corresponding Centralized destaging, RoLo eliminates the additional hardware and energy costs, potential single point of failure and performance bottleneck. Furthermore, RoLo-P/R/E, applied to specific scenes correctly, can prolong the lifecycle of the disks and improve the system's energy efficiency by reducing the disk spin up/down frequency. We propose RoLo-S to further alleviate the performance bottleneck and energy consumption caused by frequent disk head seeks in on-duty logger disks. We have implemented RoLo and RoLo-S on real disk systems. Extensive trace-driven evaluations demonstrate the advantages of the three RoLo schemes over both a RAID10 system with Centralized Logging architecture and a typical RAID10 system, and the advantages of RoLo-S over RoLo.
-
ICDCS - RoLo: A Rotated Logging Storage Architecture for Enterprise Data Centers
2010 IEEE 30th International Conference on Distributed Computing Systems, 2010Co-Authors: Yinliang Yue, Lei Tian, Hong Jiang, Fang Wang, Dan Feng, Quan Zhang, Pan ZengAbstract:We propose RoLo (Rotated Logging), a new Logging architecture for RAID10 systems for enhanced energy efficiency, performance and reliability. By spreading destaging I/O activities among short idle time slots and proactively reclaiming the stale Logging space, RoLo rotates loggers among a logical Logging space pool formed collectively from the free storage space available among mirrored disks. Therefore, without the extra dedicated log disks and the corresponding Centralized Logging, RoLo eliminates the additional hardware and energy costs, potential single point of failure and performance bottleneck. Furthermore, RoLo prolongs the lifecycle of the disks and improves the system’s energy efficiency by reducing the disk spin up/down frequency. We develop three flavors of RoLo, that is, RoLo-E/R/P, to emphasize energy efficiency, reliability, and performance respectively. Extensive trace-driven evaluations demonstrate the advantages of the three RoLo schemes over both a RAID10 system with Centralized Logging architecture and a typical RAID10 system.
Wolfgang Scherrer - One of the best experts on this subject based on the ideXlab platform.
-
complex log file synthesis for rapid sandbox benchmarking of security and computer network analysis tools
Information Systems, 2016Co-Authors: Markus Wurzenberger, Florian Skopik, Giuseppe Settanni, Wolfgang ScherrerAbstract:Today Information and Communications Technology (ICT) networks are a dominating component of our daily life. Centralized Logging allows keeping track of events occurring in ICT networks. Therefore a central log store is essential for timely detection of problems such as service quality degradations, performance issues or especially security-relevant cyber attacks. There exist various software tools such as security information and event management (SIEM) systems, log analysis tools and anomaly detection systems, which exploit log data to achieve this. While there are many products on the market, based on different approaches, the identification of the most efficient solution for a specific infrastructure, and the optimal configuration is still an unsolved problem. Today's general test environments do not sufficiently account for the specific properties of individual infrastructure setups. Thus, tests in these environments are usually not representative. However, testing on the real running productive systems exposes the network infrastructure to dangerous or unstable situations. The solution to this dilemma is the design and implementation of a highly realistic test environment, i.e. sandbox solution, that follows a different - novel - approach. The idea is to generate realistic network event sequence (NES) data that reflects the actual system behavior and which is then used to challenge network analysis software tools with varying configurations safely and realistically offline. In this paper we define a model, based on log line clustering and Markov chain simulation to create this synthetic log data. The presented model requires only a small set of real network data as an input to understand the complex real system behavior. Based on the input's characteristics highly realistic customer specified NES data is generated. To prove the applicability of the concept developed in this work, we conclude the paper with an illustrative example of evaluation and test of an existing anomaly detection system by using generated NES data. HighlightsGenerating log data that reflects realistic network behavior.Log data modeling, based on log line clustering and Markov chain simulation.Rate, analyze and improve software tools, which exploit log data.Detailed evaluation of the model and presentation of an illustrative application.Cornerstones to improve the selection, deployment and operation of IDSs.
Hyungsoo Jung - One of the best experts on this subject based on the ideXlab platform.
-
SIGMOD Conference - Border-Collie: A Wait-free, Read-optimal Algorithm for Database Logging on Multicore Hardware
Proceedings of the 2019 International Conference on Management of Data, 2019Co-Authors: Jongbin Kim, Hyuck Han, Sooyong Kang, Hyeongwon Jang, Seohui Son, Hyungsoo JungAbstract:Actions changing the state of databases are all logged with proper ordering being imposed. Database engines obeying this golden rule of Logging enforce total ordering on all events, and this poses challenges in addressing the scalability bottlenecks of database Logging on multicore hardware. We reexamined the problem of database Logging and realized that in any given log history, obtaining an upper bound on the size of a set that preserves the happen-before relation is the essence of the matter. Based on our understanding, we propose Border-Collie, a wait-free and read-optimal algorithm for database Logging that finds such an upper bound even with some worker threads often being idle. We show that (1) Border-Collie always finds the largest set of logged events satisfying the condition in a finite number of steps (i.e., wait-free), (2) the number of logged events to be read is also minimal (i.e., read-optimal), and (3) both properties hold even with threads being in intermittent work. Experimental results demonstrated that Border-Collie proves our claims under various workloads; Border-Collie outperforms the state-of-the-art Centralized Logging techniques (i.e., Eleda and ERMIA) by up to ~2X and exhibits almost the same throughput with much shorter commit latency than the state-of-the-art deCentralized Logging techniques (i.e., Silo and FOEDUS).
-
Scalable database Logging for multicores
Proceedings of the VLDB Endowment, 2017Co-Authors: Hyungsoo Jung, Hyuck Han, Sooyong KangAbstract:Modern databases, guaranteeing atomicity and durability, store transaction logs in a volatile, central log buffer and then flush the log buffer to non-volatile storage by the write-ahead Logging principle. Buffering logs in central log store has recently faced a severe multicore scalability problem, and log flushing has been challenged by synchronous I/O delay. We have designed and implemented a fast and scalable Logging method, Eleda, that can migrate a surge of transaction logs from volatile memory to stable storage without risking durable transaction atomicity. Our efficient implementation of Eleda is enabled by a highly concurrent data structure, Grasshopper, that eliminates a multicore scalability problem of Centralized Logging and enhances system utilization in the presence of synchronous I/O delay. We implemented Eleda and plugged it to WiredTiger and Shore-MT by replacing their log managers. Our evaluation showed that Eleda-based transaction systems improve performance up to 71 x, thus showing the applicability of Eleda.
Scherrerwolfgang - One of the best experts on this subject based on the ideXlab platform.
-
Complex log file synthesis for rapid sandbox-benchmarking of security- and computer network analysis tools
Information Systems, 2016Co-Authors: Wurzenbergermarkus, Skopikflorian, Settannigiuseppe, ScherrerwolfgangAbstract:Today Information and Communications Technology (ICT) networks are a dominating component of our daily life. Centralized Logging allows keeping track of events occurring in ICT networks. Therefore ...