The Experts below are selected from a list of 6063 Experts worldwide ranked by ideXlab platform

Che Renhai - One of the best experts on this subject based on the ideXlab platform.

  • Hardware/software codesign for performance and lifetime enhancement in NAND-flash-based embedded storage Systems
    The Hong Kong Polytechnic University, 2016
    Co-Authors: Che Renhai
    Abstract:

    PolyU Library Call No.: [THS] LG51 .H577P COMP 2016 Chenxvi, 121 pages :color illustrationsEmbedded Systems (e.g. smartphones) have become an integral part in peoples' daily life. In the last several years, most of efforts in improving embedded Systems have been focusing on enhancing the network performance and CPU speed. On the other hand, the performance of NAND-flash-based storage in embedded Systems is stagnant, and has become one of the major performance bottlenecks in embedded Systems. Even worse, the performance and lifetimes ofNAND flash memory are substantially degraded with the advent of multi-level cell and triple-level cell flash memory for increasing the capacity. In this thesis, we have addressed these issues from several aspects including the integration of the emerging hardware and the cross-layer software management for the optimization of the performance and lifetime. First, we focus on employing the self-healing NAND flash memory to improve the lifetime and performance. Researchers have recently discovered that heating can cause worn-out NAND flash cells to become reusable and greatly extend the lifetime of flash memory cells. However, the heating process consumes a substantial amount of power, and some fundamental changes are required for existing NAND flash management techniques. In particular, all existing wear-leveling techniques are based on the principle of evenly distributing writes and erases. For self-healing NAND flash, this may cause NAND flash cells to be worn out in a short period of time. Moreover, frequently healing these cells may drain the energy quickly in battery-driven mobile devices, which is defined as the concentrated heating problem. We propose a novel wear-leveling scheme called DHeating (Dispersed Heating) to address the problem. In DHeating, rather than evenly distributing writes and erases over a time period, write and erase operations are scheduled on a small number of flash memory cells at a time, so that these cells can be worn out and healed much earlier than other cells. In this way, we can avoid quick energy depletion caused by concentrated heating. In addition, the heating process takes several seconds and has become the new performance bottleneck. In order to address this issue, we propose a lazy heating repair scheme. The lazy heating repair scheme can ease the long time heating effect by delaying the heating operation and using the System idle time to repair. Furthermore, the flash memory's reliability becomes worse with the flash memory cells reaching the expected worn-out time. We propose an early heating strategy to solve the reliability problem. With the extended lifetime provided by self-healing, we can trade some lifetimes for reliability. The idea is to start the healing process earlier than the expected worn-out time. We evaluate our scheme based on an embedded platform. The experimental results show that the proposed scheme can effectively prolong the consecutive heating time interval, alleviate the long time heating effect, and enhance the reliability for self-healing flash memory.Second, we jointly optimize the NAND flash memory's lifetime and performance with the integration of NVMs. Novel NVMs (non-volatile memories), such as PCM (Phase Change Memory) and STT-RAM (Spin-TransferTorque Random Access Memory), can provide fast read/write operations. In this thesis, we propose a unified NVM/flash architecture to improve the I/O performance. A transparent scheme, vFlash (Virtualized Flash), is also proposed to manage the unified architecture. Within vFlash, inter-app and intra-app techniques are proposed to optimize the application performance by exploiting the historical locality and I/O access patterns of applications. Since vFlash is on the bottom of the I/O stack, the application features willbe lost. Therefore, we also propose a cross-layer technique to transfer the application information from the application layer to the vFlash layer. The proposed scheme is evaluated based on an Android platform, and the experimental results show that the proposed scheme can effectively improve the I/O performance of mobile devices. Third, we study the problem of performance and lifetime enhancement in the mobile virtualization environment. Mobile virtualization introduces extra layers in software stacks, which leads to performance degradation. Especially, each I/O operation has to pass through several software layers to reach the NAND-flash-based storage Systems. This thesis targets at optimizing I/O for mobile virtualization, since I/O becomes one of major performance bottlenecks that seriously affects the performance of mobile devices. Among all the I/O operations, a large percentage is updating Metadata. Frequent updating Metadata not only degrades overall I/O performance but also severely reduces flash memory lifetime. In this thesis, we propose a novel I/O optimization technique to identify the Metadata of a guest File System which is stored in a VM (Virtual Machine) image File and frequently updated. Then, these Metadata are stored in a small additional NVM (Non-Volatile Memory) which is faster and more endurable to greatly improve flash memory's performance and lifetime. To the best of our knowledge, this is the first work to identify the File System Metadata from regular data in a guest OS VM image File under mobile virtualization. The proposed scheme is evaluated on a real hardware embedded platform. The experimental results show that the proposed techniques can improve write performance to 45.21% in mobile devices with virtualization.Department of ComputingPh.D., Department of Computing, The Hong Kong Polytechnic University, 2016Doctorat

  • Hardware/software codesign for performance and lifetime enhancement in NAND-flash-based embedded storage Systems
    The Hong Kong Polytechnic University, 2016
    Co-Authors: Che Renhai
    Abstract:

    PolyU Library Call No.: [THS] LG51 .H577P COMP 2016 Chenxvi, 121 pages :color illustrationsEmbedded Systems (e.g. smartphones) have become an integral part in peoples' daily life. In the last several years, most of efforts in improving embedded Systems have been focusing on enhancing the network performance and CPU speed. On the other hand, the performance of NAND-flash-based storage in embedded Systems is stagnant, and has become one of the major performance bottlenecks in embedded Systems. Even worse, the performance and lifetimes ofNAND flash memory are substantially degraded with the advent of multi-level cell and triple-level cell flash memory for increasing the capacity. In this thesis, we have addressed these issues from several aspects including the integration of the emerging hardware and the cross-layer software management for the optimization of the performance and lifetime. First, we focus on employing the self-healing NAND flash memory to improve the lifetime and performance. Researchers have recently discovered that heating can cause worn-out NAND flash cells to become reusable and greatly extend the lifetime of flash memory cells. However, the heating process consumes a substantial amount of power, and some fundamental changes are required for existing NAND flash management techniques. In particular, all existing wear-leveling techniques are based on the principle of evenly distributing writes and erases. For self-healing NAND flash, this may cause NAND flash cells to be worn out in a short period of time. Moreover, frequently healing these cells may drain the energy quickly in battery-driven mobile devices, which is defined as the concentrated heating problem. We propose a novel wear-leveling scheme called DHeating (Dispersed Heating) to address the problem. In DHeating, rather than evenly distributing writes and erases over a time period, write and erase operations are scheduled on a small number of flash memory cells at a time, so that these cells can be worn out and healed much earlier than other cells. In this way, we can avoid quick energy depletion caused by concentrated heating. In addition, the heating process takes several seconds and has become the new performance bottleneck. In order to address this issue, we propose a lazy heating repair scheme. The lazy heating repair scheme can ease the long time heating effect by delaying the heating operation and using the System idle time to repair. Furthermore, the flash memory's reliability becomes worse with the flash memory cells reaching the expected worn-out time. We propose an early heating strategy to solve the reliability problem. With the extended lifetime provided by self-healing, we can trade some lifetimes for reliability. The idea is to start the healing process earlier than the expected worn-out time. We evaluate our scheme based on an embedded platform. The experimental results show that the proposed scheme can effectively prolong the consecutive heating time interval, alleviate the long time heating effect, and enhance the reliability for self-healing flash memory.Second, we jointly optimize the NAND flash memory's lifetime and performance with the integration of NVMs. Novel NVMs (non-volatile memories), such as PCM (Phase Change Memory) and STT-RAM (Spin-TransferTorque Random Access Memory), can provide fast read/write operations. In this thesis, we propose a unified NVM/flash architecture to improve the I/O performance. A transparent scheme, vFlash (Virtualized Flash), is also proposed to manage the unified architecture. Within vFlash, inter-app and intra-app techniques are proposed to optimize the application performance by exploiting the historical locality and I/O access patterns of applications. Since vFlash is on the bottom of the I/O stack, the application features willbe lost. Therefore, we also propose a cross-layer technique to transfer the application information from the application layer to the vFlash layer. The proposed scheme is evaluated based on an Android platform, and the experimental results show that the proposed scheme can effectively improve the I/O performance of mobile devices. Third, we study the problem of performance and lifetime enhancement in the mobile virtualization environment. Mobile virtualization introduces extra layers in software stacks, which leads to performance degradation. Especially, each I/O operation has to pass through several software layers to reach the NAND-flash-based storage Systems. This thesis targets at optimizing I/O for mobile virtualization, since I/O becomes one of major performance bottlenecks that seriously affects the performance of mobile devices. Among all the I/O operations, a large percentage is updating Metadata. Frequent updating Metadata not only degrades overall I/O performance but also severely reduces flash memory lifetime. In this thesis, we propose a novel I/O optimization technique to identify the Metadata of a guest File System which is stored in a VM (Virtual Machine) image File and frequently updated. Then, these Metadata are stored in a small additional NVM (Non-Volatile Memory) which is faster and more endurable to greatly improve flash memory's performance and lifetime. To the best of our knowledge, this is the first work to identify the File System Metadata from regular data in a guest OS VM image File under mobile virtualization. The proposed scheme is evaluated on a real hardware embedded platform. The experimental results show that the proposed techniques can improve write performance to 45.21% in mobile devices with virtualization.Department of ComputingPh.D., Department of Computing, The Hong Kong Polytechnic University, 2016Doctoratepublished_fina

Zhang Jiangang - One of the best experts on this subject based on the ideXlab platform.

Felix C Freiling - One of the best experts on this subject based on the ideXlab platform.

  • forensic application fingerprinting based on File System Metadata
    2013 Seventh International Conference on IT Security Incident Management and IT Forensics, 2013
    Co-Authors: Sven Kalber, Andreas Dewald, Felix C Freiling
    Abstract:

    While much work has been invested in tools for aquisition and extraction of digital evidence, there are only few tools that allow for automatic event reconstruction. In this paper, we present a generic approach for forensic event reconstruction based on digital evidence from File Systems. Our approach applies the idea of fingerprinting to changes made by applications in File System Metadata. We present a System with which it is possible to automatically compute File System fingerprints of individual actions. Using NTFS timestamps as an example, we show that with our approach it is possible to automatically reconstruct actions performed by different applications even if the set of Files accessed by those actions overlap.

Mikael Ronstrom - One of the best experts on this subject based on the ideXlab platform.

  • hopsfs scaling hierarchical File System Metadata using newsql databases
    File and Storage Technologies, 2017
    Co-Authors: Salman Niazi, Mahmoud Ismail, Seif Haridi, Jim Dowling, Steffen Grohsschmiedt, Mikael Ronstrom
    Abstract:

    Recent improvements in both the performance and scalability of shared-nothing, transactional, in-memory NewSQL databases have reopened the research question of whether distributed Metadata for hierarchical File Systems can be managed using commodity databases. In this paper, we introduce HopsFS, a next generation distribution of the Hadoop Distributed File System (HDFS) that replaces HDFS' single node in-memory Metadata service, with a distributed Metadata service built on a NewSQL database. By removing the Metadata bottleneck, HopsFS enables an order of magnitude larger and higher throughput clusters compared to HDFS. Metadata capacity has been increased to at least 37 times HDFS' capacity, and in experiments based on a workload trace from Spotify, we show that HopsFS supports 16 to 37 times the throughput of Apache HDFS. HopsFS also has lower latency for many concurrent clients, and no downtime during failover. Finally, as Metadata is now stored in a commodity database, it can be safely extended and easily exported to external Systems for online analysis and free-text search.

Qing Zheng - One of the best experts on this subject based on the ideXlab platform.

  • slimdb a space efficient key value storage engine for semi sorted data
    Very Large Data Bases, 2017
    Co-Authors: Qing Zheng, Joy Arulraj, Garth A Gibson
    Abstract:

    Modern key-value stores often use write-optimized indexes and compact in-memory indexes to speed up read and write performance. One popular write-optimized index is the Log-structured merge-tree (LSM-tree) which provides indexed access to write-intensive data. It has been increasingly used as a storage backbone for many services, including File System Metadata management, graph processing engines, and machine learning feature storage engines. Existing LSM-tree implementations often exhibit high write amplifications caused by compaction, and lack optimizations to maximize read performance on solid-state disks. The goal of this paper is to explore techniques that leverage common workload characteristics shared by many Systems using key-value stores to reduce the read/write amplification overhead typically associated with general-purpose LSM-tree implementations. Our experiments show that by applying these design techniques, our new implementation of a key-value store, SlimDB, can be two to three times faster, use less memory to cache Metadata indices, and show lower tail latency in read operations compared to popular LSM-tree implementations such as LevelDB and RocksDB.

  • BatchFS: Scaling the File System Control Plane with Client-Funded Metadata Servers
    2016
    Co-Authors: Kai Ren, Qing Zheng, Garth Gibson
    Abstract:

    Email: {zhengq, kair, garth} @ cs.cmu.edu Abstract—Parallel File Systems are often characterized by a layered architecture that decouples Metadata management from I/O operations, allowing File Systems to facilitate fast concurrent access to File contents. However, Metadata intensive workloads are still likely to bottleneck at the File System control plane due to namespace synchronization, which taxes application performance through lock contention on directories, transaction serialization, and RPC overheads. In this paper, we propose a client-driven File System Metadata architecture, BatchFS, that is optimized for non-interactive, or batch, workloads. To avoid Metadata bottlenecks, BatchFS features a relaxed consistency model marked by lazy namespace synchronization and optimistic Metadata verification. Capable of executing namespace operations on client-provisioned resources without contacting any Metadata server, BatchFS clients are able to delay namespace synchronization until synchroniza-tion is really needed. Our goal in this vision paper is to handle these delayed operations securely and efficiently with Metadata verification and bulk insertion. Preliminary experiments demon-strate that our client-funded Metadata architecture outperforms a traditional synchronous File System by orders of magnitude. I

  • BatchFS: Scaling the File System Control Plane with Client-Funded Metadata Servers
    2015
    Co-Authors: Qing Zheng, Kai Ren, Garth Gibson
    Abstract:

    Abstract—Parallel File Systems are often characterized by a layered architecture that decouples Metadata management from I/O operations, allowing File Systems to facilitate fast concurrent access to File contents. However, Metadata intensive workloads are still likely to bottleneck at the File System control plane due to namespace synchronization, which taxes application performance through lock contention on directories, transaction serialization, and RPC overheads. In this paper, we propose a client-driven File System Metadata architecture, BatchFS, that is optimized for non-interactive, or batch, workloads. To avoid Metadata bottlenecks, BatchFS features a relaxed consistency model marked by lazy namespace synchronization and optimistic Metadata verification. Capable of executing namespace operations on client-provisioned resources without contacting any Metadata server, BatchFS clients are able to delay namespace synchronization until synchroniza-tion is really needed. Our goal in this vision paper is to handle these delayed operations securely and efficiently with Metadata verification and bulk insertion. Preliminary experiments demon-strate that our client-funded Metadata architecture outperforms a traditional synchronous File System by orders of magnitude. I

  • indexfs scaling File System Metadata performance with stateless caching and bulk insertion
    IEEE International Conference on High Performance Computing Data and Analytics, 2014
    Co-Authors: Kai Ren, Qing Zheng, Swapnil Patil, Garth A Gibson
    Abstract:

    The growing size of modern storage Systems is expected to exceed billions of objects, making Metadata scalability critical to overall performance. Many existing distributed File Systems only focus on providing highly parallel fast access to File data, and lack a scalable Metadata service. In this paper, we introduce a middleware design called IndexFS that adds support to existing File Systems such as PVFS, Lustre, and HDFS for scalable high-performance operations on Metadata and small Files. IndexFS uses a table-based architecture that incrementally partitions the namespace on a per-directory basis, preserving server and disk locality for small directories. An optimized log-structured layout is used to store Metadata and small Files efficiently. We also propose two client-based storm-free caching techniques: bulk namespace insertion for creation intensive workloads such as N-N checkpointing; and stateless consistent Metadata caching for hot spot mitigation. By combining these techniques, we have demonstrated IndexFS scaled to 128 Metadata servers. Experiments show our out-of-core Metadata throughput out-performing existing solutions such as PVFS, Lustre, and HDFS by 50% to two orders of magnitude.