The Experts below are selected from a list of 312 Experts worldwide ranked by ideXlab platform

Yarsun Hsu - One of the best experts on this subject based on the ideXlab platform.

  • HPCC - DPCT: distributed parity cache table for redundant parallel File System
    High Performance Computing and Communications, 2006
    Co-Authors: Sheng-kai Hung, Yarsun Hsu
    Abstract:

    Using parity information to protect data from loss in a parallel File System is a straightforward and cost-effective method. However, the “small-write” phenomenon can lead to poor write performance. This is still true in the distributed paradigm even when File System cache is used. The Local File System knows nothing about a stripe and thus can not benefit from the related blocks of a stripe. We propose a distributed parity cache table (DPCT) which knows the related blocks of a stripe and can use them to improve the performance of parity calculation and parity updating. This high level cache can benefit from previous reads and can aggregate small writes to improve the overall performance. We implement this mechanism in our reliable parallel File System (RPFS). The experimental results show that both read and write performance can be improved with DPCT support. The improvement comes from the fact that we can reduce the number of disk accesses by DPCT. This matches our quantitative analysis which shows that the number of disk accesses can be reduced from N to N(1–H), where N is the number of I/O nodes and H is the DPCT hit ratio.

  • DPCT : Distributed parity cache table for redundant parallel File System
    Lecture Notes in Computer Science, 2006
    Co-Authors: Sheng-kai Hung, Yarsun Hsu
    Abstract:

    Using parity information to protect data from loss in a parallel File System is a straightforward and cost-effective method. However, the small-write phenomenon can lead to poor write performance. This is still true in the distributed paradigm even when File System cache is used. The Local File System knows nothing about a stripe and thus can not benefit from the related blocks of a stripe. We propose a distributed parity cache table (DPCT) which knows the related blocks of a stripe and can use them to improve the performance of parity calculation and parity updating. This high level cache can benefit from previous reads and can aggregate small writes to improve the overall performance. We implement this mechanism in our reliable parallel File System (RPFS). The experimental results show that both read and write performance can be improved with DPCT support. The improvement comes from the fact that we can reduce the number of disk accesses by DPCT. This matches our quantitative analysis which shows that the number of disk accesses can be reduced from N to N(1 - H), where N is the number of I/O nodes and H is the DPCT hit ratio.

Sheng-kai Hung - One of the best experts on this subject based on the ideXlab platform.

  • Asia-Pacific Computer Systems Architecture Conference - Striping cache: a global cache for striped network File System
    Advances in Computer Systems Architecture, 2006
    Co-Authors: Sheng-kai Hung
    Abstract:

    Using caching to enhance performance has been widely used in the computer System. This is still true in the distributed paradigm. In the distributed environment, caches are distributed in each of the nodes and can be collected to form a global cache. However, the overall performance cannot benefit from the global cache without efficient cooperation of these global resources. The Local File System in each node knows nothing about a stripe and thus can not benefit from the related blocks of a stripe. We propose a striping cache (SC) which knows the related blocks of a stripe and can use them to improve the performance of a striped network File System. This high level cache can benefit from previous reads and can aggregate small writes to improve the overall performance. We implement this mechanism in our reliable parallel File System (RPFS). The experimental results show that both read and write performance can be improved with SC support. The improvement comes from the fact that we can reduce the number of disk accesses by employing SC.

  • HPCC - DPCT: distributed parity cache table for redundant parallel File System
    High Performance Computing and Communications, 2006
    Co-Authors: Sheng-kai Hung, Yarsun Hsu
    Abstract:

    Using parity information to protect data from loss in a parallel File System is a straightforward and cost-effective method. However, the “small-write” phenomenon can lead to poor write performance. This is still true in the distributed paradigm even when File System cache is used. The Local File System knows nothing about a stripe and thus can not benefit from the related blocks of a stripe. We propose a distributed parity cache table (DPCT) which knows the related blocks of a stripe and can use them to improve the performance of parity calculation and parity updating. This high level cache can benefit from previous reads and can aggregate small writes to improve the overall performance. We implement this mechanism in our reliable parallel File System (RPFS). The experimental results show that both read and write performance can be improved with DPCT support. The improvement comes from the fact that we can reduce the number of disk accesses by DPCT. This matches our quantitative analysis which shows that the number of disk accesses can be reduced from N to N(1–H), where N is the number of I/O nodes and H is the DPCT hit ratio.

  • DPCT : Distributed parity cache table for redundant parallel File System
    Lecture Notes in Computer Science, 2006
    Co-Authors: Sheng-kai Hung, Yarsun Hsu
    Abstract:

    Using parity information to protect data from loss in a parallel File System is a straightforward and cost-effective method. However, the small-write phenomenon can lead to poor write performance. This is still true in the distributed paradigm even when File System cache is used. The Local File System knows nothing about a stripe and thus can not benefit from the related blocks of a stripe. We propose a distributed parity cache table (DPCT) which knows the related blocks of a stripe and can use them to improve the performance of parity calculation and parity updating. This high level cache can benefit from previous reads and can aggregate small writes to improve the overall performance. We implement this mechanism in our reliable parallel File System (RPFS). The experimental results show that both read and write performance can be improved with DPCT support. The improvement comes from the fact that we can reduce the number of disk accesses by DPCT. This matches our quantitative analysis which shows that the number of disk accesses can be reduced from N to N(1 - H), where N is the number of I/O nodes and H is the DPCT hit ratio.

Garth A Gibson - One of the best experts on this subject based on the ideXlab platform.

  • tablefs enhancing metadata efficiency in the Local File System
    USENIX Annual Technical Conference, 2013
    Co-Authors: Kai Ren, Garth A Gibson
    Abstract:

    File Systems that manage magnetic disks have long recognized the importance of sequential allocation and large transfer sizes for File data. Fast random access has dominated metadata lookup data structures with increasing use of B-trees on-disk. Yet our experiments with workloads dominated by metadata and small File access indicate that even sophisticated Local disk File Systems like Ext4, XFS and Btrfs leave a lot of opportunity for performance improvement in workloads dominated by metadata and small Files. In this paper we present a stacked File System, TABLEFS, which uses another Local File System as an object store. TABLEFS organizes all metadata into a single sparse table backed on disk using a Log-Structured Merge (LSM) tree, LevelDB in our experiments. By stacking, TABLEFS asks only for efficient large File allocation and access from the underlying Local File System. By using an LSM tree, TABLEFS ensures metadata is written to disk in large, non-overwrite, sorted and indexed logs. Even an inefficient FUSE based user level implementation of TABLEFS can perform comparably to Ext4, XFS and Btrfs on data-intensive benchmarks, and can outperform them by 50% to as much as 1000% for metadata-intensive workloads. Such promising performance results from TABLEFS suggest that Local disk File Systems can be significantly improved by more aggressive aggregation and batching of metadata updates.

  • tablefs embedding a nosql database inside the Local File System
    Asia-Pacific Magnetic Recording Conference, 2012
    Co-Authors: Kai Ren, Garth A Gibson
    Abstract:

    Conventional File Systems are optimzed for large File transfers instead of workloads that are dominated by metadata and small File accesses. This paper examines using techniques adopted from NoSQL databases to manage File System metadata and small Files, which feature high rates of change and efficient out-of-core data representation. A FUSE File System prototype was built by storing File System metadata and small Files into a modern key-value store: LevelDB. We demonstrate that such techniques can improve the performance of modern Local File Systems in Linux for workloads dominated by metadata and tiny Files.

Stian Soiland-reyes - One of the best experts on this subject based on the ideXlab platform.

  • Application and Packaging Pointer (app) URI scheme
    2018
    Co-Authors: Marcos Cáceres, Stian Soiland-reyes
    Abstract:

    This specification proposes the Application and Packaging Pointer URI scheme "app". app URIs can be used to consume or reference hypermedia resources bundled inside a File archive or an application package, as well as to resolve URIs for archive resources within a programmatic framework. This URI scheme provides mechanisms to generate a unique base URI to represent the root of the archive, so that relative URI references in a bundled resource can be resolved within the archive without having to extract the archive content on the Local File System. An app URI can be used for purposes of isolation (e.g. when consuming multiple archives), security constraints (avoiding "climb out" from the archive), or for externally identiyfing sub-resources referenced by hypermedia formats.

  • The Archive and Packaging Pointer (app) URI scheme
    2018
    Co-Authors: Marcos Cáceres, Stian Soiland-reyes
    Abstract:

    This Internet-Draft proposes the Archive and Packaging Pointer URI scheme "app". app URIs can be used to consume or reference hypermedia resources bundled inside a File archive or an application package, as well as to resolve URIs for archive resources within a programmatic framework. This URI scheme provides mechanisms to generate a unique base URI to represent the root of the archive, so that relative URI references in a bundled resource can be resolved within the archive without having to extract the archive content on the Local File System. An app URI can be used for purposes of isolation (e.g. when consuming multiple archives), security constraints (avoiding "climb out" from the archive), or for externally identiyfing sub-resources referenced by hypermedia formats.

Haiying Shen - One of the best experts on this subject based on the ideXlab platform.

  • CLOUD - Performance Measurement on Scale-Up and Scale-Out Hadoop with Remote and Local File Systems
    2016 IEEE 9th International Conference on Cloud Computing (CLOUD), 2016
    Co-Authors: Zhuozhao Li, Haiying Shen
    Abstract:

    MapReduce is a popular computing model for parallel data processing on large-scale datasets, which can vary from gigabytes to terabytes and petabytes. Though Hadoop MapReduce normally uses Hadoop Distributed File System (HDFS) Local File System, it can be configured to use a remote File System. Then, an interesting question is raised: for a given application, which is the best running platform among the different combinations of scale-up and scale-out Hadoop with remote and Local File Systems. However, there has been no previous research on how different types of applications (e.g., CPU-intensive, data-intensive) with different characteristics (e.g., input data size) can benefit from the different platforms. Thus, in this paper, we conduct a comprehensive performance measurement of different applications on scale-up and scaleout clusters configured with HDFS and a remote File System (i.e., OFS), respectively. We identify and study how different job characteristics (e.g., input data size, the number of File reads/writes, and the amount of computations) affect the performance of different applications on the different platforms. This study is expected to provide a guidance for users to choose the best platform to run different applications with different characteristics in the environment that provides both remote and Local storage, such as HPC cluster.