The Experts below are selected from a list of 1383 Experts worldwide ranked by ideXlab platform

Peter Sobe - One of the best experts on this subject based on the ideXlab platform.

  • concurrent updates on striped data streams in Clustered Server systems
    International Parallel and Distributed Processing Symposium, 2001
    Co-Authors: Peter Sobe
    Abstract:

    Data Striping is widely used and well-understood in Redundant Arrays of Independent Disks (RAID), whereby an array of disks is controlled by a single device controller. By striping and adding parity information, distribution of I/O load and fault tolerance can be assured. This study focuses the problem of update/read consistency of files that are striped among several nodes, e.g. in a Clustered Server for media streaming applications. Common RAID systems offer a single entry point so that a read operation delivers either the content before an update or the updated content. This property is not present when arbitrary nodes may access data stripes without a centralized access facility. Thus, coordination is necessary among updates and concurrent read operations. A common solution is to lock files or blocks that are currently being updated. We propose an alternative solution to ensure delivery of valid content during updates without locking blocks globally. This can be used for instance to update the original content of video Servers, without the need of temporary copies and access limitations.

  • IPDPS - Concurrent updates on striped data streams in Clustered Server systems
    Proceedings 15th International Parallel and Distributed Processing Symposium. IPDPS 2001, 1
    Co-Authors: Peter Sobe
    Abstract:

    Data Striping is widely used and well-understood in Redundant Arrays of Independent Disks (RAID), whereby an array of disks is controlled by a single device controller. By striping and adding parity information, distribution of I/O load and fault tolerance can be assured. This study focuses the problem of update/read consistency of files that are striped among several nodes, e.g. in a Clustered Server for media streaming applications. Common RAID systems offer a single entry point so that a read operation delivers either the content before an update or the updated content. This property is not present when arbitrary nodes may access data stripes without a centralized access facility. Thus, coordination is necessary among updates and concurrent read operations. A common solution is to lock files or blocks that are currently being updated. We propose an alternative solution to ensure delivery of valid content during updates without locking blocks globally. This can be used for instance to update the original content of video Servers, without the need of temporary copies and access limitations.

Suyoung Chi - One of the best experts on this subject based on the ideXlab platform.

  • an implementation of a high throughput data ingestion system for machine logs in manufacturing industry
    International Conference on Ubiquitous and Future Networks, 2016
    Co-Authors: Jaehui Park, Suyoung Chi
    Abstract:

    This paper aims at presenting a case study of designing and implementing a data ingestion system for manufacturers. In our implementation, Clustered Server architecture for high throughput data ingestion is proposed with regard to following factors: receiving stream data, i.e., machine logs, from a set of milling machines, storing them in a centralized messaging queue, and sinking to external systems with ease. Especially, we leverage the power of the open sources frameworks, Apache Kafka, Apache Hadoop File System and Apache Flume to cope with the data streams from a large number of machines in the factory floors. As this is an on-going study, we only illustrate our implementation details with structural diagrams, but exclude the theoretical study and the performance evaluation results in this paper.

  • ICUFN - An implementation of a high throughput data ingestion system for machine logs in manufacturing industry
    2016 Eighth International Conference on Ubiquitous and Future Networks (ICUFN), 2016
    Co-Authors: Jaehui Park, Suyoung Chi
    Abstract:

    This paper aims at presenting a case study of designing and implementing a data ingestion system for manufacturers. In our implementation, Clustered Server architecture for high throughput data ingestion is proposed with regard to following factors: receiving stream data, i.e., machine logs, from a set of milling machines, storing them in a centralized messaging queue, and sinking to external systems with ease. Especially, we leverage the power of the open sources frameworks, Apache Kafka, Apache Hadoop File System and Apache Flume to cope with the data streams from a large number of machines in the factory floors. As this is an on-going study, we only illustrate our implementation details with structural diagrams, but exclude the theoretical study and the performance evaluation results in this paper.

Jaehui Park - One of the best experts on this subject based on the ideXlab platform.

  • an implementation of a high throughput data ingestion system for machine logs in manufacturing industry
    International Conference on Ubiquitous and Future Networks, 2016
    Co-Authors: Jaehui Park, Suyoung Chi
    Abstract:

    This paper aims at presenting a case study of designing and implementing a data ingestion system for manufacturers. In our implementation, Clustered Server architecture for high throughput data ingestion is proposed with regard to following factors: receiving stream data, i.e., machine logs, from a set of milling machines, storing them in a centralized messaging queue, and sinking to external systems with ease. Especially, we leverage the power of the open sources frameworks, Apache Kafka, Apache Hadoop File System and Apache Flume to cope with the data streams from a large number of machines in the factory floors. As this is an on-going study, we only illustrate our implementation details with structural diagrams, but exclude the theoretical study and the performance evaluation results in this paper.

  • ICUFN - An implementation of a high throughput data ingestion system for machine logs in manufacturing industry
    2016 Eighth International Conference on Ubiquitous and Future Networks (ICUFN), 2016
    Co-Authors: Jaehui Park, Suyoung Chi
    Abstract:

    This paper aims at presenting a case study of designing and implementing a data ingestion system for manufacturers. In our implementation, Clustered Server architecture for high throughput data ingestion is proposed with regard to following factors: receiving stream data, i.e., machine logs, from a set of milling machines, storing them in a centralized messaging queue, and sinking to external systems with ease. Especially, we leverage the power of the open sources frameworks, Apache Kafka, Apache Hadoop File System and Apache Flume to cope with the data streams from a large number of machines in the factory floors. As this is an on-going study, we only illustrate our implementation details with structural diagrams, but exclude the theoretical study and the performance evaluation results in this paper.

Baoqun Yin - One of the best experts on this subject based on the ideXlab platform.

  • Dynamic file grouping for load balancing in streaming media Clustered Server systems
    International Journal of Control Automation and Systems, 2009
    Co-Authors: Qi Jiang, Baoqun Yin
    Abstract:

    A dynamic file grouping strategy is presented to address the load balancing problem in streaming media Clustered Server systems. This strategy increases the Server cluster availability by balancing the workloads among the Servers within a cluster. Additionally, it improves the access hit ratio of cached files in delivery Servers to alleviate the limitation of I/O bandwidth of storage node. First, the load balancing problem is formulated as a two layers semi-Markov switching state-space control process. This analytic model captures the behaviors of streaming media Clustered Server systems accurately, and is with constructional flexibility and scalability. Then, a policy iteration based reinforcement learning algorithm is proposed to optimize the file grouping policy online. By utilizing the features of the event-driven policy, the proposed optimization algorithm is adaptive and with less computational cost. Simulation results demonstrate the effectiveness of the proposed approach.

  • Dynamic File Grouping for Load Balancing in Streaming Media Clustered Server Systems
    2007 International Conference on Information Acquisition, 2007
    Co-Authors: Qi Jiang, Baoqun Yin
    Abstract:

    A dynamic file grouping strategy is presented to address the load balancing problem in streaming media Clustered Server systems. This strategy increases the Server cluster availability by balancing the workloads among the Servers within a cluster. Additionally, it improves the access hit ratio of cached files in delivery Servers to alleviate the limitation of I/O bandwidth of storage node. First, the load balancing problem is formulated as a two layer semi-Markov switching state-space control process. Then, a gradient-based reinforcement learning algorithm is proposed to optimize the grouping policy online. This analytic model captures the behaviors of streaming media Clustered Server systems accurately, and is with constructional flexibility and scalability. By utilizing the features of the event-driven policy, the proposed optimization algorithm is adaptive and with less computational cost. Simulation results demonstrate the effectiveness of the proposed approach.

Qi Jiang - One of the best experts on this subject based on the ideXlab platform.

  • Dynamic file grouping for load balancing in streaming media Clustered Server systems
    International Journal of Control Automation and Systems, 2009
    Co-Authors: Qi Jiang, Baoqun Yin
    Abstract:

    A dynamic file grouping strategy is presented to address the load balancing problem in streaming media Clustered Server systems. This strategy increases the Server cluster availability by balancing the workloads among the Servers within a cluster. Additionally, it improves the access hit ratio of cached files in delivery Servers to alleviate the limitation of I/O bandwidth of storage node. First, the load balancing problem is formulated as a two layers semi-Markov switching state-space control process. This analytic model captures the behaviors of streaming media Clustered Server systems accurately, and is with constructional flexibility and scalability. Then, a policy iteration based reinforcement learning algorithm is proposed to optimize the file grouping policy online. By utilizing the features of the event-driven policy, the proposed optimization algorithm is adaptive and with less computational cost. Simulation results demonstrate the effectiveness of the proposed approach.

  • Dynamic File Grouping for Load Balancing in Streaming Media Clustered Server Systems
    2007 International Conference on Information Acquisition, 2007
    Co-Authors: Qi Jiang, Baoqun Yin
    Abstract:

    A dynamic file grouping strategy is presented to address the load balancing problem in streaming media Clustered Server systems. This strategy increases the Server cluster availability by balancing the workloads among the Servers within a cluster. Additionally, it improves the access hit ratio of cached files in delivery Servers to alleviate the limitation of I/O bandwidth of storage node. First, the load balancing problem is formulated as a two layer semi-Markov switching state-space control process. Then, a gradient-based reinforcement learning algorithm is proposed to optimize the grouping policy online. This analytic model captures the behaviors of streaming media Clustered Server systems accurately, and is with constructional flexibility and scalability. By utilizing the features of the event-driven policy, the proposed optimization algorithm is adaptive and with less computational cost. Simulation results demonstrate the effectiveness of the proposed approach.