The Experts below are selected from a list of 210 Experts worldwide ranked by ideXlab platform
Lukas Reitz - One of the best experts on this subject based on the ideXlab platform.
-
A Selective and Incremental Backup Scheme for Task Pools
2018 International Conference on High Performance Computing & Simulation (HPCS), 2018Co-Authors: Claudia Fohry, Jonas Posner, Lukas ReitzAbstract:Checkpointing is a common approach to prevent loss of a program's state after permanent node failures. When it is performed on application-level, less data need to be saved. This paper suggests an uncoordinated application-level checkpointing technique for task pools. It selectively and Incrementally saves only those tasks that have stayed in the pool during some period of time and that have not been saved before. The checkpoints are held in a resilient in-memory data store. Our technique applies to any task pool variant in which workers operate at the top of local pools, and work stealing operates at the bottom. Furthermore, the tasks must be free of side effects, and the final result must be calculated by reduction from individual task results. We implemented the technique for the lifeline-based global load balancing variant of task pools. This variant couples random victim selection with an overlay graph for termination detection. A fault-tolerant realization already exists in the form of a Java library, called JFT_GLB. It uses the APGAS and Hazelcast libraries underneath. Our implementation modifies JFT_GLB by replacing its nonselective checkpointing scheme with our new one. In experiments, we compared the overhead of the new scheme to that of JFT_GLB, with UTS, BC and two synthetic benchmarks. The new scheme required slightly more running time when local pools were small, and paid off otherwise.
-
HPCS - A Selective and Incremental Backup Scheme for Task Pools
2018 International Conference on High Performance Computing & Simulation (HPCS), 2018Co-Authors: Claudia Fohry, Jonas Posner, Lukas ReitzAbstract:Checkpointing is a common approach to prevent loss of a program's state after permanent node failures. When it is performed on application-level, less data need to be saved. This paper suggests an uncoordinated application-level checkpointing technique for task pools. It selectively and Incrementally saves only those tasks that have stayed in the pool during some period of time and that have not been saved before. The checkpoints are held in a resilient in-memory data store. Our technique applies to any task pool variant in which workers operate at the top of local pools, and work stealing operates at the bottom. Furthermore, the tasks must be free of side effects, and the final result must be calculated by reduction from individual task results. We implemented the technique for the lifeline-based global load balancing variant of task pools. This variant couples random victim selection with an overlay graph for termination detection. A fault-tolerant realization already exists in the form of a Java library, called JFT_GLB. It uses the APGAS and Hazelcast libraries underneath. Our implementation modifies JFT_GLB by replacing its nonselective checkpointing scheme with our new one. In experiments, we compared the overhead of the new scheme to that of JFT_GLB, with UTS, BC and two synthetic benchmarks. The new scheme required slightly more running time when local pools were small, and paid off otherwise.
Shih-yu Lu - One of the best experts on this subject based on the ideXlab platform.
-
Encrypted Incremental Backup without Server-Side Software
2013 27th International Conference on Advanced Information Networking and Applications Workshops, 2013Co-Authors: Shih-yu LuAbstract:With the development of science and technology, the capacity of hard disk and quantity of data are getting bigger and bigger. Backup become important mechanism for the user or enterprises who don't want lose their data. When the amount of data is small, it's fine to do full Backup every time. If the amount of data is huge, it will take a long time to do full Backup every time. In this case, most enterprise will use Incremental Backup for saving time. Incremental Backup preserves data by not creating multiple copies, therefore Incremental Backup is faster than full Backup. In order to solve security problem, encryption is necessary when Backup data storage in storage server. But there is one problem, most Incremental Backup cannot be encrypted. So, this paper will show how to do encrypted Incremental Backup without server-side software requirement. Encrypted Incremental Backup collect information from every file and stores information into one file called checksum file. That information include filename, checksum, last modified time, file size, delete stamp, and encryption key. When Backup start, client will collect filename, checksum, last modified time, and file size. Client gets another checksum file from storage server. By comparing those two checksum files, system will know what files have been changed, and should be transmitted in Backup at this time. Before those files transmit to storage server, client will generate random keys to encrypt those files, and store encrypted keys in checksum file. System will use another key to encrypt checksum file, and using admin password to encrypt/decrypt this KEK (key encryption key).
-
AINA Workshops - Encrypted Incremental Backup without Server-Side Software
2013 27th International Conference on Advanced Information Networking and Applications Workshops, 2013Co-Authors: Shih-yu LuAbstract:With the development of science and technology, the capacity of hard disk and quantity of data are getting bigger and bigger. Backup become important mechanism for the user or enterprises who don't want lose their data. When the amount of data is small, it's fine to do full Backup every time. If the amount of data is huge, it will take a long time to do full Backup every time. In this case, most enterprise will use Incremental Backup for saving time. Incremental Backup preserves data by not creating multiple copies, therefore Incremental Backup is faster than full Backup. In order to solve security problem, encryption is necessary when Backup data storage in storage server. But there is one problem, most Incremental Backup cannot be encrypted. So, this paper will show how to do encrypted Incremental Backup without server-side software requirement. Encrypted Incremental Backup collect information from every file and stores information into one file called checksum file. That information include filename, checksum, last modified time, file size, delete stamp, and encryption key. When Backup start, client will collect filename, checksum, last modified time, and file size. Client gets another checksum file from storage server. By comparing those two checksum files, system will know what files have been changed, and should be transmitted in Backup at this time. Before those files transmit to storage server, client will generate random keys to encrypt those files, and store encrypted keys in checksum file. System will use another key to encrypt checksum file, and using admin password to encrypt/decrypt this KEK (key encryption key).
Claudia Fohry - One of the best experts on this subject based on the ideXlab platform.
-
A Selective and Incremental Backup Scheme for Task Pools
2018 International Conference on High Performance Computing & Simulation (HPCS), 2018Co-Authors: Claudia Fohry, Jonas Posner, Lukas ReitzAbstract:Checkpointing is a common approach to prevent loss of a program's state after permanent node failures. When it is performed on application-level, less data need to be saved. This paper suggests an uncoordinated application-level checkpointing technique for task pools. It selectively and Incrementally saves only those tasks that have stayed in the pool during some period of time and that have not been saved before. The checkpoints are held in a resilient in-memory data store. Our technique applies to any task pool variant in which workers operate at the top of local pools, and work stealing operates at the bottom. Furthermore, the tasks must be free of side effects, and the final result must be calculated by reduction from individual task results. We implemented the technique for the lifeline-based global load balancing variant of task pools. This variant couples random victim selection with an overlay graph for termination detection. A fault-tolerant realization already exists in the form of a Java library, called JFT_GLB. It uses the APGAS and Hazelcast libraries underneath. Our implementation modifies JFT_GLB by replacing its nonselective checkpointing scheme with our new one. In experiments, we compared the overhead of the new scheme to that of JFT_GLB, with UTS, BC and two synthetic benchmarks. The new scheme required slightly more running time when local pools were small, and paid off otherwise.
-
HPCS - A Selective and Incremental Backup Scheme for Task Pools
2018 International Conference on High Performance Computing & Simulation (HPCS), 2018Co-Authors: Claudia Fohry, Jonas Posner, Lukas ReitzAbstract:Checkpointing is a common approach to prevent loss of a program's state after permanent node failures. When it is performed on application-level, less data need to be saved. This paper suggests an uncoordinated application-level checkpointing technique for task pools. It selectively and Incrementally saves only those tasks that have stayed in the pool during some period of time and that have not been saved before. The checkpoints are held in a resilient in-memory data store. Our technique applies to any task pool variant in which workers operate at the top of local pools, and work stealing operates at the bottom. Furthermore, the tasks must be free of side effects, and the final result must be calculated by reduction from individual task results. We implemented the technique for the lifeline-based global load balancing variant of task pools. This variant couples random victim selection with an overlay graph for termination detection. A fault-tolerant realization already exists in the form of a Java library, called JFT_GLB. It uses the APGAS and Hazelcast libraries underneath. Our implementation modifies JFT_GLB by replacing its nonselective checkpointing scheme with our new one. In experiments, we compared the overhead of the new scheme to that of JFT_GLB, with UTS, BC and two synthetic benchmarks. The new scheme required slightly more running time when local pools were small, and paid off otherwise.
Mario Graff - One of the best experts on this subject based on the ideXlab platform.
-
pgsync a multiple reader single writer table replication tool for high loaded distributed relational sql databases
Electronics Robotics and Automotive Mechanics Conference, 2007Co-Authors: Eric Sadit Tellez, J Ortiz, Mario GraffAbstract:This paper presents a new tool to preserve coherency of data in a set of Relational Databases: PGSYNC. It performs replication over a set of single tables with a simply multiple-reader/single-writer scheme. PGSYNC claims to be closely to real-time, simple, low latent, secure, multi-platform, and open source solution to the synchronization database problem. It's specially designed to process that generates high volumes of SQL queries. Additionally we present a real life heavy loaded system using PGSYNC as Incremental Backup manager and work load distributor.
-
PGSYNC: A Multiple-Reader/Single-Writer Table Replication Tool For High Loaded Distributed Relational SQL Databases
Electronics Robotics and Automotive Mechanics Conference (CERMA 2007), 2007Co-Authors: Eric Sadit Tellez, J Ortiz, Mario GraffAbstract:This paper presents a new tool to preserve coherency of data in a set of Relational Databases: PGSYNC. It performs replication over a set of single tables with a simply multiple-reader/single-writer scheme. PGSYNC claims to be closely to real-time, simple, low latent, secure, multi-platform, and open source solution to the synchronization database problem. It's specially designed to process that generates high volumes of SQL queries. Additionally we present a real life heavy loaded system using PGSYNC as Incremental Backup manager and work load distributor.
Jonas Posner - One of the best experts on this subject based on the ideXlab platform.
-
A Selective and Incremental Backup Scheme for Task Pools
2018 International Conference on High Performance Computing & Simulation (HPCS), 2018Co-Authors: Claudia Fohry, Jonas Posner, Lukas ReitzAbstract:Checkpointing is a common approach to prevent loss of a program's state after permanent node failures. When it is performed on application-level, less data need to be saved. This paper suggests an uncoordinated application-level checkpointing technique for task pools. It selectively and Incrementally saves only those tasks that have stayed in the pool during some period of time and that have not been saved before. The checkpoints are held in a resilient in-memory data store. Our technique applies to any task pool variant in which workers operate at the top of local pools, and work stealing operates at the bottom. Furthermore, the tasks must be free of side effects, and the final result must be calculated by reduction from individual task results. We implemented the technique for the lifeline-based global load balancing variant of task pools. This variant couples random victim selection with an overlay graph for termination detection. A fault-tolerant realization already exists in the form of a Java library, called JFT_GLB. It uses the APGAS and Hazelcast libraries underneath. Our implementation modifies JFT_GLB by replacing its nonselective checkpointing scheme with our new one. In experiments, we compared the overhead of the new scheme to that of JFT_GLB, with UTS, BC and two synthetic benchmarks. The new scheme required slightly more running time when local pools were small, and paid off otherwise.
-
HPCS - A Selective and Incremental Backup Scheme for Task Pools
2018 International Conference on High Performance Computing & Simulation (HPCS), 2018Co-Authors: Claudia Fohry, Jonas Posner, Lukas ReitzAbstract:Checkpointing is a common approach to prevent loss of a program's state after permanent node failures. When it is performed on application-level, less data need to be saved. This paper suggests an uncoordinated application-level checkpointing technique for task pools. It selectively and Incrementally saves only those tasks that have stayed in the pool during some period of time and that have not been saved before. The checkpoints are held in a resilient in-memory data store. Our technique applies to any task pool variant in which workers operate at the top of local pools, and work stealing operates at the bottom. Furthermore, the tasks must be free of side effects, and the final result must be calculated by reduction from individual task results. We implemented the technique for the lifeline-based global load balancing variant of task pools. This variant couples random victim selection with an overlay graph for termination detection. A fault-tolerant realization already exists in the form of a Java library, called JFT_GLB. It uses the APGAS and Hazelcast libraries underneath. Our implementation modifies JFT_GLB by replacing its nonselective checkpointing scheme with our new one. In experiments, we compared the overhead of the new scheme to that of JFT_GLB, with UTS, BC and two synthetic benchmarks. The new scheme required slightly more running time when local pools were small, and paid off otherwise.
