The Experts below are selected from a list of 6399 Experts worldwide ranked by ideXlab platform
Scott Klasky - One of the best experts on this subject based on the ideXlab platform.
-
Archive migration through Workflow Automation
2007Co-Authors: Norbert Podhorszki, Bertram Ludäscher, Scott KlaskyAbstract:The Center for Plasma Edge Simulation project aims to automate the tedious tasks of simulation monitoring, data archival and coupling simulation codes using the Kepler scientific Workflow environment. The technology has been successfully applied for migrating a combustion data archive of 10TB from NERSC to ORNL, where there were no other automated solutions for this task. This paper describes the Workflow that migrates large files from mass storage systems using external tools and temporary staging to disks, performing different stages in a pipeline-parallel fashion, parallelizing file transfers and doing special checkpointing to make the Workflow restartable and also perform operations that failed earlier. The advantage of creating/using such a Workflow over specialized data migration services is its independence from specific systems so it can be used by configuring the external tools to be used. The advantage over scripts is the robust exection (handling failures and timeouts) and efficiency (parallelization wherever possible).
-
Workflow Automation for processing plasma fusion simulation data
Workflows in Support of Large-Scale Science, 2007Co-Authors: Norbert Podhorszki, Bertram Ludaescher, Scott KlaskyAbstract:The Center for Plasma Edge Simulation project aims to automate the tedious tasks of monitoring the simulation, archiving and post-processing the output. This paper describes the tasks and requirements, the several components developed within the Kepler Workflow system to provide the required functionality and the automated Workflow solution. Besides functionality, the focus is on the robust execution of the Workflow. A user-level checkpoint/restart model has been developed that allows the Workflow to restart, moreover, it also performs operations that failed during the previous run.
-
WORKS@HPDC - Workflow Automation for processing plasma fusion simulation data
Proceedings of the 2nd workshop on Workflows in support of large-scale science - WORKS '07, 2007Co-Authors: Norbert Podhorszki, Bertram Ludaescher, Scott KlaskyAbstract:The Center for Plasma Edge Simulation project aims to automate the tedious tasks of monitoring the simulation, archiving and post-processing the output. This paper describes the tasks and requirements, the several components developed within the Kepler Workflow system to provide the required functionality and the automated Workflow solution. Besides functionality, the focus is on the robust execution of the Workflow. A user-level checkpoint/restart model has been developed that allows the Workflow to restart, moreover, it also performs operations that failed during the previous run.
Norbert Podhorszki - One of the best experts on this subject based on the ideXlab platform.
-
Archive migration through Workflow Automation
2007Co-Authors: Norbert Podhorszki, Bertram Ludäscher, Scott KlaskyAbstract:The Center for Plasma Edge Simulation project aims to automate the tedious tasks of simulation monitoring, data archival and coupling simulation codes using the Kepler scientific Workflow environment. The technology has been successfully applied for migrating a combustion data archive of 10TB from NERSC to ORNL, where there were no other automated solutions for this task. This paper describes the Workflow that migrates large files from mass storage systems using external tools and temporary staging to disks, performing different stages in a pipeline-parallel fashion, parallelizing file transfers and doing special checkpointing to make the Workflow restartable and also perform operations that failed earlier. The advantage of creating/using such a Workflow over specialized data migration services is its independence from specific systems so it can be used by configuring the external tools to be used. The advantage over scripts is the robust exection (handling failures and timeouts) and efficiency (parallelization wherever possible).
-
Workflow Automation for processing plasma fusion simulation data
Workflows in Support of Large-Scale Science, 2007Co-Authors: Norbert Podhorszki, Bertram Ludaescher, Scott KlaskyAbstract:The Center for Plasma Edge Simulation project aims to automate the tedious tasks of monitoring the simulation, archiving and post-processing the output. This paper describes the tasks and requirements, the several components developed within the Kepler Workflow system to provide the required functionality and the automated Workflow solution. Besides functionality, the focus is on the robust execution of the Workflow. A user-level checkpoint/restart model has been developed that allows the Workflow to restart, moreover, it also performs operations that failed during the previous run.
-
WORKS@HPDC - Workflow Automation for processing plasma fusion simulation data
Proceedings of the 2nd workshop on Workflows in support of large-scale science - WORKS '07, 2007Co-Authors: Norbert Podhorszki, Bertram Ludaescher, Scott KlaskyAbstract:The Center for Plasma Edge Simulation project aims to automate the tedious tasks of monitoring the simulation, archiving and post-processing the output. This paper describes the tasks and requirements, the several components developed within the Kepler Workflow system to provide the required functionality and the automated Workflow solution. Besides functionality, the focus is on the robust execution of the Workflow. A user-level checkpoint/restart model has been developed that allows the Workflow to restart, moreover, it also performs operations that failed during the previous run.
Miguel Calejo - One of the best experts on this subject based on the ideXlab platform.
-
intelligent document routing as a first step towards Workflow Automation a case study implemented in sql
Leveraging Applications of Formal Methods, 2010Co-Authors: Carlos Soares, Miguel CalejoAbstract:In large and complex organizations, the development of Workflow Automation projects is hard. In some cases, a first important step in that direction is the Automation of the routing of incoming documents. In this paper, we describe a project to develop a system for the first routing of incoming letters to the right department within a large, public portuguese institution. We followed a data mining approach, where data representing previous routings were analyzed to obtain a model that can be used to route future documents. The approach followed was strongly influenced by some of the limitations imposed by the customer: the budget available was small and the solution should be developed in SQL to facilitate integration with the existing system. The system developed was able to obtain satisfactory results. However, as in any Data Mining project, most of the effort was dedicated to activities other than modelling (e.g., data preparation), which means that there is still plenty of room for improvement.
-
ISoLA (1) - Intelligent document routing as a first step towards Workflow Automation: a case study implemented in SQL
Lecture Notes in Computer Science, 2010Co-Authors: Carlos Soares, Miguel CalejoAbstract:In large and complex organizations, the development of Workflow Automation projects is hard. In some cases, a first important step in that direction is the Automation of the routing of incoming documents. In this paper, we describe a project to develop a system for the first routing of incoming letters to the right department within a large, public portuguese institution. We followed a data mining approach, where data representing previous routings were analyzed to obtain a model that can be used to route future documents. The approach followed was strongly influenced by some of the limitations imposed by the customer: the budget available was small and the solution should be developed in SQL to facilitate integration with the existing system. The system developed was able to obtain satisfactory results. However, as in any Data Mining project, most of the effort was dedicated to activities other than modelling (e.g., data preparation), which means that there is still plenty of room for improvement.
Carlos Soares - One of the best experts on this subject based on the ideXlab platform.
-
intelligent document routing as a first step towards Workflow Automation a case study implemented in sql
Leveraging Applications of Formal Methods, 2010Co-Authors: Carlos Soares, Miguel CalejoAbstract:In large and complex organizations, the development of Workflow Automation projects is hard. In some cases, a first important step in that direction is the Automation of the routing of incoming documents. In this paper, we describe a project to develop a system for the first routing of incoming letters to the right department within a large, public portuguese institution. We followed a data mining approach, where data representing previous routings were analyzed to obtain a model that can be used to route future documents. The approach followed was strongly influenced by some of the limitations imposed by the customer: the budget available was small and the solution should be developed in SQL to facilitate integration with the existing system. The system developed was able to obtain satisfactory results. However, as in any Data Mining project, most of the effort was dedicated to activities other than modelling (e.g., data preparation), which means that there is still plenty of room for improvement.
-
ISoLA (1) - Intelligent document routing as a first step towards Workflow Automation: a case study implemented in SQL
Lecture Notes in Computer Science, 2010Co-Authors: Carlos Soares, Miguel CalejoAbstract:In large and complex organizations, the development of Workflow Automation projects is hard. In some cases, a first important step in that direction is the Automation of the routing of incoming documents. In this paper, we describe a project to develop a system for the first routing of incoming letters to the right department within a large, public portuguese institution. We followed a data mining approach, where data representing previous routings were analyzed to obtain a model that can be used to route future documents. The approach followed was strongly influenced by some of the limitations imposed by the customer: the budget available was small and the solution should be developed in SQL to facilitate integration with the existing system. The system developed was able to obtain satisfactory results. However, as in any Data Mining project, most of the effort was dedicated to activities other than modelling (e.g., data preparation), which means that there is still plenty of room for improvement.
Nataliia N. Kussul - One of the best experts on this subject based on the ideXlab platform.
-
Service-oriented infrastructure for flood mapping using optical and SAR satellite data
International Journal of Digital Earth, 2013Co-Authors: Nataliia N. Kussul, Sergii V. Skakun, Andrii Yu. Shelestov, Olga M. Kussul, Jibo XieAbstract:In this paper, we present the service-oriented infrastructure within the Wide Area Grid project that was carried out within the Working Group on Information Systems and Services of the Committee on Earth Observation Satellites. The developed infrastructure integrates services and computational resources of several regional and national Grid systems: Ukrainian Academician Grid (with satellite data processing Grid segment, UASpaceGrid) and Grid system at the Center on Earth Observation and Digital Earth of Chinese Academy of Sciences. The study focuses on integrating geo-information services on flood mapping provided by Ukrainian and Chinese entities to benefit from information acquired from multiple sources. We also describe services for Workflow Automation and management in Grid environment and provide an example of Workflow Automation for generating flood maps from optical and synthetic-aperture radar satellite imagery. We also discuss issues of enabling trust for the infrastructure using certificates an...
-
The Wide Area Grid Testbed for Flood Monitoring Using Earth Observation Data
IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2012Co-Authors: Nataliia N. Kussul, Sergii V. Skakun, Andrii Yu. Shelestov, Guoqing Li, Olga M. KussulAbstract:In this paper we present the results of the Wide Area Grid (WAG) project that was carried out within the Working Group on Information Systems and Services (WGISS) of the Committee on Earth Observation Satellites (CEOS). We describe the InterGrid infrastructure that integrates several regional and national Grid systems: Ukrainian Academician Grid (with satellite data processing Grid segment, UASpaceGrid) and Grid system at the Center on Earth Observation and Digital Earth of Chinese Academy of Sciences (CEODE-CAS). The study also focuses on Workflow Automation and management in Grid environment, and provides an example of Workflow Automation for generating flood maps from satellite imagery. Examples of utilizing the developed infrastructure for operational flood mapping are given as well.
-
Grid and Cloud Database Management - Grid Technologies for Satellite Data Processing and Management Within International Disaster Monitoring Projects
Grid and Cloud Database Management, 2011Co-Authors: Nataliia N. Kussul, Andrii Yu. Shelestov, Sergii V. SkakunAbstract:This chapter describes the use of Grid technologies for satellite data processing and management within international disaster monitoring projects carried out by the Space Research Institute NASU-NSAU, Ukraine (SRI NASU-NSAU). This includes the integration of the Ukrainian and Russian satellite monitoring systems at the data level, and the development of the InterGrid infrastructure that integrates several regional and national Grid systems. A problem of Grid and Sensor Web integration is discussed with several solutions and case-studies given. This study also focuses on Workflow Automation and management in Grid environment, and provides an example of Workflow Automation for generating flood maps from images acquired by the Advanced Synthetic Aperture Radar (ASAR) instrument aboard the Envisat satellite.
