The Experts below are selected from a list of 4245 Experts worldwide ranked by ideXlab platform
Mohammad Karim Sohrabi - One of the best experts on this subject based on the ideXlab platform.
-
A novel coral reefs optimization algorithm for Materialized View selection in data warehouse environments
Applied Intelligence, 2019Co-Authors: Hossein Azgomi, Mohammad Karim SohrabiAbstract:High response time of analytical queries is one of the most challenging issues of data warehouses. Complicated nature of analytical queries and enormous volume of data are the most important reasons of this high response time. The aim of Materialized View selection is to reduce the response time of these analytical queries. For this purpose, the search space is firstly constructed by producing the set of all possible Views based on given queries and then, the (semi-) optimal set of Materialized Views will be selected so that the queries can be answered at the lowest cost using them. Various Materialized View selection methods have been proposed in the literature, most of which are randomized methods due to the time-consuming nature of this problem. Randomized View selection methods choose a semi-optimal set of proper Views for materialization in an appropriate time using one or a combination of some meta-heuristic(s). In this paper, a novel coral reefs optimization-based method is introduced for Materialized View selection in a data warehouse. Coral reefs optimization algorithm is an optimization method that solves problems by simulating the coral behaviors for placement and growth in reefs. In the proposed method, each solution of the problem is considered as a coral, which is always trying to be placed and grow in the reefs. In each step, special operators of the coral reefs optimization algorithm are applied on the solutions. After several steps, better solutions are more likely to survive and grow on the reefs. The best solution is finally chosen as the final solution of the problem. The practical evaluations of the proposed method show that this method offers higher quality solutions than other similar random methods in terms of coverage rate of queries.
-
evolutionary game theory approach to Materialized View selection in data warehouses
Knowledge Based Systems, 2019Co-Authors: Mohammad Karim Sohrabi, Hossein AzgomiAbstract:Abstract The data warehouse contains a number of Views that are used to respond to the system queries. On the one hand, the time consuming process of responding to analytical queries of the data warehouse requires to store intermediate Views for efficient query answering, and on the other hand, large numbers and high volumes of intermediate Views, make the storage of all Views impossible. Hence, choosing the optimal set of Views for materialization is one of the most important decisions in the data warehouses design, in order to increase efficiency of query answering. Since the search space of the problem is very large, searching among the collections of all possible Views of a data warehouse is very expensive and thus, it is necessary to use methods to solve the problem in an acceptable time. Random methods, such as game theory-based optimization approaches, try to increase the speed of selecting Materialized Views by finding near optimal solutions. In this article, an evolutionary game theory-based method to Materialized View selection in the data warehouse is represented which exploits the multiple View processing plan structure to represent the search space of the problem. In this method, a population of players is created, each of which is a solution to the problem. Three strategies are considered for each player and at each repetition of the game, players attempt to choose the best strategy for themselves. At the end of the game, the final solution is calculated according to the strategies selected by the players. Our empirical evaluations revealed that the proposed method has appropriate convergence for large data warehouses and its execution time is very good. It is also shown that the quality of the solutions of the proposed method is more appropriate than other similar random methods.
-
a game theory based framework for Materialized View selection in data warehouses
Engineering Applications of Artificial Intelligence, 2018Co-Authors: Hossein Azgomi, Mohammad Karim SohrabiAbstract:Abstract Data warehouses exploit On-Line Analytical Processing (OLAP) to make rapid answers for analytical queries. Huge amount of aggregated data within a data warehouse on the one hand, and complex analytical queries raised in a data warehouse on the other hand, increase response time to queries extremely. To solve this problem, a number of Views are derived and extracted from original base tables and queries have been answered using them. Since materialization of all possible Views is not effective because of limitation of storage and maintenance overhead, selecting an optimal set of Views for materialization is crucial to maximize data warehouse performance. In this paper, a game theory based framework for the Materialized View selection is proposed. In the proposed framework, query processing and View maintenance costs play a game against each other as two players and continue the game until reach the equilibrium. According to the framework, a new static method, called Game Theory based Materialized View selection (GTMV), has been proposed. Verification of proposed approach has been evaluated using several synthetic and real world datasets. Experimental results show that the GTMV method has better performance comparing previous algorithms and substantially outperform former methods.
-
tsgv a table like structure based greedy method for Materialized View selection in data warehouses
Turkish Journal of Electrical Engineering and Computer Sciences, 2017Co-Authors: Mohammad Karim Sohrabi, Hossein AzgomiAbstract:Since a data warehouse deals with huge amounts of data and complex analytical queries, online processing and answering to users' queries in data warehouses can be a serious challenge. Materialized Views are used to speed up query processing rather than direct access to the database in on-line analytical processing. Since the large number and high volume of Views prevents all of the Views from being stored, selection of a proper subset of Views to materialization is inevitable. Proposing an appropriate method for selecting the optimal subset of Views for materialization plays an essential role in increasing the efficiency of responding to data warehouse queries. In this paper, a greedy Materialized View selection algorithm is represented, which selects a proper set of Views for materialization from a novel table-like structure. The information in this table-like structure is extracted from a multivalue processing plan. This table-like structure-based greedy View selection (TSGV) method is evaluated using the queries of an analytical database, and the query-processing and View maintenance costs of the selected subset are both considered in this evaluation. The experimental results show that TSGV operates better than previously represented methods in terms of time.
Hossein Azgomi - One of the best experts on this subject based on the ideXlab platform.
-
A novel coral reefs optimization algorithm for Materialized View selection in data warehouse environments
Applied Intelligence, 2019Co-Authors: Hossein Azgomi, Mohammad Karim SohrabiAbstract:High response time of analytical queries is one of the most challenging issues of data warehouses. Complicated nature of analytical queries and enormous volume of data are the most important reasons of this high response time. The aim of Materialized View selection is to reduce the response time of these analytical queries. For this purpose, the search space is firstly constructed by producing the set of all possible Views based on given queries and then, the (semi-) optimal set of Materialized Views will be selected so that the queries can be answered at the lowest cost using them. Various Materialized View selection methods have been proposed in the literature, most of which are randomized methods due to the time-consuming nature of this problem. Randomized View selection methods choose a semi-optimal set of proper Views for materialization in an appropriate time using one or a combination of some meta-heuristic(s). In this paper, a novel coral reefs optimization-based method is introduced for Materialized View selection in a data warehouse. Coral reefs optimization algorithm is an optimization method that solves problems by simulating the coral behaviors for placement and growth in reefs. In the proposed method, each solution of the problem is considered as a coral, which is always trying to be placed and grow in the reefs. In each step, special operators of the coral reefs optimization algorithm are applied on the solutions. After several steps, better solutions are more likely to survive and grow on the reefs. The best solution is finally chosen as the final solution of the problem. The practical evaluations of the proposed method show that this method offers higher quality solutions than other similar random methods in terms of coverage rate of queries.
-
evolutionary game theory approach to Materialized View selection in data warehouses
Knowledge Based Systems, 2019Co-Authors: Mohammad Karim Sohrabi, Hossein AzgomiAbstract:Abstract The data warehouse contains a number of Views that are used to respond to the system queries. On the one hand, the time consuming process of responding to analytical queries of the data warehouse requires to store intermediate Views for efficient query answering, and on the other hand, large numbers and high volumes of intermediate Views, make the storage of all Views impossible. Hence, choosing the optimal set of Views for materialization is one of the most important decisions in the data warehouses design, in order to increase efficiency of query answering. Since the search space of the problem is very large, searching among the collections of all possible Views of a data warehouse is very expensive and thus, it is necessary to use methods to solve the problem in an acceptable time. Random methods, such as game theory-based optimization approaches, try to increase the speed of selecting Materialized Views by finding near optimal solutions. In this article, an evolutionary game theory-based method to Materialized View selection in the data warehouse is represented which exploits the multiple View processing plan structure to represent the search space of the problem. In this method, a population of players is created, each of which is a solution to the problem. Three strategies are considered for each player and at each repetition of the game, players attempt to choose the best strategy for themselves. At the end of the game, the final solution is calculated according to the strategies selected by the players. Our empirical evaluations revealed that the proposed method has appropriate convergence for large data warehouses and its execution time is very good. It is also shown that the quality of the solutions of the proposed method is more appropriate than other similar random methods.
-
a game theory based framework for Materialized View selection in data warehouses
Engineering Applications of Artificial Intelligence, 2018Co-Authors: Hossein Azgomi, Mohammad Karim SohrabiAbstract:Abstract Data warehouses exploit On-Line Analytical Processing (OLAP) to make rapid answers for analytical queries. Huge amount of aggregated data within a data warehouse on the one hand, and complex analytical queries raised in a data warehouse on the other hand, increase response time to queries extremely. To solve this problem, a number of Views are derived and extracted from original base tables and queries have been answered using them. Since materialization of all possible Views is not effective because of limitation of storage and maintenance overhead, selecting an optimal set of Views for materialization is crucial to maximize data warehouse performance. In this paper, a game theory based framework for the Materialized View selection is proposed. In the proposed framework, query processing and View maintenance costs play a game against each other as two players and continue the game until reach the equilibrium. According to the framework, a new static method, called Game Theory based Materialized View selection (GTMV), has been proposed. Verification of proposed approach has been evaluated using several synthetic and real world datasets. Experimental results show that the GTMV method has better performance comparing previous algorithms and substantially outperform former methods.
-
tsgv a table like structure based greedy method for Materialized View selection in data warehouses
Turkish Journal of Electrical Engineering and Computer Sciences, 2017Co-Authors: Mohammad Karim Sohrabi, Hossein AzgomiAbstract:Since a data warehouse deals with huge amounts of data and complex analytical queries, online processing and answering to users' queries in data warehouses can be a serious challenge. Materialized Views are used to speed up query processing rather than direct access to the database in on-line analytical processing. Since the large number and high volume of Views prevents all of the Views from being stored, selection of a proper subset of Views to materialization is inevitable. Proposing an appropriate method for selecting the optimal subset of Views for materialization plays an essential role in increasing the efficiency of responding to data warehouse queries. In this paper, a greedy Materialized View selection algorithm is represented, which selects a proper set of Views for materialization from a novel table-like structure. The information in this table-like structure is extracted from a multivalue processing plan. This table-like structure-based greedy View selection (TSGV) method is evaluated using the queries of an analytical database, and the query-processing and View maintenance costs of the selected subset are both considered in this evaluation. The experimental results show that TSGV operates better than previously represented methods in terms of time.
Qing Li - One of the best experts on this subject based on the ideXlab platform.
-
algorithms for Materialized View design in data warehousing environment
Very Large Data Bases, 1997Co-Authors: Jian Yang, Kamalakar Karlapalem, Qing LiAbstract:Selecting Views to materialize is one of the most important decisions in designing a data warehouse. In this paper, we present a framework for analyzing the issues in selecting Views to materialize so as to achieve the best combination of good query performance and low View maintenance. We first develop a heuristic algorithm which can provide a feasible solution based on individual optimal query plans. We also map the Materialized View design problem as O-l integer programming problem, whose solution can guarantee an optimal solution.
-
tackling the challenges of Materialized View design in data warehousing environment
International Workshop on Research Issues in Data Engineering, 1997Co-Authors: Jian Yang, Kamalakar Karlapalem, Qing LiAbstract:The design of Materialized Views in a data warehousing environment is an important problem which has been largely overlooked in the past. If one regards data warehouse queries as integrated Views over the base databases, then there is a need to select a set of Views to be Materialized so that the best combination of good performance and low maintenance cost can be achieved. The authors compare Materialized View design (MVD) work with related problems such as common subexpressions and multiple query processing, discuss the unique requirements of MVD, and outline possible solutions of addressing some of the challenging issues of MVD.
Masahide Nakamura - One of the best experts on this subject based on the ideXlab platform.
-
using Materialized View as a service of scallop4sc for smart city application services
2014Co-Authors: Shintaro Yamamoto, Sachio Saiki, Shinsuke Matsumoto, Masahide NakamuraAbstract:Smart city provides various value-added services by collecting large-scale data from houses and infrastructures within a city. However, it takes a long time and man-hour and needs knowledge about big data processing for individual applications to use and process the large-scale raw data directly. To reduce the response time, we use the concept of Materialized View of database, and Materialized View to be as a service. And we propose Materialized View to be as as service (MVaaS). In our proposition, a developer of an application can efficiently and dynamically use large-scale data from smart city by describing simple data specification without considering distributed processes and Materialized Views. In this paper, we design an architecture of MVaaS using MapReduce on Hadoop and HBase KVS. And we demonstrate the effectiveness of MVaaS through three case studies. If these services uses raw data, it needs enormous time of calculation and is not realistic.
-
Materialized View as a service for large scale house log in smart city
IEEE International Conference on Cloud Computing Technology and Science, 2013Co-Authors: Shintaro Yamamoto, Sachio Saiki, Shinsuke Matsumoto, Masahide NakamuraAbstract:Smart city provides various value-added services by collecting large-scale data from houses and infrastructures within a city. To use such large-scale raw data, individual applications usually take expensive computation effort and large processing time. To reduce the effort and time, we propose Materialized View as a Service (MVaaS). Using the MVaaS, each application can easily and dynamically construct its own Materialized View, in which the raw data is cached in an appropriate format for the application. Once the View is constructed, the application can quickly access necessary data. In this paper, we design a framework of MVaaS specifically for large-scale house log, managed in our smart-city data platform Scallop4SC. In the framework, each application first specifies how the raw data should be filtered, grouped and aggregated. For a given data specification, MVaaS dynamically constructs a MapReduce batch program that converts the raw data into a desired View. The batch is then executed on Hadoop, and the resultant View is stored in HBase. We conduct an experimental evaluation to compare the response time between cases with and without the proposed MVaaS.
-
implementing Materialized View of large scale power consumption log using mapreduce
Software Engineering Artificial Intelligence Networking and Parallel Distributed Computing, 2013Co-Authors: Shintaro Yamamoto, Sachio Saiki, Shinsuke Matsumoto, Masahide NakamuraAbstract:Smart city provides various value-added services by collecting large-scale data from houses and infrastructures within a city. However, it takes a long time for individual applications to use and process the large-scale raw data directly. To reduce the response time, we use the concept of Materialized View of database. For a given requirement of an application, the proposed method constructs a Materialized View for caching the application-specific data. In this paper, we especially develop a method that uses MapReduce for large-scale power consumption data stored in HBase KVS. We conduct an experimental evaluation to compare the response time between cases with and without the Materialized View. As a result, the proposed method with Materialized View is effective especially when application repeatedly access the same data, or when the application-specific data is derived from a large set of raw data.
-
Materialized View as a Service for Large-Scale House Log in Smart City
2013Co-Authors: Shintaro Yamamoto, Sachio Saiki, Shinsuke Matsumoto, Masahide NakamuraAbstract:Smart city provides various value-added services by collecting large-scale data from houses and infrastructures within a city. To use such large-scale raw data, individual applications usually take expensive computation effort and large processing time. To reduce the effort and time, we propose Materialized View as a Service (MVaaS). Using the MVaaS, each application can easily and dynamically construct its own Materialized View, in which the raw data is cached in an appropriate format for the application. Once the View is constructed, the application can quickly access necessary data. In this paper, we design a framework of MVaaS specifically for large-scale house log, managed in our smart-city data platform Scallop4SC. In the framework, each application first specifies how the raw data should be filtered, grouped and aggregated. For a given data specification, MVaaS dynamically constructs a MapReduce batch program that converts the raw data into a desired View. The batch is then executed on Hadoop, and the resultant View is stored in HBase. We conduct an experimental evaluation to compare the response time between cases with and without the proposed MVaaS. Keywords-
Jian Yang - One of the best experts on this subject based on the ideXlab platform.
-
algorithms for Materialized View design in data warehousing environment
Very Large Data Bases, 1997Co-Authors: Jian Yang, Kamalakar Karlapalem, Qing LiAbstract:Selecting Views to materialize is one of the most important decisions in designing a data warehouse. In this paper, we present a framework for analyzing the issues in selecting Views to materialize so as to achieve the best combination of good query performance and low View maintenance. We first develop a heuristic algorithm which can provide a feasible solution based on individual optimal query plans. We also map the Materialized View design problem as O-l integer programming problem, whose solution can guarantee an optimal solution.
-
tackling the challenges of Materialized View design in data warehousing environment
International Workshop on Research Issues in Data Engineering, 1997Co-Authors: Jian Yang, Kamalakar Karlapalem, Qing LiAbstract:The design of Materialized Views in a data warehousing environment is an important problem which has been largely overlooked in the past. If one regards data warehouse queries as integrated Views over the base databases, then there is a need to select a set of Views to be Materialized so that the best combination of good performance and low maintenance cost can be achieved. The authors compare Materialized View design (MVD) work with related problems such as common subexpressions and multiple query processing, discuss the unique requirements of MVD, and outline possible solutions of addressing some of the challenging issues of MVD.
