The Experts below are selected from a list of 477 Experts worldwide ranked by ideXlab platform
Alain Crolotte - One of the best experts on this subject based on the ideXlab platform.
-
Dynamic Statistics Collection in the Teradata Unified Data Architecture
2017 IEEE 33rd International Conference on Data Engineering (ICDE), 2017Co-Authors: Jin Kim Sung, Mohammed Al-kateb, Alain Crolotte, Chengyang Zhang, Paul Sinclair, Linda RoseAbstract:The Unified Data Architecture (UDA) of Teradata is an inclusive multisystem data analytics solution. A key challenge for query optimization under the UDA is to find optimal plans for queries that access data on heterogeneous remote data stores. The challenge comes primarily from the lack of statistics for data stored on remote systems. In this paper, we present techniques implemented in Teradata Database for dynamically collecting statistics on data fetched from a remote system and feeding these statistics back to the query optimizer during query execution. We demonstrate the performance impact of dynamic statistics collection and feedback with experiments conducted on a system that consists of Teradata Database and a remote Hadoop server.
-
optimizing union all join queries in Teradata
International Conference on Data Engineering, 2017Co-Authors: Mohammed Alkateb, Alain Crolotte, Paul L Sinclair, Sanjay NairAbstract:The UNION ALL set operator is useful for combining data from multiple sources. With the emergence of big data ecosystems in which data is typically stored on multiple systems, UNION ALL has become even more important. In this paper, we present optimization techniques implemented in Teradata Database for join queries with UNION ALL. Instead of spooling all branches of UNION ALL before performing join operations, we propose cost-based pushing of joins into branches. Join pushing not only addresses the prohibitive cost of spooling all branches, but also helps in exposing more efficient join methods (e.g., direct hash-based joins) which, otherwise, would not be considered by the query optimizer. The geography of relations being pushed to UNION ALL branches is also adjusted to avoid unnecessary redistributions and duplications of data. We conclude the paper with a performance study that demonstrates the impact of the proposed optimization techniques on query performance.
-
Deployment of Parallel Data Warehouse Joint Design in Teradata
2016Co-Authors: Ladjel Bellatreche, Alain Crolotte, Ahmad Ghazal, Soumia Benkrid, Alfredo CuzzocreaAbstract:Abstract. Parallel databases play an important role in delivering per-formance required by data warehousing applications. Such performance can not be accomplished without a good scheme for fragmentation and allocation of data. Both problems are NP-hard problems and optimal solutions are impractical in real life situations. These two problems were addressed extensively in research and there are a lot of heuristic based efficient solutions that provide good results. In these solutions, the frag-mentation and allocation were done separately. In our previous work, we proposed a genetic solution that solves both problems simultane-ously which we refer to as a the joint solution. The joint solution was tested on some known benchmarks using internal simulations of a paral-lel database. These simulations are not reliable since it lacked the real life aspects of parallelism. This paper addresses this issue by applying our joint solution on the Teradata DBMS. We experiment the SSB bench-mark (TPC-H like) on a Teradata appliance running TD 13.10. The re-sults shows a significant improvement over previous results that performs fragmentation and allocation sequentially.
-
Adaptive Optimizations of Recursive Queries in Teradata
2014Co-Authors: Ahmad Ghazal, Alain Crolotte, Dawit Seid, Mohammed Al-katebAbstract:Recursive queries were introduced as part of ANSI SQL 99 to support processing of hierarchical data like air flight schedules, bill-of-materials, data cube dimension hierarchies, and ancestordescendant information (e.g. XML data stored in relations). Recently, recursive queries have also found extensive use in web data analysis such as social network and click stream data. Teradata implemented recursive queries in V2R6 using static plans whereby a query is executed in multiple iterations, each iteration corresponding to one level of the recursion. Such a static planning strategy may not be optimal since the demographics of intermediate results from recursive iterations often vary to a great extent. Gathering feedback at each iteration could address this problem by providing size estimates to the optimizer which, in turn, can produce an execution plan for the next iteration. However, such a full feedback scheme suffers from lack of pipelining and the inability to exploit global optimizations across the different recursion iterations. In this paper, we propose adaptive optimization techniques that avoid the issues with static as well as full feedback optimization approaches. Our approach employs a mix of multi-iteration pre-planning and dynamic feedback techniques which are generally applicable to any recursive query implementation in an RDBMS. We also validated the effectiveness of our proposed techniques by conducting experiments on a prototype implementation using a real-life social network data from the FriendFeed online blogging service
-
Oracle Corp.,
2013Co-Authors: Ahmad Ghazal, Alain Crolotte, Tilmann Rabl, Hansarno Jacobsen, Meikel Poess, Francois Raab, Teradata CorpAbstract:There is a tremendous interest in big data by academia, industryandalarge user base. Several commercial andopen source providers unleashed a variety of products to support big data storage and processing. As these products mature, there is a need to evaluate and compare the performance of these systems. In this paper, we present BigBench, an end-to-end big data benchmark proposal. The underlying business model of BigBench is a product retailer. The proposal covers a data model and synthetic data generator that addresses the variety, velocity and volume aspects of bigdata systems containing structured, semi-structured and unstructured data. The structured part of the BigBench data model is adopted from the TPC-DS benchmark, which is enriched with semistructured and unstructured data components. The semistructured part captures registered and guest user clicks on the retailer’s website. The unstructured data captures product reviews submitted online. The data generator designed for BigBench provides scalable volumes of raw data based on a scale factor. The BigBench workload is designed around a set of queries against the data model. From a business prospective, the queries cover the different categories of big data analytics proposed by McKinsey. From a technical prospective, the queries are designed to span three different dimensions based on data sources, query processing types and analytic techniques. We illustrate the feasibility of BigBench by implementing it on the Teradata Aster Database. The test includes generating and loading a 200 Gigabyte BigBench data set and testing the workload by executing the BigBench queries (written using Teradata Aster SQL-MR)and reporting their response times. Categories and Subject Descriptors D.2.8[Software Engineering]: Metrics—performance measure
Ahmad Ghazal - One of the best experts on this subject based on the ideXlab platform.
-
Deployment of Parallel Data Warehouse Joint Design in Teradata
2016Co-Authors: Ladjel Bellatreche, Alain Crolotte, Ahmad Ghazal, Soumia Benkrid, Alfredo CuzzocreaAbstract:Abstract. Parallel databases play an important role in delivering per-formance required by data warehousing applications. Such performance can not be accomplished without a good scheme for fragmentation and allocation of data. Both problems are NP-hard problems and optimal solutions are impractical in real life situations. These two problems were addressed extensively in research and there are a lot of heuristic based efficient solutions that provide good results. In these solutions, the frag-mentation and allocation were done separately. In our previous work, we proposed a genetic solution that solves both problems simultane-ously which we refer to as a the joint solution. The joint solution was tested on some known benchmarks using internal simulations of a paral-lel database. These simulations are not reliable since it lacked the real life aspects of parallelism. This paper addresses this issue by applying our joint solution on the Teradata DBMS. We experiment the SSB bench-mark (TPC-H like) on a Teradata appliance running TD 13.10. The re-sults shows a significant improvement over previous results that performs fragmentation and allocation sequentially.
-
Adaptive Optimizations of Recursive Queries in Teradata
2014Co-Authors: Ahmad Ghazal, Alain Crolotte, Dawit Seid, Mohammed Al-katebAbstract:Recursive queries were introduced as part of ANSI SQL 99 to support processing of hierarchical data like air flight schedules, bill-of-materials, data cube dimension hierarchies, and ancestordescendant information (e.g. XML data stored in relations). Recently, recursive queries have also found extensive use in web data analysis such as social network and click stream data. Teradata implemented recursive queries in V2R6 using static plans whereby a query is executed in multiple iterations, each iteration corresponding to one level of the recursion. Such a static planning strategy may not be optimal since the demographics of intermediate results from recursive iterations often vary to a great extent. Gathering feedback at each iteration could address this problem by providing size estimates to the optimizer which, in turn, can produce an execution plan for the next iteration. However, such a full feedback scheme suffers from lack of pipelining and the inability to exploit global optimizations across the different recursion iterations. In this paper, we propose adaptive optimization techniques that avoid the issues with static as well as full feedback optimization approaches. Our approach employs a mix of multi-iteration pre-planning and dynamic feedback techniques which are generally applicable to any recursive query implementation in an RDBMS. We also validated the effectiveness of our proposed techniques by conducting experiments on a prototype implementation using a real-life social network data from the FriendFeed online blogging service
-
Oracle Corp.,
2013Co-Authors: Ahmad Ghazal, Alain Crolotte, Tilmann Rabl, Hansarno Jacobsen, Meikel Poess, Francois Raab, Teradata CorpAbstract:There is a tremendous interest in big data by academia, industryandalarge user base. Several commercial andopen source providers unleashed a variety of products to support big data storage and processing. As these products mature, there is a need to evaluate and compare the performance of these systems. In this paper, we present BigBench, an end-to-end big data benchmark proposal. The underlying business model of BigBench is a product retailer. The proposal covers a data model and synthetic data generator that addresses the variety, velocity and volume aspects of bigdata systems containing structured, semi-structured and unstructured data. The structured part of the BigBench data model is adopted from the TPC-DS benchmark, which is enriched with semistructured and unstructured data components. The semistructured part captures registered and guest user clicks on the retailer’s website. The unstructured data captures product reviews submitted online. The data generator designed for BigBench provides scalable volumes of raw data based on a scale factor. The BigBench workload is designed around a set of queries against the data model. From a business prospective, the queries cover the different categories of big data analytics proposed by McKinsey. From a technical prospective, the queries are designed to span three different dimensions based on data sources, query processing types and analytic techniques. We illustrate the feasibility of BigBench by implementing it on the Teradata Aster Database. The test includes generating and loading a 200 Gigabyte BigBench data set and testing the workload by executing the BigBench queries (written using Teradata Aster SQL-MR)and reporting their response times. Categories and Subject Descriptors D.2.8[Software Engineering]: Metrics—performance measure
-
bigbench towards an industry standard benchmark for big data analytics
International Conference on Management of Data, 2013Co-Authors: Ahmad Ghazal, Alain Crolotte, Tilmann Rabl, Meikel Poess, Minqing Hu, Francois Raab, Hansarno JacobsenAbstract:There is a tremendous interest in big data by academia, industry and a large user base. Several commercial and open source providers unleashed a variety of products to support big data storage and processing. As these products mature, there is a need to evaluate and compare the performance of these systems. In this paper, we present BigBench, an end-to-end big data benchmark proposal. The underlying business model of BigBench is a product retailer. The proposal covers a data model and synthetic data generator that addresses the variety, velocity and volume aspects of big data systems containing structured, semi-structured and unstructured data. The structured part of the BigBench data model is adopted from the TPC-DS benchmark, which is enriched with semi-structured and unstructured data components. The semi-structured part captures registered and guest user clicks on the retailer's website. The unstructured data captures product reviews submitted online. The data generator designed for BigBench provides scalable volumes of raw data based on a scale factor. The BigBench workload is designed around a set of queries against the data model. From a business prospective, the queries cover the different categories of big data analytics proposed by McKinsey. From a technical prospective, the queries are designed to span three different dimensions based on data sources, query processing types and analytic techniques. We illustrate the feasibility of BigBench by implementing it on the Teradata Aster Database. The test includes generating and loading a 200 Gigabyte BigBench data set and testing the workload by executing the BigBench queries (written using Teradata Aster SQL-MR) and reporting their response times.
-
temporal query processing in Teradata
Extending Database Technology, 2013Co-Authors: Mohammed Alkateb, Alain Crolotte, Ahmad Ghazal, Ramesh Bhashyam, Jaiprakash G Chimanchode, Sai Pavan PakalaAbstract:The importance of temporal data management is evident by the temporal features recently released in major commercial database systems. In Teradata, the temporal feature is based on the TSQL2 specification. In this paper, we present Teradata's implementation approach for temporal query processing. There are two common approaches to support temporal query processing in a database engine. One is through functional query rewrites to convert a temporal query to a semantically-equivalent non-temporal counterpart, mostly by adding time-based constraints. The other is a native support that implements temporal database operations such as scans and joins directly in the DBMS internals. These approaches have competing pros and cons. The rewrite approach is generally simpler to implement. But it adds a structural complexity to original query, which can pose a potential challenge to query optimizer and cause it to generate sub-optimal plans. A native support is expected to perform better. But it usually involves a higher cost of implementation, maintenance, and extension. We discuss why and describe how Teradata adopted the rewrite approach. In addition, we present an evaluation of our approach through a performance study conducted on a variation of the TPC-H benchmark with temporal tables and queries.
Mohammed Alkateb - One of the best experts on this subject based on the ideXlab platform.
-
in database distributed machine learning demonstration using Teradata sql engine
Very Large Data Bases, 2019Co-Authors: Sandeep Singh Sandha, Mohammed Alkateb, Sanjay Nair, Wellington Cabrera, Mani SrivastavaAbstract:Machine learning has enabled many interesting applications and is extensively being used in big data systems. The popular approach - training machine learning models in frameworks like Tensorflow, Pytorch and Keras - requires movement of data from database engines to analytical engines, which adds an excessive overhead on data scientists and becomes a performance bottleneck for model training. In this demonstration, we give a practical exhibition of a solution for the enablement of distributed machine learning natively inside database engines. During the demo, the audience will interactively use Python APIs in Jupyter Notebooks to train multiple linear regression models on synthetic regression datasets and neural network models on vision and sensory datasets directly inside Teradata SQL Engine.
-
in database distributed machine learning demonstration using Teradata sql engine
Very Large Data Bases, 2019Co-Authors: Sandeep Singh Sandha, Mohammed Alkateb, Sanjay Nair, Wellington Cabrera, Mani SrivastavaAbstract:Machine learning has enabled many interesting applications and is extensively being used in big data systems. The popular approach - training machine learning models in frameworks like Tensorflow, ...
-
joins over union all queries in Teradata demonstration of optimized execution
International Conference on Management of Data, 2018Co-Authors: Mohammed Alkateb, Paul L Sinclair, Sanjay Nair, Mark Sirek, Mohamed Y EltabakhAbstract:The UNION ALL set operator is useful for combining data from multiple sources. With the emergence and prevalence of big data ecosystems in which data is typically stored on multiple systems, UNION ALL has become even more important in many analytical queries. In this project, we demonstrate novel cost-based optimization techniques implemented in Teradata Database for join queries involving UNION ALL views and derived tables. Instead of the naive and traditional way of spooling each UNION ALL branch to a common spool prior to performing join operations, which can be prohibitively expensive, we demonstrate new techniques developed in Teradata Database including: 1) Cost-based pushing of joins into UNION ALL branches, 2) Branch grouping strategy prior to join pushing, 3) Geography adjustment of the pushed relations to avoid unnecessary redistribution or duplication, 4) Iterative join decomposition of a pushed join to multiple joins, and 5) Combining multiple join steps into a single multisource join step. In the demonstration, we use the Teradata Visual Explain tool, which offers a rich set of visual rendering capabilities of query plans, the display of various metadata information for each plan step, and several interactive UGI options for end-users.
-
optimizing union all join queries in Teradata
International Conference on Data Engineering, 2017Co-Authors: Mohammed Alkateb, Alain Crolotte, Paul L Sinclair, Sanjay NairAbstract:The UNION ALL set operator is useful for combining data from multiple sources. With the emergence of big data ecosystems in which data is typically stored on multiple systems, UNION ALL has become even more important. In this paper, we present optimization techniques implemented in Teradata Database for join queries with UNION ALL. Instead of spooling all branches of UNION ALL before performing join operations, we propose cost-based pushing of joins into branches. Join pushing not only addresses the prohibitive cost of spooling all branches, but also helps in exposing more efficient join methods (e.g., direct hash-based joins) which, otherwise, would not be considered by the query optimizer. The geography of relations being pushed to UNION ALL branches is also adjusted to avoid unnecessary redistributions and duplications of data. We conclude the paper with a performance study that demonstrates the impact of the proposed optimization techniques on query performance.
-
hybrid row column partitioning in Teradata
Very Large Data Bases, 2016Co-Authors: Mohammed Alkateb, Paul L Sinclair, Carrie BallingerAbstract:Data partitioning is an indispensable ingredient of database systems due to the performance improvement it can bring to any given mixed workload. Data can be partitioned horizontally or vertically. While some commercial proprietary and open source database systems have one flavor or mixed flavors of these partitioning forms, Teradata Database offers a unique hybrid row-column store solution that seamlessly combines both of these partitioning schemes. The key feature of this hybrid solution is that either row, column, or combined partitions are all stored and handled in the same way internally by the underlying file system storage layer. In this paper, we present the main characteristics and explain the implementation approach of Teradata's row-column store. We also discuss query optimization techniques applicable specifically to partitioned tables. Furthermore, we present a performance study that demonstrates how different partitioning options impact the performance of various queries.
Jon Page - One of the best experts on this subject based on the ideXlab platform.
-
a study of a parallel database machine and its performance the ncr Teradata dbc 1012
British National Conference on Databases, 1992Co-Authors: Jon PageAbstract:Parallel processing has long been recognised as a highly efficient method of applying large amounts of computing power to specialised applications. The world of commercial computing however, has for the most part, failed to supply suitable work to justify the creation of commercial parallel systems. The advent of the relational model, with its parallel language — SQL — and the need to hold large amounts of data, has given this opportunity. This paper introduces some of the benefit to be gained from large scale parallel hardware using the Teradata DBC/1012 database computer, currently in use at many large commercial organisations, as a prime example.
Sanjay Nair - One of the best experts on this subject based on the ideXlab platform.
-
in database distributed machine learning demonstration using Teradata sql engine
Very Large Data Bases, 2019Co-Authors: Sandeep Singh Sandha, Mohammed Alkateb, Sanjay Nair, Wellington Cabrera, Mani SrivastavaAbstract:Machine learning has enabled many interesting applications and is extensively being used in big data systems. The popular approach - training machine learning models in frameworks like Tensorflow, Pytorch and Keras - requires movement of data from database engines to analytical engines, which adds an excessive overhead on data scientists and becomes a performance bottleneck for model training. In this demonstration, we give a practical exhibition of a solution for the enablement of distributed machine learning natively inside database engines. During the demo, the audience will interactively use Python APIs in Jupyter Notebooks to train multiple linear regression models on synthetic regression datasets and neural network models on vision and sensory datasets directly inside Teradata SQL Engine.
-
in database distributed machine learning demonstration using Teradata sql engine
Very Large Data Bases, 2019Co-Authors: Sandeep Singh Sandha, Mohammed Alkateb, Sanjay Nair, Wellington Cabrera, Mani SrivastavaAbstract:Machine learning has enabled many interesting applications and is extensively being used in big data systems. The popular approach - training machine learning models in frameworks like Tensorflow, ...
-
joins over union all queries in Teradata demonstration of optimized execution
International Conference on Management of Data, 2018Co-Authors: Mohammed Alkateb, Paul L Sinclair, Sanjay Nair, Mark Sirek, Mohamed Y EltabakhAbstract:The UNION ALL set operator is useful for combining data from multiple sources. With the emergence and prevalence of big data ecosystems in which data is typically stored on multiple systems, UNION ALL has become even more important in many analytical queries. In this project, we demonstrate novel cost-based optimization techniques implemented in Teradata Database for join queries involving UNION ALL views and derived tables. Instead of the naive and traditional way of spooling each UNION ALL branch to a common spool prior to performing join operations, which can be prohibitively expensive, we demonstrate new techniques developed in Teradata Database including: 1) Cost-based pushing of joins into UNION ALL branches, 2) Branch grouping strategy prior to join pushing, 3) Geography adjustment of the pushed relations to avoid unnecessary redistribution or duplication, 4) Iterative join decomposition of a pushed join to multiple joins, and 5) Combining multiple join steps into a single multisource join step. In the demonstration, we use the Teradata Visual Explain tool, which offers a rich set of visual rendering capabilities of query plans, the display of various metadata information for each plan step, and several interactive UGI options for end-users.
-
optimizing union all join queries in Teradata
International Conference on Data Engineering, 2017Co-Authors: Mohammed Alkateb, Alain Crolotte, Paul L Sinclair, Sanjay NairAbstract:The UNION ALL set operator is useful for combining data from multiple sources. With the emergence of big data ecosystems in which data is typically stored on multiple systems, UNION ALL has become even more important. In this paper, we present optimization techniques implemented in Teradata Database for join queries with UNION ALL. Instead of spooling all branches of UNION ALL before performing join operations, we propose cost-based pushing of joins into branches. Join pushing not only addresses the prohibitive cost of spooling all branches, but also helps in exposing more efficient join methods (e.g., direct hash-based joins) which, otherwise, would not be considered by the query optimizer. The geography of relations being pushed to UNION ALL branches is also adjusted to avoid unnecessary redistributions and duplications of data. We conclude the paper with a performance study that demonstrates the impact of the proposed optimization techniques on query performance.