The Experts below are selected from a list of 2616 Experts worldwide ranked by ideXlab platform
Xiong Nan - One of the best experts on this subject based on the ideXlab platform.
-
TSM2X: High-Performance Tall-and-Skinny Matrix-Matrix Multiplication on GPUs
2020Co-Authors: Rivera Cody, Chen Jieyang, Xiong Nan, Song, Shuaiwen Leon, Tao DingwenAbstract:Linear Algebra Operations have been widely used in big data analytics and scientific computations. Many works have been done on optimizing Linear Algebra Operations on GPUs with regular-shaped input. However, few works focus on fully utilizing GPU resources when the input is not regular-shaped. Current optimizations do not consider fully utilizing the memory bandwidth and computing power; therefore, they can only achieve sub-optimal performance. In this paper, we propose two efficient algorithms---TSM2R and TSM2L---for two classes of tall-and-skinny matrix-matrix multiplications on GPUs. Both of them focus on optimizing Linear Algebra Operation with at least one of the input matrices is tall-and-skinny. Specifically, TSM2R is designed for a large regular-shaped matrix multiplying a tall-and-skinny matrix, while TSM2L is designed for a tall-and-skinny matrix multiplying a small regular-shaped matrix. We implement our proposed algorithms and test on several modern NVIDIA GPU micro-architectures. Experiments show that, compared to the current state-of-the-art works, (1) TSM2R speeds up the computation by 1.1x~3x and improves the memory bandwidth utilization and computing power utilization by 8%~47.6% and 7%~37.3%, respectively, when the regular-shaped matrix size is relatively large or medium; and (2) TSM2L speeds up the computation by 1.1x~3.5x and improve the memory bandwidth utilization by up to 55\% when the regular-shaped matrix size is relatively small.Comment: 15 pages, 14 figures, submitted to JPD
-
Optimizing Tall-and-Skinny Matrix-Matrix Multiplication on GPUs
eScholarship University of California, 2018Co-Authors: Xiong NanAbstract:Linear Algebra Operations have been widely used in big data analytics and scientific computations. Many works have been done on optimizing Linear Algebra Operations on GPUs with regular-sized input. However, few works are focusing on how to fully utilize the underlying GPU resources when the input size is not regular. Current optimizations lack of considering fully utilizing the memory bandwidth and computing power, therefore they could only achieve sub-optimal performance. In this paper, we propose a performant tall-and-skinny matrix-matrix multiplication algorithm on GPUs -- TSM2. It focuses on optimizing Linear Algebra Operation with none regular sized input. We implement the proposed algorithm and test on three different Nvidia GPU micro-architectures: Kepler, Maxwell, and Pascal. Experiments show that our TSM2 speedups the computation by 1.1x - 3x, improves memory bandwidth utilization by 8% - 47.6%, and improves computing power utilization by 7% - 37.3% comparing to the current state-of-the-art works. We replace the original matrix Operations in K-means and Algorithm-Bases Fault Tolerance (ABFT) with TSM2 and achieve up to 1.89x and 1.90x speed up
Tao Dingwen - One of the best experts on this subject based on the ideXlab platform.
-
TSM2X: High-Performance Tall-and-Skinny Matrix-Matrix Multiplication on GPUs
2020Co-Authors: Rivera Cody, Chen Jieyang, Xiong Nan, Song, Shuaiwen Leon, Tao DingwenAbstract:Linear Algebra Operations have been widely used in big data analytics and scientific computations. Many works have been done on optimizing Linear Algebra Operations on GPUs with regular-shaped input. However, few works focus on fully utilizing GPU resources when the input is not regular-shaped. Current optimizations do not consider fully utilizing the memory bandwidth and computing power; therefore, they can only achieve sub-optimal performance. In this paper, we propose two efficient algorithms---TSM2R and TSM2L---for two classes of tall-and-skinny matrix-matrix multiplications on GPUs. Both of them focus on optimizing Linear Algebra Operation with at least one of the input matrices is tall-and-skinny. Specifically, TSM2R is designed for a large regular-shaped matrix multiplying a tall-and-skinny matrix, while TSM2L is designed for a tall-and-skinny matrix multiplying a small regular-shaped matrix. We implement our proposed algorithms and test on several modern NVIDIA GPU micro-architectures. Experiments show that, compared to the current state-of-the-art works, (1) TSM2R speeds up the computation by 1.1x~3x and improves the memory bandwidth utilization and computing power utilization by 8%~47.6% and 7%~37.3%, respectively, when the regular-shaped matrix size is relatively large or medium; and (2) TSM2L speeds up the computation by 1.1x~3.5x and improve the memory bandwidth utilization by up to 55\% when the regular-shaped matrix size is relatively small.Comment: 15 pages, 14 figures, submitted to JPD
Enrique S Quintanaorti - One of the best experts on this subject based on the ideXlab platform.
-
efficient symmetric band matrix matrix multiplication on gpus
IEEE International Conference on High Performance Computing Data and Analytics, 2014Co-Authors: Ernesto Dufrechou, Pablo Ezzatti, Enrique S Quintanaorti, Alfredo RemonAbstract:Matrix-matrix multiplication is an important Linear Algebra Operation with a myriad of applications in scientific and engineering computing. Due to the relevance and inner parallelism of this Operation, there exist many high performance implementations for a variety of hardware platforms. Exploit the structure of the matrices involved in the Operation in general provides relevant time and memory savings. This is the case, e.g., when one of the matrices is a symmetric band matrix. This work presents two efficient specialized implementations of the Operation when a symmetric band matrix is involved and the target architecture contains a graphics processor (GPU). In particular, both implementations exploit the structure of the matrices to leverage the vast parallelism of the underlying hardware. The experimental results show remarkable reductions in the computation time over the tuned implementations of the same Operation provided by MKL and CUBLAS.
-
representing Linear Algebra algorithms in code the flame application program interfaces
ACM Transactions on Mathematical Software, 2005Co-Authors: Paolo Bientinesi, Enrique S Quintanaorti, Robert A Van De GeijnAbstract:In this article, we present a number of Application Program Interfaces (APIs) for coding Linear Algebra algorithms. On the surface, these APIs for the MATLAB M-script and C programming languages appear to be simple, almost trivial, extensions of those languages. Yet with them, the task of programming and maintaining families of algorithms for a broad spectrum of Linear Algebra Operations is greatly simplified. In combination with our Formal Linear Algebra Methods Environment (FLAME) approach to deriving such families of algorithms, dozens of algorithms for a single Linear Algebra Operation can be derived, verified to be correct, implemented, and tested, often in a matter of minutes per algorithm. Since the algorithms are expressed in code much like they are explained in a classroom setting, these APIs become not just a tool for implementing libraries, but also a valuable tool for teaching the algorithms that are incorporated in the libraries. In combination with an extension of the Parallel Linear Algebra Package (PLAPACK) API, the approach presents a migratory path from algorithm to MATLAB implementation to high-performance sequential implementation to parallel implementation. Finally, the APIs are being used to create a repository of algorithms and implementations for Linear Algebra Operations, the FLAME Interface REpository (FIRE), which already features hundreds of algorithms for dozens of commonly encountered Linear Algebra Operations.
Rivera Cody - One of the best experts on this subject based on the ideXlab platform.
-
TSM2X: High-Performance Tall-and-Skinny Matrix-Matrix Multiplication on GPUs
2020Co-Authors: Rivera Cody, Chen Jieyang, Xiong Nan, Song, Shuaiwen Leon, Tao DingwenAbstract:Linear Algebra Operations have been widely used in big data analytics and scientific computations. Many works have been done on optimizing Linear Algebra Operations on GPUs with regular-shaped input. However, few works focus on fully utilizing GPU resources when the input is not regular-shaped. Current optimizations do not consider fully utilizing the memory bandwidth and computing power; therefore, they can only achieve sub-optimal performance. In this paper, we propose two efficient algorithms---TSM2R and TSM2L---for two classes of tall-and-skinny matrix-matrix multiplications on GPUs. Both of them focus on optimizing Linear Algebra Operation with at least one of the input matrices is tall-and-skinny. Specifically, TSM2R is designed for a large regular-shaped matrix multiplying a tall-and-skinny matrix, while TSM2L is designed for a tall-and-skinny matrix multiplying a small regular-shaped matrix. We implement our proposed algorithms and test on several modern NVIDIA GPU micro-architectures. Experiments show that, compared to the current state-of-the-art works, (1) TSM2R speeds up the computation by 1.1x~3x and improves the memory bandwidth utilization and computing power utilization by 8%~47.6% and 7%~37.3%, respectively, when the regular-shaped matrix size is relatively large or medium; and (2) TSM2L speeds up the computation by 1.1x~3.5x and improve the memory bandwidth utilization by up to 55\% when the regular-shaped matrix size is relatively small.Comment: 15 pages, 14 figures, submitted to JPD
Robert A Van De Geijn - One of the best experts on this subject based on the ideXlab platform.
-
representing Linear Algebra algorithms in code the flame application program interfaces
ACM Transactions on Mathematical Software, 2005Co-Authors: Paolo Bientinesi, Enrique S Quintanaorti, Robert A Van De GeijnAbstract:In this article, we present a number of Application Program Interfaces (APIs) for coding Linear Algebra algorithms. On the surface, these APIs for the MATLAB M-script and C programming languages appear to be simple, almost trivial, extensions of those languages. Yet with them, the task of programming and maintaining families of algorithms for a broad spectrum of Linear Algebra Operations is greatly simplified. In combination with our Formal Linear Algebra Methods Environment (FLAME) approach to deriving such families of algorithms, dozens of algorithms for a single Linear Algebra Operation can be derived, verified to be correct, implemented, and tested, often in a matter of minutes per algorithm. Since the algorithms are expressed in code much like they are explained in a classroom setting, these APIs become not just a tool for implementing libraries, but also a valuable tool for teaching the algorithms that are incorporated in the libraries. In combination with an extension of the Parallel Linear Algebra Package (PLAPACK) API, the approach presents a migratory path from algorithm to MATLAB implementation to high-performance sequential implementation to parallel implementation. Finally, the APIs are being used to create a repository of algorithms and implementations for Linear Algebra Operations, the FLAME Interface REpository (FIRE), which already features hundreds of algorithms for dozens of commonly encountered Linear Algebra Operations.