The Experts below are selected from a list of 32466 Experts worldwide ranked by ideXlab platform
Vipin Chaudhary - One of the best experts on this subject based on the ideXlab platform.
-
Design issues for a high-performance distributed Shared Memory on symmetrical multiprocessor clusters
Cluster Computing, 1999Co-Authors: Vipin ChaudharyAbstract:Clusters of Symmetrical Multiprocessors (SMPs) have recently become the norm for high-performance economical computing solutions. Multiple nodes in a cluster can be used for parallel programming using a message passing library. An alternate approach is to use a software Distributed Shared Memory (DSM) to provide a view of Shared Memory to the application programmer. This paper describes Strings, a high performance distributed Shared Memory System designed for such SMP clusters. The distinguishing feature of this System is the use of a fully multi-threaded runtime System, using kernel level threads. Strings allows multiple application threads to be run on each node in a cluster. Since most modern UNIX Systems can multiplex these threads on kernel level light weight processes, applications written using Strings can exploit multiple processors on a SMP machine. This paper describes some of the architectural details of the System and illustrates the performance improvements with benchmark programs from the SPLASH-2 suite, some computational kernels as well as a full fledged application. It is found that using multiple processes on SMP nodes provides good speedups only for a few of the programs. Multiple application threads can improve the performance in some cases, but other programs show a slowdown. If kernel threads are used additionally, the overall performance improves significantly in all programs tested. Other design decisions also have a beneficial impact, though to a lesser degree.
-
strings a high performance distributed Shared Memory for symmetrical multiprocessor clusters
High Performance Distributed Computing, 1998Co-Authors: Sumit Roy, Vipin ChaudharyAbstract:Introduces Strings, a high-performance distributed Shared Memory System designed for clusters of symmetrical multiprocessors (SMPs). The distinguishing feature of this System is the use of a fully multithreaded runtime System, written using POSIX threads. Strings also allows multiple application threads to be run on each node in a cluster. Since most modern UNIX Systems can multiplex these threads on kernel-level lightweight processes, applications written using Strings can use all the processors in a SMP machine. This paper describes some of the architectural details of the System and analyzes the performance improvements with two example programs and a few benchmark programs from the SPLASH-2 suite.
Jack Dongarra - One of the best experts on this subject based on the ideXlab platform.
-
lu factorization with partial pivoting for a multicore System with accelerators
IEEE Transactions on Parallel and Distributed Systems, 2013Co-Authors: Jakub Kurzak, Piotr Luszczek, Mathieu Faverge, Jack DongarraAbstract:LU factorization with partial pivoting is a canonical numerical procedure and the main component of the high performance LINPACK benchmark. This paper presents an implementation of the algorithm for a hybrid, Shared Memory, System with standard CPU cores and GPU accelerators. The difficulty of implementing the algorithm for such a System lies in the disproportion between the computational power of the CPUs, compared to the GPUs, and in the meager bandwidth of the communication link between their Memory Systems. An additional challenge comes from the complexity of the Memory-bound and synchronization-rich nature of the panel factorization component of the block LU algorithm, imposed by the use of partial pivoting. The challenges are tackled with the use of a data layout geared toward complex Memory hierarchies, autotuning of GPU kernels, fine-grain parallelization of Memory-bound CPU operations and dynamic scheduling of tasks to different devices. Performance in excess of one TeraFLOPS is achieved using four AMD Magny Cours CPUs and four NVIDIA Fermi GPUs.
-
LU Factorization with Partial Pivoting for a Multi-CPU, Multi-GPU Shared Memory System
2012Co-Authors: Jakub Kurzak, Piotr Luszczek, Mathieu Faverge, Jack DongarraAbstract:LU factorization with partial pivoting is a canonical numerical procedure and the main component of the High Performance Linpack benchmark. This article presents an implementation of the algorithm for a hybrid, Shared Memory, System with standard CPU cores and GPU accelerators. The optimizations include lookahead, dynamic task scheduling, fine grain parallelism for Memory-bound operations, autotuning, and data layout geared towards complex Memory hierarchies. Performance in excess of one Tera flop/s is achieved using four AMD Magny Cours CPUs and four NVIDIA Fermi GPUs.
-
programming the lu factorization for a multicore System with accelerators
IEEE International Conference on High Performance Computing Data and Analytics, 2012Co-Authors: Jakub Kurzak, Piotr Luszczek, Mathieu Faverge, Jack DongarraAbstract:LU factorization with partial pivoting is a canonical numerical procedure and the main component of the High Performance LINPACK benchmark. This article presents an implementation of the algorithm for a hybrid, Shared Memory, System with standard CPU cores and GPU accelerators. Performance in excess of one TeraFLOPS is achieved using four AMD Magny Cours CPUs and four NVIDIA Fermi GPUs.
-
a comprehensive study of task coalescing for selecting parallelism granularity in a two stage bidiagonal reduction
International Parallel and Distributed Processing Symposium, 2012Co-Authors: Azzam Haidar, Piotr Luszczek, Hatem Ltaief, Jack DongarraAbstract:We present new high performance numerical kernels combined with advanced optimization techniques that significantly increase the performance of parallel bidiagonal reduction. Our approach is based on developing efficient fine-grained computational tasks as well as reducing overheads associated with their high-level scheduling during the so-called bulge chasing procedure that is an essential phase of a scalable bidiagonalization procedure. In essence, we coalesce multiple tasks in a way that reduces the time needed to switch execution context between the scheduler and useful computational tasks. At the same time, we maintain the crucial information about the tasks and their data dependencies between the coalescing groups. This is the necessary condition to preserve numerical correctness of the computation. We show our annihilation strategy based on multiple applications of single orthogonal reflectors. Despite non-trivial characteristics in computational complexity and Memory access patterns, our optimization approach smoothly applies to the annihilation scenario. The coalescing positively influences another equally important aspect of the bulge chasing stage: the Memory reuse. For the tasks within the coalescing groups, the data is retained in high levels of the cache hierarchy and, as a consequence, operations that are normally Memory-bound increase their ratio of computation to off-chip communication and become compute-bound which renders them amenable to efficient execution on multicore architectures. The performance for the new two-stage bidiagonal reduction is staggering. Our implementation results in up to 50-fold and 12-fold improvement (~130 Gflop/s) compared to the equivalent routines from LAPACK V3.2 and Intel MKL V10.3, respectively, on an eight socket hexa-core AMD Opteron multicore Shared-Memory System with a matrix size of 24000 × 24000. Last but not least, we provide a comprehensive study on the impact of the coalescing group size in terms of cache utilization and power consumption in the context of this new two-stage bidiagonal reduction.
-
parallel reduction to condensed forms for symmetric eigenvalue problems using aggregated fine grained and Memory aware kernels
IEEE International Conference on High Performance Computing Data and Analytics, 2011Co-Authors: Azzam Haidar, Hatem Ltaief, Jack DongarraAbstract:This paper introduces a novel implementation in reducing a symmetric dense matrix to tridiagonal form, which is the preprocessing step toward solving symmetric eigenvalue problems. Based on tile algorithms, the reduction follows a two-stage approach, where the tile matrix is first reduced to symmetric band form prior to the final condensed structure. The challenging trade-off between algorithmic performance and task granularity has been tackled through a grouping technique, which consists of aggregating fine-grained and Memory-aware computational tasks during both stages, while sustaining the application's overall high performance. A dynamic runtime environment System then schedules the different tasks in an out-of-order fashion. The performance for the tridiagonal reduction reported in this paper is unprecedented. Our implementation results in up to 50-fold and 12-fold improvement (130 Gflop/s) compared to the equivalent routines from LAPACK V3.2 and Intel MKL V10.3, respectively, on an eight socket hexa-core AMD Opteron multicore Shared-Memory System with a matrix size of 24000 x 24000.
Jakub Kurzak - One of the best experts on this subject based on the ideXlab platform.
-
LU Factorization with Partial Pivoting for a Multi-CPU, Multi-GPU Shared Memory System - eScholarship
2014Co-Authors: Jakub KurzakAbstract:LU Factorization with Partial Pivoting for a Multi-CPU, Multi-GPU Shared Memory System – LAPACK Working Note 266 Jakub Kurzak Piotr Luszczek Mathieu Faverge Electrical Engineering and Computer Science, University of Tennessee Jack Dongarra Electrical Engineering and Computer Science, University of Tennessee Computer Science and Mathematics Division, Oak Ridge National Laboratory School of Mathematics & School of Computer Science, University of Manchester Abstract LU factorization with partial pivoting is a canonical numerical procedure and the main component of the High Performance LINPACK benchmark. This article presents an implementation of the algorithm for a hybrid, Shared Memory, System with standard CPU cores and GPU accelerators. Performance in excess of one TeraFLOPS is achieved using four AMD Magny Cours CPUs and four NVIDIA Fermi GPUs. Introduction This paper presents an implementation of the canonical formulation of the LU factorization, which relies on partial (row) pivoting for numerical stability. It is equivalent to the DGETRF function in the LAPACK numerical library. Since the algorithm is coded in double precision, it can serve as the basis for an implementa- tion of the High Performance LINPACK benchmark (HPL) [1]. The target platform is a hybrid, multi-CPU, multi-GPU Shared Memory System. Background The LAPACK block LU factorization is the main point of reference here, and LAPACK naming convention is followed. The LU factorization of a matrix M has the form M = PLU, where L is a unit lower triangular matrix, U is an upper
-
lu factorization with partial pivoting for a multicore System with accelerators
IEEE Transactions on Parallel and Distributed Systems, 2013Co-Authors: Jakub Kurzak, Piotr Luszczek, Mathieu Faverge, Jack DongarraAbstract:LU factorization with partial pivoting is a canonical numerical procedure and the main component of the high performance LINPACK benchmark. This paper presents an implementation of the algorithm for a hybrid, Shared Memory, System with standard CPU cores and GPU accelerators. The difficulty of implementing the algorithm for such a System lies in the disproportion between the computational power of the CPUs, compared to the GPUs, and in the meager bandwidth of the communication link between their Memory Systems. An additional challenge comes from the complexity of the Memory-bound and synchronization-rich nature of the panel factorization component of the block LU algorithm, imposed by the use of partial pivoting. The challenges are tackled with the use of a data layout geared toward complex Memory hierarchies, autotuning of GPU kernels, fine-grain parallelization of Memory-bound CPU operations and dynamic scheduling of tasks to different devices. Performance in excess of one TeraFLOPS is achieved using four AMD Magny Cours CPUs and four NVIDIA Fermi GPUs.
-
LU Factorization with Partial Pivoting for a Multi-CPU, Multi-GPU Shared Memory System
2012Co-Authors: Jakub Kurzak, Piotr Luszczek, Mathieu Faverge, Jack DongarraAbstract:LU factorization with partial pivoting is a canonical numerical procedure and the main component of the High Performance Linpack benchmark. This article presents an implementation of the algorithm for a hybrid, Shared Memory, System with standard CPU cores and GPU accelerators. The optimizations include lookahead, dynamic task scheduling, fine grain parallelism for Memory-bound operations, autotuning, and data layout geared towards complex Memory hierarchies. Performance in excess of one Tera flop/s is achieved using four AMD Magny Cours CPUs and four NVIDIA Fermi GPUs.
-
programming the lu factorization for a multicore System with accelerators
IEEE International Conference on High Performance Computing Data and Analytics, 2012Co-Authors: Jakub Kurzak, Piotr Luszczek, Mathieu Faverge, Jack DongarraAbstract:LU factorization with partial pivoting is a canonical numerical procedure and the main component of the High Performance LINPACK benchmark. This article presents an implementation of the algorithm for a hybrid, Shared Memory, System with standard CPU cores and GPU accelerators. Performance in excess of one TeraFLOPS is achieved using four AMD Magny Cours CPUs and four NVIDIA Fermi GPUs.
Yale N Patt - One of the best experts on this subject based on the ideXlab platform.
-
Fairness via source throttling: A configurable and high-performance fairness substrate for multi-core Memory Systems
2012Co-Authors: Eiman Ebrahimi, Chang Joo Lee, Onur Mutlu, Yale N PattAbstract:Cores in chip-multiprocessors (CMPs) share multiple Memory subSystem resources. If resource sharing is unfair, some applications can be delayed significantly while others are unfairly prioritized. Previous research proposed separate fairness mechanisms for each resource. Such resource-based fairness mechanisms implemented independently in each resource can make contradictory decisions, leading to low fairness and performance loss. Therefore, a coordinated mechanism that provides fairness in the entire Shared Memory System is desirable. This article proposes a new approach that provides fairness in the entire Shared Memory System, thereby eliminating the need for and complexity of developing fairness mechanisms for each resource. Our technique, Fairness via Source Throttling (FST), estimates unfairness in the entire Memory System. If unfairness is above a System-software-set threshold, FST throttles down cores causing unfairness by limiting the number of requests they create and the frequency at which they do. As such, our source-based fairness control ensures fairness decisions are made in tandem in the entire Memory System. FST enforces thread priorities/weights, and enables System-software to enforce different fairness objectives in the Memory System. Our evaluations show that FST provides the best System fairness and performance compared t
-
fairness via source throttling a configurable and high performance fairness substrate for multi core Memory Systems
Architectural Support for Programming Languages and Operating Systems, 2010Co-Authors: Eiman Ebrahimi, Chang Joo Lee, Onur Mutlu, Yale N PattAbstract:Cores in a chip-multiprocessor (CMP) System share multiple hardware resources in the Memory subSystem. If resource sharing is unfair, some applications can be delayed significantly while others are unfairly prioritized. Previous research proposed separate fairness mechanisms in each individual resource. Such resource-based fairness mechanisms implemented independently in each resource can make contradictory decisions, leading to low fairness and loss of performance. Therefore, a coordinated mechanism that provides fairness in the entire Shared Memory System is desirable. This paper proposes a new approach that provides fairness in the entire Shared Memory System, thereby eliminating the need for and complexity of developing fairness mechanisms for each individual resource. Our technique, Fairness via Source Throttling (FST), estimates the unfairness in the entire Shared Memory System. If the estimated unfairness is above a threshold set by System software, FST throttles down cores causing unfairness by limiting the number of requests they can inject into the System and the frequency at which they do. As such, our source-based fairness control ensures fairness decisions are made in tandem in the entire Memory System. FST also enforces thread priorities/weights, and enables System software to enforce different fairness objectives and fairness-performance tradeoffs in the Memory System. Our evaluations show that FST provides the best System fairness and performance compared to four Systems with no fairness control and with state-of-the-art fairness mechanisms implemented in both Shared caches and Memory controllers.
John L Hennessy - One of the best experts on this subject based on the ideXlab platform.
-
softflash analyzing the performance of clustered distributed virtual Shared Memory
Architectural Support for Programming Languages and Operating Systems, 1996Co-Authors: Andrew Erlichson, Neal T Nuckolls, Greg Chesson, John L HennessyAbstract:One potentially attractive way to build large-scale Shared-Memory machines is to use small-scale to medium-scale Shared-Memory machines as clusters that are interconnected with an off-the-shelf network. To create a Shared-Memory programming environment across the clusters, it is possible to use a virtual Shared-Memory software layer. Because of the low latency and high bandwidth of the interconnect available within each cluster, there are clear advantages in making the clusters as large as possible. The critical question then becomes whether the latency and bandwidth of the top-level network and the software System are sufficient to support the communication demands generated by the clusters.To explore these questions, we have built an aggressive kernel implementation of a virtual Shared-Memory System using SGI multiprocessors and 100Mbyte/sec HIPPI interconnects. The System obtains speedups on 32 processors (four nodes, eight processors per node plus additional reserved protocol processors) that range from 6.9 on the communication-intensive FFT program to 21.6 on Ocean (both from the SPLASH 2 suite). In general, clustering is effective in reducing internode miss rates, but as the cluster size increases, increases in the remote latency, mostly due to increased TLB synchronization cost, offset the advantages. For communication-intensive applications, such as FFT, the overhead of sending out network requests, the limited network bandwidth, and the long network latency prevent the achievement of good performance. Overall, this approach still appears promising, but our results indicate that large low latency networks may be needed to make cluster-based virtual Shared-Memory machines broadly useful as large-scale Shared-Memory multiprocessors.