The Experts below are selected from a list of 77187 Experts worldwide ranked by ideXlab platform

Marc Snir - One of the best experts on this subject based on the ideXlab platform.

  • numa aware shared memory Collective Communication for mpi
    High Performance Distributed Computing, 2013
    Co-Authors: Torsten Hoefler, Marc Snir
    Abstract:

    As the number of cores per node keeps increasing, it becomes increasingly important for MPI to leverage shared memory for intranode Communication. This paper investigates the design and optimizations of MPI Collectives for clusters of NUMA nodes. We develop performance models for Collective Communication using shared memory, and we develop several algorithms for various Collectives. Experiments are conducted on both Xeon X5650 and Opteron 6100 InfiniBand clusters. The measurements agree with the model and indicate that different algorithms dominate for short vectors and long vectors. We compare our shared-memory allreduce with several traditional MPI implementations -- Open MPI, MPICH2, and MVAPICH2 -- that utilize system shared memory to facilitate interprocess Communication. On a 16-node Xeon cluster and 8-node Opteron cluster, our implementation achieves on average 2.5X and 2.3X speedup over MVAPICH2, respectively. Our techniques enable an efficient implementation of Collective operations on future multi- and manycore systems.

  • generalized communicators in the message passing interface
    IEEE Transactions on Parallel and Distributed Systems, 2001
    Co-Authors: E D Demaine, Marc Snir, Ian Foster, Carl Kesselman
    Abstract:

    We propose extensions to the message passing interface (MPI) that generalize the MPI communicator concept to allow multiple Communication endpoints per process, dynamic creation of endpoints, and the transfer of endpoints between processes. The generalized communicator construct can be used to express a wide range of interesting Communication structures, including Collective Communication operations involving multiple threads per process, Communications between dynamically created threads or processes, and object-oriented applications in which Communications are directed to specific objects. Furthermore, this enriched functionality can be provided in a manner that preserves backward compatibility with MPI. We describe the proposed extensions, illustrate their use with examples, and describe a prototype implementation in the popular MPI implementation MPICH.

  • generalized communicators in the message passing interface
    Proceedings. Second MPI Developer's Conference, 1996
    Co-Authors: Ian Foster, Carl Kesselman, Marc Snir
    Abstract:

    We propose extensions to the Message Passing Interface (MPI) that generalize the MPI communicator concept to allow multiple Communication endpoints per process, dynamic creation of endpoints, and the transfer of endpoints between processes. The generalized communicator construct can be used to express a wide range of interesting Communication structures, including Collective Communication operations involving multiple threads per process, Communications between dynamically created threads, and object-oriented applications in which Communications are directed to specific objects. Furthermore, this enriched functionality can be provided in a manner that preserves backward compatibility with MPI. We describe the proposed extensions, illustrate their use with examples, and discuss implementation issues.

  • ccl a portable and tunable Collective Communication library for scalable parallel computers
    International Parallel Processing Symposium, 1994
    Co-Authors: V. Bala, Jehoshua Bruck, R. Cypher, P. Elustondo, S. Kipnis, Marc Snir
    Abstract:

    A Collective Communication library for parallel computers includes frequently used operations such as broadcast, reduce, scatter, gather, concatenate, synchronize, and shift. Such a library provides users with a convenient programming interface, efficient Communication operations, and the advantage of portability. A library of this nature, the Collective Communication library (CCL), intended for the line of scalable parallel computer products by IBM, has been designed. CCL is part of the parallel application programming interface of the recently announced IBM 9076 Scalable POWERparallel System 1 (SP1). We examine several issues related to the functionality, correctness, and performance of a portable Collective Communication library while focusing on three novel aspects in the design and implementation of CCL: (i) the introduction of process groups, (ii) the definition of semantics that ensures correctness, and (iii) the design of new and tunable algorithms based on a realistic point-to-point Communication model. >

Jehoshua Bruck - One of the best experts on this subject based on the ideXlab platform.

  • efficient message passing interface mpi for parallel computing on clusters of workstations
    Journal of Parallel and Distributed Computing, 1997
    Co-Authors: Jehoshua Bruck, Danny Dolev, Marcelcătălin Rosu, Ray Strong
    Abstract:

    Parallel computing on clusters of workstations and personal computers has very high potential, since it leverages existing hardware and software. Parallel programming environments offer the user a convenient way to express parallel computation and Communication. In fact, recently, a Message Passing Interface (MPI) has been proposed as an industrial standard for writing “portable” message-passing parallel programs. The Communication part of MPI consists of the usual point-to-point Communication as well as Collective Communication. However, existing implementations of programming environments for clusters are built on top of a point-to-point Communication layer (send and receive) over local area networks (LANs) and, as a result, suffer from poor performance in the Collective Communication part. In this paper, we present an efficient design and implementation of the Collective Communication part in MPI that is optimized for clusters of workstations. Our system consists of two main components: the MPI-CCL layer that includes the Collective Communication functionality of MPI and a User-Level Reliable Transport Protocol (URTP) that interfaces with the LAN Data-Link Layer and leverages the fact that the LAN is a broadcast medium. Our system is integrated with the operating system via an efficient kernel extension mechanism that we developed. The kernel extension significantly improves the performance of our implementation as it can handle part of the Communication overhead without involving user space. We have implemented our system on a collection of IBM RS/6000 workstations connected via a 10-Mbit Ethernet LAN. Our performance measurements are taken from typical scientific programs that run in a parallel mode by means of the MPI. The hypothesis behind our design is that the system's performance will be bounded by interactions between the kernel and user space rather than by the bandwidth delivered by the LAN Data-Link Layer. Our results indicate that the performance of our MPI Broadcast (on top of Ethernet) is about twice as fast as a recently published software implementation of broadcast on top of ATM.

  • efficient message passing interface mpi for parallel computing on clusters of workstations
    ACM Symposium on Parallel Algorithms and Architectures, 1995
    Co-Authors: Jehoshua Bruck, Danny Dolev, Marcelcătălin Rosu, Ray Strong
    Abstract:

    Parallel computing on clusters of workstations and personal computers has very high potential, since it leverages existing hardware and software. Parallel programming environments offer the user a convenient way to express parallel computation and Communication. In fact, recently, a Message Passing Interface (MPI) has been proposed as an industrial standard for writing "portable" message-passing parallel programs. The Communication part of MPI consists of the usual point-to-point Communication as well as Collective Communication. However, existing implementations of programming environments for clusters are built on top of a point-to-point Communication layer (send and receive) over local area networks (LANs) and, as a result, suffer from poor performance in the Collective Communication part. In this paper, we present an efficient design and implementation of the Collective Communication part in MPI that is optimized for clusters of workstations. Our system consists of two main components: the MPI-CCL layer that includes the Collective Communication functionality of MPI and a User-level Reliable Transport Protocol (URTP) that interfaces with the LAN Data-link layer and leverages the fact that the LAN is a broadcast medium. Our system is integrated with the operating system via an efficient kernel extension mechanism that we developed. The kernel extension significantly improves the performance of our implementation as it can handle part of the Communication overhead without involving user space. We have implemented our system on a collection of IBM RS/6000 workstations con- nected via a lOMbit Ethernet LAN. Our performance measurements are taken from typical scientific programs that run in a parallel mode by means of the MPI. The hypothesis behind our design is that system's performance will be bounded by interactions between the kernel and user space rather than by the bandwidth delivered by the LAN Data-Link Layer. Our results indicate that the performance of our MPI Broadcast (on top of Ethernet) is about twice as fast as a recently published software implementation of broadcast on top of ATM.

  • pcode an efficient and reliable Collective Communication protocol for unreliable broadcast domain
    International Parallel Processing Symposium, 1995
    Co-Authors: Jehoshua Bruck, Danny Dolev, R Orni, Ray Strong
    Abstract:

    Existing programming environments for clusters are typically built on top of a point-to-point Communication layer (send and receive) over local area networks (LANs) and, as a result, suffer from poor performance in the Collective Communication part. For example, a broadcast that is implemented using a TCP/IP protocol (which is a point-to-point protocol) over a LAN is obviously an efficient as it is not utilizing the fact that the LAN is a broadcast medium. We have observed that the main difference between a distributed computing paradigm and a message passing parallel computing paradigm is that, in a distributed environment the activity of every processor is independent while in a parallel environment the collection of the user-Communication layers in the processors can be modeled as a single global program. We have formalized the requirements by defining the notion of a correct global program. This notion provides a precise specification, of the interface between the transport layer and the user-Communication. Layer. We have developed PCODE, a new Communication protocol that is driven by a global program, and proved its correctness. We have implemented the PCODE protocol on a collection of IBM RS/6000 workstations and on a collection of Silicon Graphics Indigo workstations, both communicating via UDP broadcast. The experimental results we obtained indicate that the performance advantage of PCODE over the current point-to-point approach (TCP) can be as high as an order of magnitude on a cluster of 16 workstations. >

  • ccl a portable and tunable Collective Communication library for scalable parallel computers
    International Parallel Processing Symposium, 1994
    Co-Authors: V. Bala, Jehoshua Bruck, R. Cypher, P. Elustondo, S. Kipnis, Marc Snir
    Abstract:

    A Collective Communication library for parallel computers includes frequently used operations such as broadcast, reduce, scatter, gather, concatenate, synchronize, and shift. Such a library provides users with a convenient programming interface, efficient Communication operations, and the advantage of portability. A library of this nature, the Collective Communication library (CCL), intended for the line of scalable parallel computer products by IBM, has been designed. CCL is part of the parallel application programming interface of the recently announced IBM 9076 Scalable POWERparallel System 1 (SP1). We examine several issues related to the functionality, correctness, and performance of a portable Collective Communication library while focusing on three novel aspects in the design and implementation of CCL: (i) the introduction of process groups, (ii) the definition of semantics that ensures correctness, and (iii) the design of new and tunable algorithms based on a realistic point-to-point Communication model. >

  • the ibm external user interface for scalable parallel systems
    Parallel Computing, 1994
    Co-Authors: V. Bala, Jehoshua Bruck, R. Cypher, S. Kipnis, Raymond M Bryant, Peter De Jong, Pablo Elustondo, D Frye, Gail Irwin, Richard D Lawrence
    Abstract:

    Abstract The IBM External User Interface (EUI) for scalable parallel systems is a parallel programming library designed for the IBM line of scalable parallel computers. The first computer in this line, the IBM 9076 SP1, was announced in February 1993. In essence, the EUI is a library of coordination and Communication routines that can be invoked from within FORTRAN or C application programs. The EUI consists of four main components: task management routines, message passing routines, task group routines, and Collective Communication routines. This paper examines several aspects of the design and development of the EUI.

Ray Strong - One of the best experts on this subject based on the ideXlab platform.

  • efficient message passing interface mpi for parallel computing on clusters of workstations
    Journal of Parallel and Distributed Computing, 1997
    Co-Authors: Jehoshua Bruck, Danny Dolev, Marcelcătălin Rosu, Ray Strong
    Abstract:

    Parallel computing on clusters of workstations and personal computers has very high potential, since it leverages existing hardware and software. Parallel programming environments offer the user a convenient way to express parallel computation and Communication. In fact, recently, a Message Passing Interface (MPI) has been proposed as an industrial standard for writing “portable” message-passing parallel programs. The Communication part of MPI consists of the usual point-to-point Communication as well as Collective Communication. However, existing implementations of programming environments for clusters are built on top of a point-to-point Communication layer (send and receive) over local area networks (LANs) and, as a result, suffer from poor performance in the Collective Communication part. In this paper, we present an efficient design and implementation of the Collective Communication part in MPI that is optimized for clusters of workstations. Our system consists of two main components: the MPI-CCL layer that includes the Collective Communication functionality of MPI and a User-Level Reliable Transport Protocol (URTP) that interfaces with the LAN Data-Link Layer and leverages the fact that the LAN is a broadcast medium. Our system is integrated with the operating system via an efficient kernel extension mechanism that we developed. The kernel extension significantly improves the performance of our implementation as it can handle part of the Communication overhead without involving user space. We have implemented our system on a collection of IBM RS/6000 workstations connected via a 10-Mbit Ethernet LAN. Our performance measurements are taken from typical scientific programs that run in a parallel mode by means of the MPI. The hypothesis behind our design is that the system's performance will be bounded by interactions between the kernel and user space rather than by the bandwidth delivered by the LAN Data-Link Layer. Our results indicate that the performance of our MPI Broadcast (on top of Ethernet) is about twice as fast as a recently published software implementation of broadcast on top of ATM.

  • efficient message passing interface mpi for parallel computing on clusters of workstations
    ACM Symposium on Parallel Algorithms and Architectures, 1995
    Co-Authors: Jehoshua Bruck, Danny Dolev, Marcelcătălin Rosu, Ray Strong
    Abstract:

    Parallel computing on clusters of workstations and personal computers has very high potential, since it leverages existing hardware and software. Parallel programming environments offer the user a convenient way to express parallel computation and Communication. In fact, recently, a Message Passing Interface (MPI) has been proposed as an industrial standard for writing "portable" message-passing parallel programs. The Communication part of MPI consists of the usual point-to-point Communication as well as Collective Communication. However, existing implementations of programming environments for clusters are built on top of a point-to-point Communication layer (send and receive) over local area networks (LANs) and, as a result, suffer from poor performance in the Collective Communication part. In this paper, we present an efficient design and implementation of the Collective Communication part in MPI that is optimized for clusters of workstations. Our system consists of two main components: the MPI-CCL layer that includes the Collective Communication functionality of MPI and a User-level Reliable Transport Protocol (URTP) that interfaces with the LAN Data-link layer and leverages the fact that the LAN is a broadcast medium. Our system is integrated with the operating system via an efficient kernel extension mechanism that we developed. The kernel extension significantly improves the performance of our implementation as it can handle part of the Communication overhead without involving user space. We have implemented our system on a collection of IBM RS/6000 workstations con- nected via a lOMbit Ethernet LAN. Our performance measurements are taken from typical scientific programs that run in a parallel mode by means of the MPI. The hypothesis behind our design is that system's performance will be bounded by interactions between the kernel and user space rather than by the bandwidth delivered by the LAN Data-Link Layer. Our results indicate that the performance of our MPI Broadcast (on top of Ethernet) is about twice as fast as a recently published software implementation of broadcast on top of ATM.

  • pcode an efficient and reliable Collective Communication protocol for unreliable broadcast domain
    International Parallel Processing Symposium, 1995
    Co-Authors: Jehoshua Bruck, Danny Dolev, R Orni, Ray Strong
    Abstract:

    Existing programming environments for clusters are typically built on top of a point-to-point Communication layer (send and receive) over local area networks (LANs) and, as a result, suffer from poor performance in the Collective Communication part. For example, a broadcast that is implemented using a TCP/IP protocol (which is a point-to-point protocol) over a LAN is obviously an efficient as it is not utilizing the fact that the LAN is a broadcast medium. We have observed that the main difference between a distributed computing paradigm and a message passing parallel computing paradigm is that, in a distributed environment the activity of every processor is independent while in a parallel environment the collection of the user-Communication layers in the processors can be modeled as a single global program. We have formalized the requirements by defining the notion of a correct global program. This notion provides a precise specification, of the interface between the transport layer and the user-Communication. Layer. We have developed PCODE, a new Communication protocol that is driven by a global program, and proved its correctness. We have implemented the PCODE protocol on a collection of IBM RS/6000 workstations and on a collection of Silicon Graphics Indigo workstations, both communicating via UDP broadcast. The experimental results we obtained indicate that the performance advantage of PCODE over the current point-to-point approach (TCP) can be as high as an order of magnitude on a cluster of 16 workstations. >

V. Bala - One of the best experts on this subject based on the ideXlab platform.

  • CCL: a portable and tunable Collective Communication library for scalable parallel computers
    IEEE Transactions on Parallel and Distributed Systems, 1995
    Co-Authors: V. Bala, J. Bruck, R. Cypher, P. Elustondo, A. Ho, Ching-tien Ho, S. Kipnis, M. Snir
    Abstract:

    A Collective Communication library for parallel computers includes frequently used operations such as broadcast, reduce, scatter, gather, concatenate, synchronize, and shift. Such a library provides users with a convenient programming interface, efficient Communication operations, and the advantage of portability. A library of this nature, the Collective Communication Library (CCL), intended for the line of scalable parallel computer products by IBM, has been designed. CCL is part of the parallel application programming interface of the recently announced IBM 9076 Scalable POWERparallel System 1 (SP1). In this paper, we examine several issues related to the functionality, correctness, and performance of a portable Collective Communication library while focusing on three novel aspects in the design and implementation of CCL: 1) the introduction of process groups, 2) the definition of semantics that ensures correctness, and 3) the design of new and tunable algorithms based on a realistic point-to-point Communication model.

  • ccl a portable and tunable Collective Communication library for scalable parallel computers
    International Parallel Processing Symposium, 1994
    Co-Authors: V. Bala, Jehoshua Bruck, R. Cypher, P. Elustondo, S. Kipnis, Marc Snir
    Abstract:

    A Collective Communication library for parallel computers includes frequently used operations such as broadcast, reduce, scatter, gather, concatenate, synchronize, and shift. Such a library provides users with a convenient programming interface, efficient Communication operations, and the advantage of portability. A library of this nature, the Collective Communication library (CCL), intended for the line of scalable parallel computer products by IBM, has been designed. CCL is part of the parallel application programming interface of the recently announced IBM 9076 Scalable POWERparallel System 1 (SP1). We examine several issues related to the functionality, correctness, and performance of a portable Collective Communication library while focusing on three novel aspects in the design and implementation of CCL: (i) the introduction of process groups, (ii) the definition of semantics that ensures correctness, and (iii) the design of new and tunable algorithms based on a realistic point-to-point Communication model. >

  • the ibm external user interface for scalable parallel systems
    Parallel Computing, 1994
    Co-Authors: V. Bala, Jehoshua Bruck, R. Cypher, S. Kipnis, Raymond M Bryant, Peter De Jong, Pablo Elustondo, D Frye, Gail Irwin, Richard D Lawrence
    Abstract:

    Abstract The IBM External User Interface (EUI) for scalable parallel systems is a parallel programming library designed for the IBM line of scalable parallel computers. The first computer in this line, the IBM 9076 SP1, was announced in February 1993. In essence, the EUI is a library of coordination and Communication routines that can be invoked from within FORTRAN or C application programs. The EUI consists of four main components: task management routines, message passing routines, task group routines, and Collective Communication routines. This paper examines several aspects of the design and development of the EUI.

  • process groups a mechanism for the coordination of and Communication among processes in the venus Collective Communication library
    International Parallel Processing Symposium, 1993
    Co-Authors: V. Bala, S. Kipnis
    Abstract:

    In programming massively parallel computers, it is often necessary to have sets of processes cooperate in performing certain computations and Communications. Most run-time libraries require that such sets of processes be explicitly specified in the program. In the Venus run-time Communication library however, a Process Group abstraction is used to enable implicit coordination of and Communication over dynamically determined sets of processes. The Process Groups mechanism in Venus offers an object-oriented approach for handling sets of processes and enhances the debugging and monitoring of programs. The authors describe the Process Groups mechanism in Venus, illustrate its use on the class of N-body problems, and outline some of the data structures and algorithms used to implement this mechanism in Venus. >

Dhabaleswar K. Panda - One of the best experts on this subject based on the ideXlab platform.

  • designing power aware Collective Communication algorithms for infiniband clusters
    International Conference on Parallel Processing, 2010
    Co-Authors: Krishna Kandalla, Emilio P Mancini, Sayantan Sur, Dhabaleswar K. Panda
    Abstract:

    Modern supercomputing systems have witnessed a phenomenal growth in the recent history owing to the advent of multi-core architectures and high speed networks. However, the operational and maintenance costs of these systems have also grown rapidly. Several concepts such as Dynamic Voltage and Frequency Scaling (DVFS) and CPU Throttling have been proposed to conserve the power consumed by the compute nodes during idle periods. However, it is necessary to design software stacks in a power-aware manner to minimize the amount of power drawn by the system during the execution of applications. It is also critical to minimize the performance overheads associated with power-aware algorithms, as the benefits of saving power could be lost if the application runs for a longer time. Modern multi-core architectures such as the Intel “Nehalem” allow for DVFS and CPU throttling operations to be performed with little overheads. In this paper, we explore how these features can be leveraged to design algorithms to deliver fine-grained power savings during the Communication phases of parallel applications. We also propose a theoretical model to analyze the power consumption characteristics of Communication operations. We use microbenchmarks and application benchmarks such as NAS and CPMD to measure the performance of our proposed algorithms and to demonstrate the potential for saving power with 32 and 64 processes. We observe about 8% improvement in the overall energy consumed by these applications with little performance overheads.

  • designing topology aware Collective Communication algorithms for large scale infiniband clusters case studies with scatter and gather
    IEEE International Symposium on Parallel & Distributed Processing Workshops and Phd Forum, 2010
    Co-Authors: Krishna Kandalla, Abhinav Vishnu, Hari Subramoni, Dhabaleswar K. Panda
    Abstract:

    Modern high performance computing systems are being increasingly deployed in a hierarchical fashion with multi-core computing platforms forming the base of the hierarchy. These systems are usually comprised of multiple racks, with each rack consisting of a finite number of chassis, and each chassis having multiple compute nodes or blades, based on multi-core architectures. The networks are also hierarchical with multiple levels of switches. Message exchange operations between processes that belong to different racks involve multiple hops across different switches and this directly affects the performance of Collective operations. In this paper, we take on the challenges involved in detecting the topology of large scale InfiniBand clusters and leveraging this knowledge to design efficient topology-aware algorithms for Collective operations. We also propose a Communication model to analyze the Communication costs involved in Collective operations on large scale supercomputing systems. We have analyzed the performance characteristics of two Collectives, MPI_Gather and MPI_Scatter, on such systems and we have proposed topology-aware algorithms for these operations. Our experimental results have shown that the proposed algorithms can improve the performance of these Collective operations by almost 54% at the micro-benchmark level.

  • high performance rdma based mpi implementation over infiniband
    International Conference on Supercomputing, 2003
    Co-Authors: Jiuxing Liu, P Wyckoff, Sushmitha P Kini, Dhabaleswar K. Panda
    Abstract:

    Although InfiniBand Architecture is relatively new in the high performance computing area, it offers many features which help us to improve the performance of Communication subsystems. One of these features is Remote Direct Memory Access (RDMA) operations. In this paper, we propose a new design of MPI over InfiniBand which brings the benefit of RDMA to not only large messages, but also small and control messages. We also achieve better scalability by exploiting application Communication pattern and combining send/receive operations with RDMA operations. Our RDMA-based MPI implementation currently delivers a latency of 6.8 microseconds for small messages and a peak bandwidth of 871 Million Bytes (831 Mega Bytes) per second. Performance evaluation at the MPI level shows that for small messages, our RDMA-based design can reduce the latency by 24%, increase the bandwidth by over 104%, and reduce the host overhead by up to 22%. For large messages, we improve performance by reducing the time for transferring control messages. We have also shown that our new design is beneficial to MPI Collective Communication and NAS Parallel Benchmarks.

  • application bypass broadcast in mpich over gm
    Cluster Computing and the Grid, 2003
    Co-Authors: Darius Buntinas, Dhabaleswar K. Panda, Ron Brightwell
    Abstract:

    Processes of a parallel program can become unsynchronized, or skewed, during the course of running an application. Processes can become skewed as a result of unbalanced or asymmetric rode, or through the use of heterogeneous systems, where nodes in the system have different performance characteristics, as well as random, unpredictable effects such as the processes not being started at exactly the same time, or processors receiving interrupts during computation. Geographically distributed systems may have more severe skew because of variable Communication times. Such skew can have a significant impact on the performance of Collective Communication operations which impose an implicit synchronization. The broadcast operation in MPICH is one such operation. An application-bypass broadcast operation is one which does not depend on the application running at a process to make progress. Such an operation would not be as sensitive to process skew. This paper describes the design and implementation of an application-bypass broadcast operation. We evaluated the implementation and find a factor of improvement of up to 16 for application-bypass broadcast compared to non-application-bypass broadcast when processes are skewed. Furthermore we see that as the system size increases, the effects of skew on non-application-bypass broadcast also increase. The application-bypass broadcast is much less sensitive to process skew which makes it more scalable than the non-application-bypass broadcast operation.

  • fast Collective operations using shared and remote memory access protocols on clusters
    International Parallel and Distributed Processing Symposium, 2003
    Co-Authors: Vinod Tipparaju, Jarek Nieplocha, Dhabaleswar K. Panda
    Abstract:

    This paper describes a novel methodology for implementing a common set of Collective Communication operations on clusters based on symmetric multiprocessor (SMP) nodes. Called Shared-Remote-Memory Collectives, or SRM, our approach replaces the point-to-point message passing, traditionally used in implementation of Collective message-passing operations, with a combination of shared and remote memory access (RMA) protocols that are used to implement semantics of the Collective operations directly. Appropriate embedding of the Communication graphs in a cluster maximizes the use of shared memory and reduces network Communication. Substantial performance improvements are achieved over the highly optimized commercial IBM implementation and the open-source MPICH implementation of MPI across a wide range of message sizes on the IBM SP. For example, depending on the message size and number of processors, SRM implementation of broadcast, reduce, and barrier outperforms IBM MPI/spl I.bar/Bcast by 27-84%, MPI/spl I.bar/Reduce by 24-79%, and MPI/spl I.bar/Barrier by 73% on 256 processors, respectively.