The Experts below are selected from a list of 13263 Experts worldwide ranked by ideXlab platform
William Gropp - One of the best experts on this subject based on the ideXlab platform.
-
using advanced mpi modern features of the message passing interface
2014Co-Authors: William Gropp, Torsten Hoefler, Rajeev Thakur, Ewing LuskAbstract:This book offers a practical guide to the advanced features of the MPI (message-passing interface) standard library for writing programs for parallel computers. It covers new features added in MPI-3, the latest version of the MPI standard, and updates from MPI-2. Like its companion volume, Using MPI, the book takes an informal, example-driven, tutorial approach. The material in each chapter is organized according to the complexity of the programs used as examples, starting with the simplest example and moving to more complex ones. Using Advanced MPI covers major changes in MPI-3, including changes to remote memory access and one-sided communication that simplify semantics and enable better performance on modern hardware; new features such as nonblocking and neighborhood collectives for greater scalability on large systems; and minor updates to parallel I/O and dynamic processes. It also covers support for hybrid shared-memory/message-passing programming; MPI_message, which aids in certain types of multithreaded programming; features that handle very large data; an interface that allows the programmer and the developer to access performance data; and a new binding of MPI to Fortran.
-
The MPI message-passing interface Standard: Overview and Status
High Performance Computing - Technology Methods and Applications, 2014Co-Authors: Ewing Lusk, William GroppAbstract:MPI (message-passing Inteface) has been developed over the last two years as a standard message-passing interface specification. This paper summarizes what MPI is, describes recent activities, particularly MPI implementation activities, and supplies sources for further information about MPI.
-
Optimizing the Synchronization Operations in message passing interface One-Sided Communication
The International Journal of High Performance Computing Applications, 2005Co-Authors: Rajeev Thakur, William Gropp, Brian ToonenAbstract:One-sided communication in message passing interface (MPI) requires the use of one of three different synchronization mechanisms, which indicate when the one-sided operation can be started and when the operation is completed. Efficient implementation of the synchronization mechanisms is critical to achieving good performance with one-sided communication. However, our performance measurements indicate that in many MPI implementations, the synchronization functions add significant overhead, resulting in one-sided communication performing much worse than point-to-point communication for short- and medium-sized messages. In this paper, we describe our efforts to minimize the overhead of synchronization in our implementation of one-sided communication in MPICH2. We describe our optimizations for all three synchronization mechanisms defined in MPI: fence, post-start-complete-wait, and lock-unlock. Our performance results demonstrate that, for short messages, MPICH2 performs six times faster than LAM for fence synchronization and 50% faster for post-start-complete-wait synchronization, and it performs more than twice as fast as Sun MPI for all three synchronization methods.
-
fault tolerance in message passing interface programs
IEEE International Conference on High Performance Computing Data and Analytics, 2004Co-Authors: William Gropp, Ewing LuskAbstract:In this paper we examine the topic of writing fault-tolerant message passing interface (MPI) applications. We discuss the meaning of fault tolerance in general and what the MPI Standard has to say about it. We survey several approaches to this problem, namely checkpointing, restructuring a class of standard MPI programs, modifying MPI semantics, and extending the MPI specification. We conclude that, within certain constraints, MPI can provide a useful context for writing application programs that exhibit significant degrees of fault tolerance.
-
Improving the Performance of MPI Derived Datatypes
MPI Software Technology Press, 1999Co-Authors: William Gropp, Ewing Lusk, Debbie SwiderAbstract:The message passing interface (MPI) standard provides a powerful mechanism for describing non-contiguous memory locations: derived datatypes. In addition, MPI derived datatypes have a key role in the MPI-2 I/O operations. In principle, MPI derived datatypes allow a user to more efficiently communicate noncontiguous data (for example, strided data) because the MPI implementation can move the data without any intermediate copies to or from a contiguous buffer. In practice, however, few MPI implementations provide support for datatypes that performs better than what the user can achieve by manually packing and unpacking the data into contiguous buffers before calling MPI routines with contiguous memory regions. We develop a taxonomy of MPI datatypes according to their memory reference patterns and show how to efficiently implement these patterns. The effectiveness of this approach is illustrated on a variety of platforms. 1 Introduction The message passing interface (MPI) standard [4, 1]..
Ewing Lusk - One of the best experts on this subject based on the ideXlab platform.
-
using advanced mpi modern features of the message passing interface
2014Co-Authors: William Gropp, Torsten Hoefler, Rajeev Thakur, Ewing LuskAbstract:This book offers a practical guide to the advanced features of the MPI (message-passing interface) standard library for writing programs for parallel computers. It covers new features added in MPI-3, the latest version of the MPI standard, and updates from MPI-2. Like its companion volume, Using MPI, the book takes an informal, example-driven, tutorial approach. The material in each chapter is organized according to the complexity of the programs used as examples, starting with the simplest example and moving to more complex ones. Using Advanced MPI covers major changes in MPI-3, including changes to remote memory access and one-sided communication that simplify semantics and enable better performance on modern hardware; new features such as nonblocking and neighborhood collectives for greater scalability on large systems; and minor updates to parallel I/O and dynamic processes. It also covers support for hybrid shared-memory/message-passing programming; MPI_message, which aids in certain types of multithreaded programming; features that handle very large data; an interface that allows the programmer and the developer to access performance data; and a new binding of MPI to Fortran.
-
The MPI message-passing interface Standard: Overview and Status
High Performance Computing - Technology Methods and Applications, 2014Co-Authors: Ewing Lusk, William GroppAbstract:MPI (message-passing Inteface) has been developed over the last two years as a standard message-passing interface specification. This paper summarizes what MPI is, describes recent activities, particularly MPI implementation activities, and supplies sources for further information about MPI.
-
fault tolerance in message passing interface programs
IEEE International Conference on High Performance Computing Data and Analytics, 2004Co-Authors: William Gropp, Ewing LuskAbstract:In this paper we examine the topic of writing fault-tolerant message passing interface (MPI) applications. We discuss the meaning of fault tolerance in general and what the MPI Standard has to say about it. We survey several approaches to this problem, namely checkpointing, restructuring a class of standard MPI programs, modifying MPI semantics, and extending the MPI specification. We conclude that, within certain constraints, MPI can provide a useful context for writing application programs that exhibit significant degrees of fault tolerance.
-
Improving the Performance of MPI Derived Datatypes
MPI Software Technology Press, 1999Co-Authors: William Gropp, Ewing Lusk, Debbie SwiderAbstract:The message passing interface (MPI) standard provides a powerful mechanism for describing non-contiguous memory locations: derived datatypes. In addition, MPI derived datatypes have a key role in the MPI-2 I/O operations. In principle, MPI derived datatypes allow a user to more efficiently communicate noncontiguous data (for example, strided data) because the MPI implementation can move the data without any intermediate copies to or from a contiguous buffer. In practice, however, few MPI implementations provide support for datatypes that performs better than what the user can achieve by manually packing and unpacking the data into contiguous buffers before calling MPI routines with contiguous memory regions. We develop a taxonomy of MPI datatypes according to their memory reference patterns and show how to efficiently implement these patterns. The effectiveness of this approach is illustrated on a variety of platforms. 1 Introduction The message passing interface (MPI) standard [4, 1]..
-
using mpi 2 advanced features of the message passing interface
1999Co-Authors: William Gropp, Ewing Lusk, Rajeev ThakurAbstract:From the Publisher: The message passing interface (MPI) specification is widely used for solving significant scientific and engineering problems on parallel computers. There exist more than a dozen implementations on computer platforms ranging from IBM SP-2 supercomputers to clusters of PCs running Windows NT or Linux ("Beowulf" machines). The initial MPI Standard document, MPI-1, was recently updated by the MPI Forum. The new version, MPI-2, contains both significant enhancements to the existing MPI core and new features. Using MPI is a completely up-to-date version of the authors' 1994 introduction to the core functions of MPI. It adds material on the new C++ and Fortran 90 bindings for MPI throughout the book. It contains greater discussion of datatype extents, the most frequently misunderstood feature of MPI-1, as well as material on the new extensions to basic MPI functionality added by the MPI-2 Forum in the area of MPI datatypes and collective operations. Using MPI-2 covers the new extensions to basic MPI. These include parallel I/O, remote memory access operations, and dynamic process management. The volume also includes material on tuning MPI applications for high performance on modern MPI implementations.
Neil W Bergmann - One of the best experts on this subject based on the ideXlab platform.
-
a hybrid reconfigurable cluster on chip architecture with message passing interface for image processing applications
Field-Programmable Logic and Applications, 2007Co-Authors: I Syed, John Williams, Neil W BergmannAbstract:We demonstrate a hybrid reconfigurable cluster-on-chip architecture with a cross-platform message passing interface (MPI), a cross-platform parallel image processing library and a sample application. We describe the system, network architecture, MPI library and the parallel image processing library implementations. We validate the performance, scalability and suitability of MPI as a software interface to enable cross-platform application parallelism on reconfigurable hybrid cluster-on-chip systems and desktop cluster systems. The presented results are promising, showing the suitability, scalability and performance of parallelisation of image processing algorithms with a cross-platform MPI implementation.
-
a reconfigurable cluster on chip architecture with mpi communication layer
Field-Programmable Custom Computing Machines, 2006Co-Authors: John Williams, I Syed, Jason Wu, Neil W BergmannAbstract:In this paper, the authors present a reconfigurable cluster-on-chip architecture and supporting parallel programming software library based on the well-known message passing interface (MPI) standard. The intent is to allow designers to program multi-core reconfigurable systems on chip using the same or similar methodologies that yielded tremendous productivity improvements in the workstation and HPC cluster community. Additionally the architecture is designed to support native hardware processing modules to participate in the MPI network as fully-fledged peers
Anthony Skjellum - One of the best experts on this subject based on the ideXlab platform.
-
exampi a modern design and implementation to accelerate message passing interface innovation
IEEE International Conference on High Performance Computing Data and Analytics, 2019Co-Authors: Anthony Skjellum, Martin Rufenacht, Nawrin Sultana, Derek Schafer, Ignacio Laguna, Kathryn MohrorAbstract:The difficulty of deep experimentation with message passing interface (MPI) implementations—which are quite large and complex—substantially raises the cost and complexity of proof-of-concept activities and limits the community of potential contributors to new and better MPI features and implementations alike. Our goal is to enable researchers to experiment rapidly and easily with new concepts, algorithms, and internal protocols for MPI, we introduce ExaMPI, a modern MPI-3.x subset with a robust MPI-4.x roadmap. We discuss design, early implementation, and ongoing utilization in parallel programming research, plus specific research activities enabled by ExaMPI.
-
extending the message passing interface mpi with user level schedules
arXiv: Distributed Parallel and Cluster Computing, 2019Co-Authors: Derek Schafer, Sheikh K Ghafoor, Daniel J Holmes, Martin Ruefenacht, Anthony SkjellumAbstract:Composability is one of seven reasons for the long-standing and continuing success of MPI. Extending MPI by composing its operations with user-level operations provides useful integration with the progress engine and completion notification methods of MPI. However, the existing extensibility mechanism in MPI (generalized requests) is not widely utilized and has significant drawbacks. MPI can be generalized via scheduled communication primitives, for example, by utilizing implementation techniques from existing MPI-3 nonblocking collectives and from forthcoming MPI-4 persistent and partitioned APIs. Non-trivial schedules are used internally in some MPI libraries; but, they are not accessible to end-users. message-based communication patterns can be built as libraries on top of MPI. Such libraries can have comparable implementation maturity and potentially higher performance than MPI library code, but do not require intimate knowledge of the MPI implementation. Libraries can provide performance-portable interfaces that cross MPI implementation boundaries. The ability to compose additional user-defined operations using the same progress engine benefits all kinds of general purpose HPC libraries. We propose a definition for MPI schedules: a user-level programming model suitable for creating persistent collective communication composed with new application-specific sequences of user-defined operations managed by MPI and fully integrated with MPI progress and completion notification. The API proposed offers a path to standardization for extensible communication schedules involving user-defined operations. Our approach has the potential to introduce event-driven programming into MPI (beyond the tools interface), although connecting schedules with events comprises future work. Early performance results described here are promising and indicate strong overlap potential.
-
a multithreaded message passing interface mpi architecture
Journal of Parallel and Distributed Computing, 2001Co-Authors: Boris V Protopopov, Anthony SkjellumAbstract:This paper discusses a multithreaded software architecture for message-passing interface (MPI) software specification. The architecture is thread-safe, allows for concurrent communication over several communications media (multifabric communication), efficiently utilizes available hardware concurrency over a wide range of target platforms, and allows for concurrent communication and computation within the limits imposed by the hardware. The architecture is developed in the framework of the MPICH software architecture, a well-known MPI implementation used worldwide. The proposed architecture adopts wide portability of the MPICH design and remedies some of its deficiencies such as inefficient multifabric communication and non-thread-safety. The paper also considers the issues concerning development of high-performance portable message-passing systems for general-purpose architectures. The contributions of the paper are improving architecture and addressing thread safety of modern reliable messaging software, as well as identifying and taking advantage of inherent concurrency in the message-passing software itself.
-
A high-performance, portable implementation of the MPI message passing interface standard
Parallel Computing, 1996Co-Authors: William Gropp, Nathan Doss, Ewing Lusk, Anthony SkjellumAbstract:MPI (message passing interface) is a specification for a standard library for message passing that was defined by the MPI Forum, a broadly based group of parallel computer vendors, library writers, and applications specialists. Multiple implementations of MPI have been developed. In this paper, we describe MPICH, unique among existing implementations in its design goal of combining portability with high performance. We document its portability and performance and describe the architecture by which these features are simultaneously achieved. We also discuss the set of tools that accompany the free distribution of MPICH, which constitute the beginnings of a portable parallel programming environment. A project of this scope inevitably imparts lessons about parallel computing, the specification being followed, the current hardware and software environment for parallel computing, and project management; we describe those we have learned. Finally, we discuss future developments for MPICH, including those necessary to accommodate extensions to the MPI Standard now being contemplated by the MPI Forum.
-
extending the message passing interface mpi
Software Product Lines, 1994Co-Authors: Anthony Skjellum, N E Doss, K Viswanathan, A Chowdappa, Purushotham BangaloreAbstract:MPI is the de facto message passing standard for multicomputers and networks of workstations, established by the MPI Forum, a group of universities, research centers, and national laboratories (from both the United States and Europe), as well as multi-national vendors in the area of high performance computing. MPI has been implemented already by several groups. Worldwide acceptance of MPI has been quite rapid. This paper overviews several areas in which MPI can be extended, discusses the merits of making such extensions, and begins to demonstrate how some of these extensions can be made. In some areas, such as intercommunicator extensions, significant progress has been made by us already. In other areas (such as remote memory access), we are merely proposing extensions to MPI that we have not yet reduced to practice. Furthermore, we point out that other researchers are evidently working in parallel with us on their own extension concepts for MPI. >
Marc Snir - One of the best experts on this subject based on the ideXlab platform.
-
generalized communicators in the message passing interface
IEEE Transactions on Parallel and Distributed Systems, 2001Co-Authors: E D Demaine, Ian Foster, Carl Kesselman, Marc SnirAbstract:We propose extensions to the message passing interface (MPI) that generalize the MPI communicator concept to allow multiple communication endpoints per process, dynamic creation of endpoints, and the transfer of endpoints between processes. The generalized communicator construct can be used to express a wide range of interesting communication structures, including collective communication operations involving multiple threads per process, communications between dynamically created threads or processes, and object-oriented applications in which communications are directed to specific objects. Furthermore, this enriched functionality can be provided in a manner that preserves backward compatibility with MPI. We describe the proposed extensions, illustrate their use with examples, and describe a prototype implementation in the popular MPI implementation MPICH.
-
mpi 2 extending the message passing interface
European Conference on Parallel Processing, 1996Co-Authors: Al Geist, William Gropp, Ewing Lusk, Andrew Lumsdaine, Steven Husslederman, William Saphir, Tony Skjellum, Marc SnirAbstract:This paper describes current activities of the MPI-2 Forum. The MPI-2 Forum is a group of parallel computer vendors, library writers, and application specialists working together to define a set of extensions to MPI (message passing interface). MPI was defined by the same process and now has many implementations, both vendor-proprietary and publicly available, for a wide variety of parallel computing environments. In this paper we present the salient aspects of the evolving MPI-2 document as it now stands. We discuss proposed extensions and enhancements to MPI in the areas of dynamic process management, one-sided operations, collective operations, new language binding, real-time computing, external interfaces, and miscellaneous topics.
-
generalized communicators in the message passing interface
Proceedings. Second MPI Developer's Conference, 1996Co-Authors: Ian Foster, Carl Kesselman, Marc SnirAbstract:We propose extensions to the message passing interface (MPI) that generalize the MPI communicator concept to allow multiple communication endpoints per process, dynamic creation of endpoints, and the transfer of endpoints between processes. The generalized communicator construct can be used to express a wide range of interesting communication structures, including collective communication operations involving multiple threads per process, communications between dynamically created threads, and object-oriented applications in which communications are directed to specific objects. Furthermore, this enriched functionality can be provided in a manner that preserves backward compatibility with MPI. We describe the proposed extensions, illustrate their use with examples, and discuss implementation issues.