The Experts below are selected from a list of 13467762 Experts worldwide ranked by ideXlab platform
Huiyang Zhou - One of the best experts on this subject based on the ideXlab platform.
-
CUDA-NP: Realizing Nested Thread-Level Parallelism in GPGPU Applications
Journal of Computer Science and Technology, 2015Co-Authors: Yi Yang, Chao Li, Huiyang ZhouAbstract:Parallel programs consist of series of code sections with different thread-level parallelism (TLP). As a result, it is rather common that a thread in a parallel program, such as a GPU kernel in CUDA programs, still contains both sequential code and parallel loops. In order to leverage such parallel loops, the latest NVIDIA Kepler architecture introduces dynamic parallelism, which allows a GPU thread to start another GPU kernel, thereby reducing the overhead of launching kernels from a CPU. However, with dynamic parallelism, a parent thread can only communicate with its child threads through global memory and the overhead of launching GPU kernels is non-trivial even within GPUs. In this paper, we first study a set of GPGPU benchmarks that contain parallel loops, and highlight that these benchmarks do not have a very high loop count or high degree of TLP. Consequently, the benefits of leveraging such parallel loops using dynamic parallelism are too limited to offset its overhead. We then present our proposed solution to exploit nested parallelism in CUDA, referred to as CUDA-NP. With CUDA-NP, we initially enable a high number of threads when a GPU program starts, and use control flow to activate different numbers of threads for different code sections. We implement our proposed CUDA-NP framework using a directive-based compiler approach. For a GPU kernel, an application developer only needs to add OpenMP-like pragmas for parallelizable code sections. Then, our CUDA-NP compiler automatically generates the optimized GPU kernels. It supports both the reduction and the scan primitives, explores different ways to distribute parallel loop iterations into threads, and efficiently manages on-chip resource. Our experiments show that for a set of GPGPU benchmarks, which have already been optimized and contain nested parallelism, our proposed CUDA-NP framework further improves the performance by up to 6.69 times and 2.01 times on average.
-
cuda np realizing nested thread level parallelism in gpgpu applications
ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, 2014Co-Authors: Yi Yang, Huiyang ZhouAbstract:Parallel programs consist of series of code sections with different thread-level parallelism (TLP). As a result, it is rather common that a thread in a parallel program, such as a GPU kernel in CUDA programs, still contains both se-quential code and parallel loops. In order to leverage such parallel loops, the latest Nvidia Kepler architecture intro-duces dynamic parallelism, which allows a GPU thread to start another GPU kernel, thereby reducing the overhead of launching kernels from a CPU. However, with dynamic parallelism, a parent thread can only communicate with its child threads through global memory and the overhead of launching GPU kernels is non-trivial even within GPUs. In this paper, we first study a set of GPGPU benchmarks that contain parallel loops, and highlight that these bench-marks do not have a very high loop count or high degrees of TLP. Consequently, the benefits of leveraging such par-allel loops using dynamic parallelism are too limited to offset its overhead. We then present our proposed solution to exploit nested parallelism in CUDA, referred to as CUDA-NP. With CUDA-NP, we initially enable a high number of threads when a GPU program starts, and use control flow to activate different numbers of threads for different code sections. We implemented our proposed CUDA-NP framework using a directive-based compiler approach. For a GPU kernel, an application developer only needs to add OpenMP-like pragmas for parallelizable code sections. Then, our CUDA-NP compiler automatically gen-erates the optimized GPU kernels. It supports both the reduction and the scan primitives, explores different ways to distribute parallel loop iterations into threads, and effi-ciently manages on-chip resource. Our experiments show that for a set of GPGPU benchmarks, which have already been optimized and contain nested parallelism, our pro-posed CUDA-NP framework further improves the perfor-mance by up to 6.69 times and 2.18 times on average.
-
PPOPP - CUDA-NP: realizing nested thread-level parallelism in GPGPU applications
Proceedings of the 19th ACM SIGPLAN symposium on Principles and practice of parallel programming - PPoPP '14, 2014Co-Authors: Yi Yang, Huiyang ZhouAbstract:Parallel programs consist of series of code sections with different thread-level parallelism (TLP). As a result, it is rather common that a thread in a parallel program, such as a GPU kernel in CUDA programs, still contains both se-quential code and parallel loops. In order to leverage such parallel loops, the latest Nvidia Kepler architecture intro-duces dynamic parallelism, which allows a GPU thread to start another GPU kernel, thereby reducing the overhead of launching kernels from a CPU. However, with dynamic parallelism, a parent thread can only communicate with its child threads through global memory and the overhead of launching GPU kernels is non-trivial even within GPUs. In this paper, we first study a set of GPGPU benchmarks that contain parallel loops, and highlight that these bench-marks do not have a very high loop count or high degrees of TLP. Consequently, the benefits of leveraging such par-allel loops using dynamic parallelism are too limited to offset its overhead. We then present our proposed solution to exploit nested parallelism in CUDA, referred to as CUDA-NP. With CUDA-NP, we initially enable a high number of threads when a GPU program starts, and use control flow to activate different numbers of threads for different code sections. We implemented our proposed CUDA-NP framework using a directive-based compiler approach. For a GPU kernel, an application developer only needs to add OpenMP-like pragmas for parallelizable code sections. Then, our CUDA-NP compiler automatically gen-erates the optimized GPU kernels. It supports both the reduction and the scan primitives, explores different ways to distribute parallel loop iterations into threads, and effi-ciently manages on-chip resource. Our experiments show that for a set of GPGPU benchmarks, which have already been optimized and contain nested parallelism, our pro-posed CUDA-NP framework further improves the perfor-mance by up to 6.69 times and 2.18 times on average.
Yi Yang - One of the best experts on this subject based on the ideXlab platform.
-
CUDA-NP: Realizing Nested Thread-Level Parallelism in GPGPU Applications
Journal of Computer Science and Technology, 2015Co-Authors: Yi Yang, Chao Li, Huiyang ZhouAbstract:Parallel programs consist of series of code sections with different thread-level parallelism (TLP). As a result, it is rather common that a thread in a parallel program, such as a GPU kernel in CUDA programs, still contains both sequential code and parallel loops. In order to leverage such parallel loops, the latest NVIDIA Kepler architecture introduces dynamic parallelism, which allows a GPU thread to start another GPU kernel, thereby reducing the overhead of launching kernels from a CPU. However, with dynamic parallelism, a parent thread can only communicate with its child threads through global memory and the overhead of launching GPU kernels is non-trivial even within GPUs. In this paper, we first study a set of GPGPU benchmarks that contain parallel loops, and highlight that these benchmarks do not have a very high loop count or high degree of TLP. Consequently, the benefits of leveraging such parallel loops using dynamic parallelism are too limited to offset its overhead. We then present our proposed solution to exploit nested parallelism in CUDA, referred to as CUDA-NP. With CUDA-NP, we initially enable a high number of threads when a GPU program starts, and use control flow to activate different numbers of threads for different code sections. We implement our proposed CUDA-NP framework using a directive-based compiler approach. For a GPU kernel, an application developer only needs to add OpenMP-like pragmas for parallelizable code sections. Then, our CUDA-NP compiler automatically generates the optimized GPU kernels. It supports both the reduction and the scan primitives, explores different ways to distribute parallel loop iterations into threads, and efficiently manages on-chip resource. Our experiments show that for a set of GPGPU benchmarks, which have already been optimized and contain nested parallelism, our proposed CUDA-NP framework further improves the performance by up to 6.69 times and 2.01 times on average.
-
cuda np realizing nested thread level parallelism in gpgpu applications
ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, 2014Co-Authors: Yi Yang, Huiyang ZhouAbstract:Parallel programs consist of series of code sections with different thread-level parallelism (TLP). As a result, it is rather common that a thread in a parallel program, such as a GPU kernel in CUDA programs, still contains both se-quential code and parallel loops. In order to leverage such parallel loops, the latest Nvidia Kepler architecture intro-duces dynamic parallelism, which allows a GPU thread to start another GPU kernel, thereby reducing the overhead of launching kernels from a CPU. However, with dynamic parallelism, a parent thread can only communicate with its child threads through global memory and the overhead of launching GPU kernels is non-trivial even within GPUs. In this paper, we first study a set of GPGPU benchmarks that contain parallel loops, and highlight that these bench-marks do not have a very high loop count or high degrees of TLP. Consequently, the benefits of leveraging such par-allel loops using dynamic parallelism are too limited to offset its overhead. We then present our proposed solution to exploit nested parallelism in CUDA, referred to as CUDA-NP. With CUDA-NP, we initially enable a high number of threads when a GPU program starts, and use control flow to activate different numbers of threads for different code sections. We implemented our proposed CUDA-NP framework using a directive-based compiler approach. For a GPU kernel, an application developer only needs to add OpenMP-like pragmas for parallelizable code sections. Then, our CUDA-NP compiler automatically gen-erates the optimized GPU kernels. It supports both the reduction and the scan primitives, explores different ways to distribute parallel loop iterations into threads, and effi-ciently manages on-chip resource. Our experiments show that for a set of GPGPU benchmarks, which have already been optimized and contain nested parallelism, our pro-posed CUDA-NP framework further improves the perfor-mance by up to 6.69 times and 2.18 times on average.
-
PPOPP - CUDA-NP: realizing nested thread-level parallelism in GPGPU applications
Proceedings of the 19th ACM SIGPLAN symposium on Principles and practice of parallel programming - PPoPP '14, 2014Co-Authors: Yi Yang, Huiyang ZhouAbstract:Parallel programs consist of series of code sections with different thread-level parallelism (TLP). As a result, it is rather common that a thread in a parallel program, such as a GPU kernel in CUDA programs, still contains both se-quential code and parallel loops. In order to leverage such parallel loops, the latest Nvidia Kepler architecture intro-duces dynamic parallelism, which allows a GPU thread to start another GPU kernel, thereby reducing the overhead of launching kernels from a CPU. However, with dynamic parallelism, a parent thread can only communicate with its child threads through global memory and the overhead of launching GPU kernels is non-trivial even within GPUs. In this paper, we first study a set of GPGPU benchmarks that contain parallel loops, and highlight that these bench-marks do not have a very high loop count or high degrees of TLP. Consequently, the benefits of leveraging such par-allel loops using dynamic parallelism are too limited to offset its overhead. We then present our proposed solution to exploit nested parallelism in CUDA, referred to as CUDA-NP. With CUDA-NP, we initially enable a high number of threads when a GPU program starts, and use control flow to activate different numbers of threads for different code sections. We implemented our proposed CUDA-NP framework using a directive-based compiler approach. For a GPU kernel, an application developer only needs to add OpenMP-like pragmas for parallelizable code sections. Then, our CUDA-NP compiler automatically gen-erates the optimized GPU kernels. It supports both the reduction and the scan primitives, explores different ways to distribute parallel loop iterations into threads, and effi-ciently manages on-chip resource. Our experiments show that for a set of GPGPU benchmarks, which have already been optimized and contain nested parallelism, our pro-posed CUDA-NP framework further improves the perfor-mance by up to 6.69 times and 2.18 times on average.
Andreas Koch - One of the best experts on this subject based on the ideXlab platform.
-
an open source tool flow for the composition of reconfigurable hardware thread pool architectures
Field-Programmable Custom Computing Machines, 2015Co-Authors: Jens Korinth, David De La Chevallerie, Andreas KochAbstract:With heterogeneous parallel computing becoming more accessible from general-purpose languages, such as directive-enhanced C/C++ or X10, it is now profitable to exploit the highly energy-efficient operation of reconfigurable accelerators in such frameworks. A common paradigm to present the accelerator to the programmer is as a pool of individual threads, each executed on dedicated hardware. While the actual accelerator logic can be synthesized into IP cores from a high-level language using tools such as Vivado HLS, no tools currently exist to automatically compose multiple heterogeneous accelerator cores into a unified hardware thread pool, including the assembly of external control and memory interfaces. Thread Pool Composer closes the gap in the design flow between high-level synthesis and general-purpose IP integration by automatically composing hardware thread pools and their external interfaces from high-level descriptions and opening them to software using a common API.
-
FCCM - An Open-Source Tool Flow for the Composition of Reconfigurable Hardware Thread Pool Architectures
2015 IEEE 23rd Annual International Symposium on Field-Programmable Custom Computing Machines, 2015Co-Authors: Jens Korinth, David De La Chevallerie, Andreas KochAbstract:With heterogeneous parallel computing becoming more accessible from general-purpose languages, such as directive-enhanced C/C++ or X10, it is now profitable to exploit the highly energy-efficient operation of reconfigurable accelerators in such frameworks. A common paradigm to present the accelerator to the programmer is as a pool of individual threads, each executed on dedicated hardware. While the actual accelerator logic can be synthesized into IP cores from a high-level language using tools such as Vivado HLS, no tools currently exist to automatically compose multiple heterogeneous accelerator cores into a unified hardware thread pool, including the assembly of external control and memory interfaces. Thread Pool Composer closes the gap in the design flow between high-level synthesis and general-purpose IP integration by automatically composing hardware thread pools and their external interfaces from high-level descriptions and opening them to software using a common API.
Andreas Herkersdorf - One of the best experts on this subject based on the ideXlab platform.
-
tcu a multi objective hardware thread mapping unit for hpc clusters
IEEE International Conference on High Performance Computing Data and Analytics, 2016Co-Authors: Ravi Kumar Pujari, Thomas Wild, Andreas HerkersdorfAbstract:Meeting multiple, partially orthogonal optimization targets during thread scheduling on HPC and manycore platforms simultaneously, like maximizing CPU performance, meeting deadlines of time critical tasks, minimizing power and securing thermal resilience, is a major challenge because of associated scalability and thread management overhead. We tackle these challenges by introducing the Thread Control Unit (TCU), a configurable, low-latency, low-overhead hardware thread mapper in compute nodes of an HPC cluster. The TCU takes various sensor information into account and can map threads to 4–16 CPUs of a compute node within a small and bounded number of clock cycles in round-robin, single- or multi-objective manner. The TCU design can consider not just load balancing or performance criteria but also physical constraints like temperature limits, power budgets and reliability aspects. Evaluations of different mapping policies show that multi-objective thread mapping provides about 10 to 40 % less mapping latency for periodic workloads compared to single-objective or round-robin policies. For bursty workloads under high load conditions, a 20 % reduction is achieved.
-
ISC - TCU: A Multi-Objective Hardware Thread Mapping Unit for HPC Clusters
Lecture Notes in Computer Science, 2016Co-Authors: Ravi Kumar Pujari, Thomas Wild, Andreas HerkersdorfAbstract:Meeting multiple, partially orthogonal optimization targets during thread scheduling on HPC and manycore platforms simultaneously, like maximizing CPU performance, meeting deadlines of time critical tasks, minimizing power and securing thermal resilience, is a major challenge because of associated scalability and thread management overhead. We tackle these challenges by introducing the Thread Control Unit (TCU), a configurable, low-latency, low-overhead hardware thread mapper in compute nodes of an HPC cluster. The TCU takes various sensor information into account and can map threads to 4–16 CPUs of a compute node within a small and bounded number of clock cycles in round-robin, single- or multi-objective manner. The TCU design can consider not just load balancing or performance criteria but also physical constraints like temperature limits, power budgets and reliability aspects. Evaluations of different mapping policies show that multi-objective thread mapping provides about 10 to 40 % less mapping latency for periodic workloads compared to single-objective or round-robin policies. For bursty workloads under high load conditions, a 20 % reduction is achieved.
-
a hardware based multi objective thread mapper for tiled manycore architectures
International Conference on Computer Design, 2015Co-Authors: Ravi Kumar Pujari, Thomas Wild, Andreas HerkersdorfAbstract:Thread mapping is typically performed as an integral part of cooperative or pre-emptive operating system (OS) scheduling in order to share the processor core(s) among competing applications. Schedulers usually follow a single-objective performance optimization, such as maximizing core utilization or satisfying deadlines by the prioritization of threads. Meeting multiple orthogonal objectives, like performance vs. power or thermal resilience, in the era of manycore processors is a challenge because of the associated scalability and thread management overhead. We tackle these challenges by employing a two stage thread management strategy. In the first stage (not covered in this short paper), threads are assigned to regions or compute tiles. For the second stage we introduce in this paper the TCU (Thread Control Unit), a configurable, low latency, low overhead hardware thread mapper that takes various runtime sensor parameters into account. It can map threads within a small and bounded number of clock cycles in round robin, single or multi-objective manner. TCU is designed to consider not just load balancing or performance criteria but also physical constraints like power budgets, temperature limits and reliability aspects. TCU macro achieves 150K thread mappings per second on a tiled MPSoC FPGA prototype while operating at moderate 50 Mz. Evaluations of different mapping policies show that multi-objective thread mapping provides about 10 to 40% less mapping latency for periodic and bursty traffic compared to single-objective or round robin schemes. FPGA and ASIC syntheses reveal a 9% hardware overhead for the TCU on a four core compute tile.
Periklis Akritidis - One of the best experts on this subject based on the ideXlab platform.
-
DIMVA - MemPatrol: Reliable Sideline Integrity Monitoring for High-Performance Systems
Detection of Intrusions and Malware and Vulnerability Assessment, 2017Co-Authors: Myoung Jin Nam, Wonhong Nam, Jin Young Choi, Periklis AkritidisAbstract:Integrity checking using inline reference monitors to check individual memory accesses in C/C++ programs remains prohibitively expensive for the most performance-critical applications. To address this, we developed MemPatrol, a “sideline” integrity monitor that allows us to minimize the amount of performance degradation at the expense of increased detection delay. Inspired by existing proposals, MemPatrol uses a dedicated monitor thread running in parallel with the other threads of the protected application. Previous proposals, however, either rely on costly isolation mechanisms, or introduce a vulnerability window between the attack and its detection. During this vulnerability window, malicious code can cover up memory corruption, breaking the security guarantee of “eventual detection” that comes with strong isolation. Our key contributions are (i) a novel userspace-based isolation mechanism to address the vulnerability window, and (ii) to successfully reduce the overhead incurred by the application’s threads to a level acceptable for a performance-critical application. We evaluate MemPatrol on a high-performance passive network monitoring system, demonstrating its low overheads, as well as the operator’s control of the trade-off between performance degradation and detection delay.
-
MemPatrol: Reliable Sideline Integrity Monitoring for High-Performance Systems
Detection of Intrusions and Malware and Vulnerability Assessment, 2017Co-Authors: Myoung Jin Nam, Wonhong Nam, Jin Young Choi, Periklis AkritidisAbstract:Integrity checking using inline reference monitors to check individual memory accesses in C/C++ programs remains prohibitively expensive for the most performance-critical applications. To address this, we developed MemPatrol, a “sideline” integrity monitor that allows us to minimize the amount of performance degradation at the expense of increased detection delay. Inspired by existing proposals, MemPatrol uses a dedicated monitor thread running in parallel with the other threads of the protected application. Previous proposals, however, either rely on costly isolation mechanisms, or introduce a vulnerability window between the attack and its detection. During this vulnerability window, malicious code can cover up memory corruption, breaking the security guarantee of “eventual detection” that comes with strong isolation. Our key contributions are (i) a novel userspace-based isolation mechanism to address the vulnerability window, and (ii) to successfully reduce the overhead incurred by the application’s threads to a level acceptable for a performance-critical application. We evaluate MemPatrol on a high-performance passive network monitoring system, demonstrating its low overheads, as well as the operator’s control of the trade-off between performance degradation and detection delay.