The Experts below are selected from a list of 318 Experts worldwide ranked by ideXlab platform

Timothy M. Jones - One of the best experts on this subject based on the ideXlab platform.

  • IA3@SC - Software Prefetching for Unstructured Mesh Applications
    2018 IEEE ACM 8th Workshop on Irregular Applications: Architectures and Algorithms (IA3), 2018
    Co-Authors: Ioan Hadade, Timothy M. Jones, Feng Wang, Luca Di Mare
    Abstract:

    Applications that exhibit regular memory access patterns usually benefit transparently from hardware prefetchers that bring data into the fast on-chip cache just before it is required, thereby avoiding expensive cache misses. Unfortunately, unstructured mesh applications contain irregular access patterns that are often more difficult to identify in hardware. An alternative for such workloads is Software prefetching, where special non-blocking instructions load data into the cache hierarchy. However, there are currently few examples in the literature on how to incorporate such Software Prefetches into existing applications with positive results. This paper addresses these issues by demonstrating the utility and implementation of Software prefetching in an unstructured finite volume CFD code of representative size and complexity to an industrial application and across a number of processors. We present the benefits of auto-tuning for finding the optimal prefetch distance values across different computational kernels and architectures and demonstrate the importance of choosing the right prefetch destination across the available cache levels for best performance. We discuss the impact of the data layout on the number of prefetch instructions required in kernels with indirect-access patterns and show how to integrate them on top of existing optimisations such as vectorisation. Through this we show significant full application speed-ups on a range of processors, such as the Intel Xeon Skylake CPU (15%) as well as on the in-order Intel Xeon Phi Knights Corner (1.99x) architecture and the out-of-order Knights Landing (33%) many-core processor.

  • Software Prefetching for Unstructured Mesh Applications
    2018 IEEE ACM 8th Workshop on Irregular Applications: Architectures and Algorithms (IA3), 2018
    Co-Authors: Ioan Hadade, Timothy M. Jones, Feng Wang, Luca Di Mare
    Abstract:

    Applications that exhibit regular memory access patterns usually benefit transparently from hardware prefetchers that bring data into the fast on-chip cache just before it is required, thereby avoiding expensive cache misses. Unfortunately, unstructured mesh applications contain irregular access patterns that are often more difficult to identify in hardware. An alternative for such workloads is Software prefetching, where special non-blocking instructions load data into the cache hierarchy. However, there are currently few examples in the literature on how to incorporate such Software Prefetches into existing applications with positive results. This paper addresses these issues by demonstrating the utility and implementation of Software prefetching in an unstructured finite volume CFD code of representative size and complexity to an industrial application and across a number of processors. We present the benefits of auto-tuning for finding the optimal prefetch distance values across different computational kernels and architectures and demonstrate the importance of choosing the right prefetch destination across the available cache levels for best performance. We discuss the impact of the data layout on the number of prefetch instructions required in kernels with indirect-access patterns and show how to integrate them on top of existing optimisations such as vectorisation. Through this we show significant full application speed-ups on a range of processors, such as the Intel Xeon Skylake CPU (15%) as well as on the in-order Intel Xeon Phi Knights Corner (1.99×) architecture and the out-of-order Knights Landing (33%) many-core processor.

  • CGO - Software prefetching for indirect memory accesses
    2017 IEEE ACM International Symposium on Code Generation and Optimization (CGO), 2017
    Co-Authors: Sam Ainsworth, Timothy M. Jones
    Abstract:

    Many modern data processing and HPC workloads are heavily memory-latency bound. A tempting proposition to solve this is Software prefetching, where special non-blocking loads are used to bring data into the cache hierarchy just before being required. However, these are difficult to insert to effectively improve performance, and techniques for automatic insertion are currently limited. This paper develops a novel compiler pass to automatically generate Software Prefetches for indirect memory accesses, a special class of irregular memory accesses often seen in high-performance workloads. We evaluate this across a wide set of systems, all of which gain benefit from the technique. We then evaluate the extent to which good prefetch instructions are architecture dependent. Across a set of memory-bound benchmarks, our automated pass achieves average speedups of 1.3× and 1.1× for an Intel Haswell processor and an ARM Cortex-A57, both out-of-order cores, and performance improvements of 2.1× and 3.7× for the in-order ARM Cortex-A53 and Intel Xeon Phi.

Sam Ainsworth - One of the best experts on this subject based on the ideXlab platform.

  • CGO - Software prefetching for indirect memory accesses
    2017 IEEE ACM International Symposium on Code Generation and Optimization (CGO), 2017
    Co-Authors: Sam Ainsworth, Timothy M. Jones
    Abstract:

    Many modern data processing and HPC workloads are heavily memory-latency bound. A tempting proposition to solve this is Software prefetching, where special non-blocking loads are used to bring data into the cache hierarchy just before being required. However, these are difficult to insert to effectively improve performance, and techniques for automatic insertion are currently limited. This paper develops a novel compiler pass to automatically generate Software Prefetches for indirect memory accesses, a special class of irregular memory accesses often seen in high-performance workloads. We evaluate this across a wide set of systems, all of which gain benefit from the technique. We then evaluate the extent to which good prefetch instructions are architecture dependent. Across a set of memory-bound benchmarks, our automated pass achieves average speedups of 1.3× and 1.1× for an Intel Haswell processor and an ARM Cortex-A57, both out-of-order cores, and performance improvements of 2.1× and 3.7× for the in-order ARM Cortex-A53 and Intel Xeon Phi.

Jones, Timothy M. - One of the best experts on this subject based on the ideXlab platform.

  • Software prefetching for indirect memory accesses: A microarchitectural perspective
    'Organisation for Economic Co-Operation and Development (OECD)', 2019
    Co-Authors: Ainsworth Sam, Jones, Timothy M.
    Abstract:

    Many modern data processing and HPC workloads are heavily memory-latency bound. A tempting proposition to solve this is Software prefetching, where special non-blocking loads are used to bring data into the cache hierarchy just before being required. However, these are difficult to insert to effectively improve performance, and techniques for automatic insertion are currently limited. This paper develops a novel compiler pass to automatically generate Software Prefetches for indirect memory accesses, a special class of irregular memory accesses often seen in high-performance workloads. We evaluate this across a wide set of systems, all of which gain benefit from the technique. We then evaluate the extent to which good prefetch instructions are architecture dependent, and the class of programs that are particularly amenable. Across a set of memory-bound benchmarks, our automated pass achieves average speedups of 1.3x for an Intel Haswell processor, 1.1x for both an ARM Cortex-A57 and Qualcomm Kryo, 1.2x for a Cortex-72 and an Intel Kaby Lake, and 1.35x for an Intel Xeon Phi Knight's Landing, each of which is an out-of-order core, and performance improvements of 2.1x and 2.7x for the in-order ARM Cortex-A53 and first generation Intel Xeon Phi.EPSRC [EP/K026399/1, EP/M506485/1], ARM Ltd

  • Software Prefetching for Unstructured Mesh Applications
    'Organisation for Economic Co-Operation and Development (OECD)', 2019
    Co-Authors: Jones, Timothy M., Wang F, Di Mare L
    Abstract:

    Applications that exhibit regular memory access patterns usually benefit transparently from hardware prefetchers that bring data into the fast on-chip cache just before it is required, thereby avoiding expensive cache misses. Unfortunately, unstructured mesh applications contain irregular access patterns that are often more difficult to identify in hardware. An alternative for such workloads is Software prefetching, where special non-blocking instructions load data into the cache hierarchy. However, there are currently few examples in the literature on how to incorporate such Software Prefetches into existing applications with positive results. This paper addresses these issues by demonstrating the utility and implementation of Software prefetching in an unstructured finite volume CFD code of representative size and complexity to an industrial application and across a number of processors. We present the benefits of auto-tuning for finding the optimal prefetch distance values across different computational kernels and architectures and demonstrate the importance of choosing the right prefetch destination across the available cache levels for best performance. We discuss the impact of the data layout on the number of prefetch instructions required in kernels with indirect-access patterns and show how to integrate them on top of existing optimisations such as vectorisation. Through this we show significant full application speed-ups on a range of processors, such as the Intel Xeon Skylake CPU (15%) as well as on the in-order Intel Xeon Phi Knights Corner (1.99x) architecture and the out-of-order Knights Landing (33%) many-core processor.Parts of this work were supported by the Engineering and Physical Sciences Research Council (EPSRC)and Rolls-Royce plc through the industrial CASE award 13220161 and grant EP/K026399/1. This work used the ARCHER KNL Testing and Development Platform part of the UK National Supercomputing Servic

  • Software Prefetching for Indirect Memory Accesses
    'Organisation for Economic Co-Operation and Development (OECD)', 2017
    Co-Authors: Ainsworth Sam, Jones, Timothy M.
    Abstract:

    Many modern data processing and HPC workloads are heavily memory-latency bound. A tempting proposition to solve this is Software prefetching, where special non-blocking loads are used to bring data into the cache hierarchy just before being required. However, these are difficult to insert to effectively improve performance, and techniques for automatic insertion are currently limited. This paper develops a novel compiler pass to automatically generate Software Prefetches for indirect memory accesses, a special class of irregular memory accesses often seen in high-performance workloads. We evaluate this across a wide set of systems, all of which gain benefit from the technique. We then evaluate the extent to which good prefetch instructions are architecture dependent. Across a set of memory-bound benchmarks, our automated pass achieves average speedups of 1.3$\times$ and 1.1$\times$ for an Intel Haswell processor and an ARM Cortex-A57, both out-of-order cores, and performance improvements of 2.1$\times$ and 3.7$\times$ for the in-order ARM Cortex-A53 and Intel Xeon Phi

Michael J. Flynn - One of the best experts on this subject based on the ideXlab platform.

  • An Automated Method for Software Controlled Cache Prefetching
    1998
    Co-Authors: Daniel F. Zucker, Ruby B. Lee, Michael J. Flynn
    Abstract:

    As the gap between cycle time and main memory access time increases, memory system performance becomes increasingly important. The trend to higher instruction level parallelism with superscalar processors puts even higher demands on the memory system. Prefetching is a common strategy to tolerate this increased memory latency. This paper presents a Software only technique to prefetch data to the CPU cache before it is needed in order combat this problem. The Software prefetching technique presented is motivated by emulation of a hardware stride prediction table (SPT). Performance similar, and in some cases superior, to the hardware based technique is achieved with no additional hardware costs. In the first step, a simulation of the hardware SPT is conducted to identify where useful Prefetches are best added. In the next step, Software Prefetches are added to the executable code. The technique is automated and could be implemented by a compiler as a two phase optimization of a profile st..

  • An Automated Method for Software Controlled Cache Prefetching
    1998
    Co-Authors: Daniel F. Zucker, Ruby B. Lee, Michael J. Flynn
    Abstract:

    As the gap between cycle time and main memory access time increases, memory system performance becomes increasingly important. The trend to higher instruction level parallelism with superscalar processors puts even higher demands on the memory system. Prefetching is a common strategy to tolerate this increased memory latency. This paper presents a Software only technique to prefetch data to the CPU cache before it is needed in order combat this problem. The Software prefetching technique presented is motlvated by emulation of a hardware stride prediction table (SPT). Performance similar, and in some cases superior, to the hardware based technique is achieved with no ad- ditional hardware costs. In the first step, a simulation of the hardware SPT is conducted to identify where use- ful Prefetches are best added. In the next step, Software Prefetches are added to the executable code. The technique is automated and could be implemented by a compiler as a two phase optimization of a profile step followed by an optimization step

  • HICSS (7) - An automated method for Software controlled cache prefetching
    Proceedings of the Thirty-First Hawaii International Conference on System Sciences, 1
    Co-Authors: Daniel F. Zucker, Ruby B. Lee, Michael J. Flynn
    Abstract:

    As the gap between cycle time and main memory access time increases, memory system performance becomes increasingly important. The trend to higher instruction level parallelism with superscalar processors puts even higher demands on the memory system. Prefetching is a common strategy to tolerate this increased memory latency. This paper presents a Software only technique to prefetch data to the CPU cache before it is needed in order combat this problem. The Software prefetching technique presented is motivated by emulation of a hardware stride prediction table (SPT). Performance similar, and in some cases superior, to the hardware based technique is achieved with no additional hardware costs. In the first step, a simulation of the hardware SPT is conducted to identify where useful Prefetches are best added. In the next step, Software Prefetches are added to the executable code. The technique is automated and could be implemented by a compiler as a two phase optimization of a profile step followed by an optimization step. Data is presented for both SPEC95 and multimedia benchmarks. In the best case, a performance improvement of 2.78X is observed over the same code with no prefetching at no extra hardware costs.

Ainsworth Sam - One of the best experts on this subject based on the ideXlab platform.

  • Software prefetching for indirect memory accesses: A microarchitectural perspective
    'Organisation for Economic Co-Operation and Development (OECD)', 2019
    Co-Authors: Ainsworth Sam, Jones, Timothy M.
    Abstract:

    Many modern data processing and HPC workloads are heavily memory-latency bound. A tempting proposition to solve this is Software prefetching, where special non-blocking loads are used to bring data into the cache hierarchy just before being required. However, these are difficult to insert to effectively improve performance, and techniques for automatic insertion are currently limited. This paper develops a novel compiler pass to automatically generate Software Prefetches for indirect memory accesses, a special class of irregular memory accesses often seen in high-performance workloads. We evaluate this across a wide set of systems, all of which gain benefit from the technique. We then evaluate the extent to which good prefetch instructions are architecture dependent, and the class of programs that are particularly amenable. Across a set of memory-bound benchmarks, our automated pass achieves average speedups of 1.3x for an Intel Haswell processor, 1.1x for both an ARM Cortex-A57 and Qualcomm Kryo, 1.2x for a Cortex-72 and an Intel Kaby Lake, and 1.35x for an Intel Xeon Phi Knight's Landing, each of which is an out-of-order core, and performance improvements of 2.1x and 2.7x for the in-order ARM Cortex-A53 and first generation Intel Xeon Phi.EPSRC [EP/K026399/1, EP/M506485/1], ARM Ltd

  • Software Prefetching for Indirect Memory Accesses
    'Organisation for Economic Co-Operation and Development (OECD)', 2017
    Co-Authors: Ainsworth Sam, Jones, Timothy M.
    Abstract:

    Many modern data processing and HPC workloads are heavily memory-latency bound. A tempting proposition to solve this is Software prefetching, where special non-blocking loads are used to bring data into the cache hierarchy just before being required. However, these are difficult to insert to effectively improve performance, and techniques for automatic insertion are currently limited. This paper develops a novel compiler pass to automatically generate Software Prefetches for indirect memory accesses, a special class of irregular memory accesses often seen in high-performance workloads. We evaluate this across a wide set of systems, all of which gain benefit from the technique. We then evaluate the extent to which good prefetch instructions are architecture dependent. Across a set of memory-bound benchmarks, our automated pass achieves average speedups of 1.3$\times$ and 1.1$\times$ for an Intel Haswell processor and an ARM Cortex-A57, both out-of-order cores, and performance improvements of 2.1$\times$ and 3.7$\times$ for the in-order ARM Cortex-A53 and Intel Xeon Phi