arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Sascha Hunold

61 papers · 2004–2026

2004: 12005: 02006: 12007: 12008: 42009: 22010: 32011: 42012: 02013: 22014: 22015: 42016: 52017: 42018: 32019: 12020: 62021: 12022: 22023: 42024: 52025: 52026: 1
2004peak 6 in 20202026

Frequent co-authors

Papers

showing 1–25 of 61 RSS

Reproducible MPI Benchmarking is Still Not as Easy as You Think

Sascha Hunold, Alexandra Carpen-Amarie

Hardware and Architecture · IEEE Transactions on Parallel and Distributed Systems · · cited 44× · DOI ↗ · more like this ·

The Message Passing Interface (MPI) is the prevalent programming model used on today's supercomputers. Therefore, MPI library developers are looking for the best possible performance (shortest run-time) of individual MPI functions across many different supercomputer architectures. Several MPI benchmark suites have been developed to assess the performance of MPI implementations. Unfortunately, the outcome of these benchmarks is often neither reproducible nor statistically sound. To overcome these issues, we show which experimental factors have an impact on the run-time of blocking collective MPI operations and how to measure their effect. Finally, we present a new experimental method that allows us to obtain reproducible and statistically sound measurements of MPI functions.

Predicting the Energy-Consumption of MPI Applications at Scale Using Only a Single Node

Franz C. Heinrich, Tom Cornebize, Augustin Degomme +5 more

Information Systems · · cited 42× · DOI ↗ · more like this ·

Monitoring and assessing the energy efficiency of supercomputers and data centers is crucial in order to limit and reduce their energy consumption. Applications from the domain of High Performance Computing (HPC), such as MPI applications, account for a significant fraction of the overall energy consumed by HPC centers. Simulation is a popular approach for studying the behavior of these applications in a variety of scenarios, and it is therefore advantageous to be able to study their energy consumption in a cost-efficient, controllable, and also reproducible simulation environment. Alas, simulators supporting HPC applications commonly lack the capability of predicting the energy consumption, particularly when target platforms consist of multi-core nodes. In this work, we aim to accurately predict the energy consumption of MPI applications via simulation. Firstly, we introduce the models…

Scheduling Independent Moldable Tasks on Multi-Cores with GPUs

Raphaël Bleuse, Sascha Hunold, Safia Kedad‐Sidhoum +3 more

Computer Networks and Communications · IEEE Transactions on Parallel and Distributed Systems · · cited 35× · DOI ↗ · more like this ·

We present a new approach for scheduling independent tasks on multiple CPUs and multiple GPUs. The tasks are assumed to be parallelizable on CPUs using the moldable model: the final number of cores allotted to a task can be decided and set by the scheduler. More precisely, we design an algorithm aiming at minimizing the makespan-the maximum completion time of all tasks-for this scheduling problem. The proposed algorithm combines a dual approximation scheme with a fast integer linear program (ILP). It determines both the partitioning of the tasks, i.e., whether a task should be mapped to CPUs or a GPU, and the number of CPUs allotted to a moldable task if mapped to the CPUs. A worst-case analysis shows that the algorithm has an approximation ratio of 3/2 + ε. Since the time complexity of the ILP-based algorithm could be non-polynomial, we also present a polynomial-time algorithm with an…

Implementing a classic

Jesper Larsson Träff, Antoine Rougier, Sascha Hunold

Computer Networks and Communications · · cited 28× · DOI ↗ · more like this ·

We investigate the use of the derived datatype mechanism of MPI (the Message-Passing Interface) in the implementation of the classic all-to-all communication algorithm of Bruck et al.\ (1997). Through a series of improvements to the canonical implementation of the algorithm we gradually eliminate initial and final processor-local data reorganizations, culminating in a \emph{zero-copy} version that contains no explicit, process-local data movement or copy operations: all necessary data movements are implied by MPI derived datatypes, and carried out as part of the communication operations. We furthermore show how the improved algorithm can be used to solve irregular all-to-all communication problems (that are not too irregular). The Bruck algorithm serves as a vehicle to demonstrate descriptive and performance advantages with MPI datatypes in the implementation of complex algorithms, and…

Decomposing MPI Collectives for Exploiting Multi-lane Communication

Jesper Larsson Träff, Sascha Hunold

Hardware and Architecture · · cited 26× · DOI ↗ · more like this ·

Many modern, high-performance systems increase the cumulated node-bandwidth by offering more than a single communication network and/or by having multiple connections to the network, such that a single processor-core cannot by itself saturate the off-node bandwidth. Efficient algorithms and implementations for collective operations as found in, e.g., MPI, must be explicitly designed for exploiting such multilane capabilities. We are interested in gauging to which extent this might be the case. We systematically decompose the MPI collectives into similar operations that can execute concurrently on and exploit multiple network lanes. Our decomposition is applicable to all standard MPI collectives (broadcast, gather, scatter, allgather, reduce allreduce, reduce-scatter, scan, alltoall), and our implementations' performance can be readily compared to the native collectives of any given MPI…

Predicting MPI Collective Communication Performance Using Machine Learning

Sascha Hunold, Abhinav Bhatelé, George Bosilca +1 more

Computer Networks and Communications · · cited 24× · DOI ↗ · more like this ·

The Message Passing Interface (MPI) defines the semantics of data communication operations, while the implementing libraries provide several parameterized algorithms for each operation. Each algorithm of an MPI collective operation may work best on a particular system and may be dependent on the specific communication problem. Internally, MPI libraries employ heuristics to select the best algorithm for a given communication problem when being called by an MPI application. The majority of MPI libraries allow users to override the default algorithm selection, enabling the tuning of this selection process. The problem then becomes how to select the best possible algorithm for a specific case automatically. In this paper, we address the algorithm selection problem for MPI collective communication operations. To solve this problem, we propose an auto-tuning framework for collective MPI…

Multilevel hierarchical matrix multiplication on clusters

Sascha Hunold, Thomas Rauber, Gudula Rünger

Hardware and Architecture · ERef Bayreuth (University of Bayreuth) · · cited 22× · DOI ↗ · more like this ·

Matrix-matrix multiplication is one of the core computations in many algorithms from scientific computing or numerical analysis and many efficient realizations have been invented over the years, including many parallel ones. The current trend to use clusters of PCs or SMPs for scientific computing suggests to revisit matrix-matrix multiplication and investigate efficiency and scalability of different versions on clusters. In this paper we present parallel algorithms for matrix-matrix multiplication which are built up from several algorithms in a multilevel structure. Each level is associated with a hierarchical partition of the set of available processors into disjoint subsets so that deeper levels of the algorithm employ smaller groups of processors in parallel. We perform runtime experiments on several parallel platforms and show that multilevel algorithms can lead to significant…

Isomorphic, Sparse MPI-like Collective Communication Operations for Parallel Stencil Computations

Jesper Larsson Träff, Felix Donatus Lübbe, Antoine Rougier +1 more

Hardware and Architecture · · cited 17× · DOI ↗ · more like this ·

We propose a specification and discuss implementations of collective operations for parallel stencil-like computations that are not supported well by the current MPI 3.1 neighborhood collectives. In our isomorphic, sparse collectives all processes partaking in the communication operation use similar neighborhoods of processes with which to exchange data. Our interface assumes the p processes to be arranged in a d-dimensional torus (mesh) over which neighborhoods are specified per process by identical lists of relative coordinates. This extends significantly on the functionality for Cartesian communicators, and is a much lighter mechanism than distributed graph topologies. It allows for fast, local computation of communication schedules, and can be used in more dynamic contexts than current MPI functionality. We sketch three algorithms for neighborhoods with s source and target…

Jedule: A Tool for Visualizing Schedules of Parallel Applications

Sascha Hunold, Ralf Hoffmann, Frédéric Suter

Computer Networks and Communications · HAL (Le Centre pour la Communication Scientifique Directe) · · cited 16× · DOI ↗ · more like this ·

Task scheduling is one of the most prominent problems in the era of parallel computing. We find scheduling algorithms in every domain of computer science, e.g., mapping multiprocessor tasks to clusters, mapping jobs to grid resources, or mapping fine-grained tasks to cores of multicore processors. Many tools exist that help understand or debug an application by presenting visual representations of a certain program run, e.g., visualizations of MPI traces. However, often developers want to get a global and abstract view of their schedules first. In this paper we introduce Jedule, a tool dedicated to visualize schedules of parallel applications. We demonstrate the effectiveness of Jedule by showing how it helped analyzing problems in several case studies.

Scheduling Dynamic Workflows onto Clusters of Clusters using Postponing

Sascha Hunold, Thomas Rauber, Frédéric Suter

Computer Networks and Communications · ERef Bayreuth (University of Bayreuth) · · cited 16× · DOI ↗ · more like this ·

In this article, we revisit the problem of scheduling dynamically generated directed acyclic graphs (DAGs) of multi-processor tasks (M-tasks). A DAG is a basic model for expressing workflows applications where each node represents a task of the workflow. We present a novel algorithm (DMHEFT) for scheduling dynamically generated DAGs onto a heterogeneous collection of clusters. The scheduling decisions are based on the predicted runtime of an M-task as well as the estimation of the redistribution costs between data-dependent tasks. The algorithm also takes care of unfavorable placements of M-tasks by considering the postponing of ready tasks even if idle processors are available. We evaluate the scheduling algorithm by comparing the resulting makespans to the results obtained by using other scheduling algorithms, such as RePA and MHEFT.

Autotuning MPI Collectives using Performance Guidelines

Sascha Hunold, Alexandra Carpen-Amarie

Hardware and Architecture · · cited 14× · DOI ↗ · more like this ·

MPI collective operations provide a standardized interface for performing data movements within a group of processes. The efficiency of collective communication operations depends on the actual algorithm, its implementation, and the specific communication problem (type of communication, message size, and number of processes). Many MPI libraries provide numerous algorithms for specific collective operations. The strategy for selecting an efficient algorithm is often times predefined (hard-coded) in MPI libraries, but some of them, such as Open MPI, allow users to change the algorithm manually. Finding the best algorithm for each case is a hard problem, and several approaches to tune these algorithmic parameters have been proposed. We use an orthogonal approach to the parameter-tuning of MPI collectives, that is, instead of testing individual algorithmic choices provided by an MPI…

Reproducible MPI Micro-Benchmarking Isn't As Easy As You Think

Sascha Hunold, Alexandra Carpen-Amarie, Jesper Larsson Träff

Hardware and Architecture · · cited 14× · DOI ↗ · more like this ·

The Message Passing Interface (MPI) is the prevalent programming model for supercomputers. Optimizing the performance of individual MPI functions is therefore of great interest for the HPC community. However, a fair comparison of different algorithms and implementations requires a statistically sound analysis. It is often overlooked that the time to complete an MPI communication function does not only depend on internal factors such as the algorithm but also on external factors such as the system noise. Most noise produced by the system is uncontrollable without changing the software stack, e.g., the memory allocation method used by the operating system. Possibly controllable factors have not yet been identified as such in this context. We investigate several possible factors---which have been discovered in other microbenchmarks---whether they have a significant effect on the execution…

Benchmarking Julia’s Communication Performance: Is Julia HPC ready or Full HPC?

Sascha Hunold, Sebastian Steiner

Hardware and Architecture · · cited 13× · DOI ↗ · more like this ·

Julia has quickly become one of the main programming languages for computational sciences, mainly due to its speed and flexibility. The speed and efficiency of Julia are the main reasons why researchers in the field of High Performance Computing have started porting their applications to Julia.Since Julia has a very small binding-overhead to C, many efficient computational kernels can be integrated into Julia without any noticeable performance drop. For that reason, highly tuned libraries, such as the Intel MKL or OpenBLAS, will allow Julia applications to achieve similar computational performance as their C counterparts. Yet, two questions remain: 1) How fast is Julia for memory-bound applications? 2) How efficient can MPI functions be called from a Julia application?In this paper, we will assess the performance of Julia with respect to HPC. To that end, we examine the raw throughput…

Low-Cost Tuning of Two-Step Algorithms for Scheduling Mixed-Parallel Applications onto Homogeneous Clusters

Sascha Hunold

Computer Networks and Communications · · cited 13× · DOI ↗ · more like this ·

Due to the strong increase of processing units available to the end user, expressing parallelism of an algorithm is a major challenge for many researchers. Parallel applications are often expressed using a task-parallel model (task graphs), in which tasks can be executed concurrently unless they share a dependency. If these tasks can also be executed in a data-parallel fashion, e.g., by using MPI or OpenMP, then we call it a mixed-parallel programming model. Mixed-parallel applications are often modeled as directed a cyclic graphs (DAGs), where nodes represent the tasks and edges represent data dependencies. To execute a mixed-parallel application efficiently, a good scheduling strategy is required to map the tasks to the available processors. Several algorithms for the scheduling of mixed-parallel applications onto a homogeneous cluster have been proposed. MCPA (Modified CPA) has been…

Redistribution aware two-step scheduling for mixed-parallel applications

Sascha Hunold, Thomas Rauber, Frédéric Suter

Computer Networks and Communications · ERef Bayreuth (University of Bayreuth) · · cited 13× · DOI ↗ · more like this ·

Applications raising in many scientific fields exhibit both data and task parallelism that have to be exploited efficiently. A classic approach is to structure those applications by a task graph whose nodes represent parallel computations. Scheduling such mixed-parallel applications is challenging even on a single homogeneous platform, such as a cluster. Most of the mixed-parallel application scheduling algorithms rely on two decoupled steps: allocation and mapping. This separation can induce unnecessary or costly data redistributions that have an impact on the overall performance. This is particularly true for data intensive applications. In this paper, we propose an original approach in which the allocations determined in the first step can be adapted during the second step in order to minimize the impact of these data redistributions. Two redistribution aware mapping strategies are…

Transformation of Legacy Software into Client/Server Applications through Pattern-Based Rearchitecturing

Sascha Hunold, Matthias Korch, Björn Krellner +3 more

Artificial Intelligence · ERef Bayreuth (University of Bayreuth) · · cited 13× · DOI ↗ · more like this ·

In this article, we address the problem of modularizing legacy applications with monolithic structure, primarily focusing on business software written in an object-oriented programming language. We introduce theTransFormr toolkit that guides the developer through the entire incremental transformation process. It is the goal of the transformation to separate the original software into several independent replaceable components to support the migration of legacy code to new hardware or to integrate legacy components into modern enterprise applications. We show the effectiveness of our approach by demonstrating a pattern-based transformation of classes in a case study.

Hierarchical Clock Synchronization in MPI

Sascha Hunold, Alexandra Carpen-Amarie

Hardware and Architecture · Fraunhofer-Publica (Fraunhofer-Gesellschaft) · · cited 12× · DOI ↗ · more like this ·

MPI benchmarks are used for analyzing or tuning the performance of MPI libraries. Generally, every MPI library should be adjusted to the given parallel machine, especially on supercomputers. System operators can define which algorithm should be selected for a specific MPI operation, and this decision which algorithm to select is usually made after analyzing bench-mark results. The problem is that the latency of communication operations in MPI is very sensitive to the chosen data acquisition and data processing method. For that reason, depending on how the performance is measured, system operators may end up with a completely different MPI library setup. In the present work, we focus on the problem of precisely measuring the latency of collective operations, in particular, for small payloads, where external experimental factors play a significant role. We present a novel clock…

An Overhead Analysis of MPI Profiling and Tracing Tools

Sascha Hunold, Jordy I. Ajanohoun, Ioannis Vardas +1 more

Hardware and Architecture · · cited 10× · DOI ↗ · more like this ·

MPI performance analysis tools are important instruments for finding performance bottlenecks in large-scale MPI applications. These tools commonly support either the profiling or the tracing of parallel applications. Depending on the type of analysis, the use of such a performance analysis tool may entail a significant runtime overhead on the monitored parallel application. However, overheads can occur in different stages of the performance analysis with varying severity, e.g., the overhead when initializing an MPI context is typically less problematic than when monitoring a high number of short-lived MPI function calls.

On the Impact of Synchronizing Clocks and Processes on Benchmarking MPI Collectives

Sascha Hunold, Alexandra Carpen-Amarie

Computer Networks and Communications · · cited 10× · DOI ↗ · more like this ·

We consider the problem of accurately measuring the time to complete an MPI collective operation, as the result strongly depends on how the time is measured. Our goal is to develop an experimental method that allows for reproducible measurements of MPI collectives. When executing large parallel codes, MPI processes are often skewed in time when entering a collective operation. However, to obtain reproducible measurements, it is a common approach to synchronize all processes before they call the MPI collective operation. We therefore take a closer look at two commonly used process synchronization schemes: (1) relying on MPI_Barrier or (2) applying a window-based scheme using a common global time. We analyze both schemes experimentally and show the strengths and weaknesses of each approach. As window-based schemes require the notion of global time, we thoroughly evaluate different clock…

mpisee: MPI Profiling for Communication and Communicator Structure

Ioannis Vardas, Sascha Hunold, Jordy I. Ajanohoun +1 more

Computer Networks and Communications · 2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) · · cited 9× · DOI ↗ · more like this ·

Cumulative performance profiling is a fast and lightweight method for gaining summary information about where and how communication time in parallel MPI applications is spent. MPI provides mechanisms for implementing such profilers that can be transparently used with applications. Existing profilers typically profile on a process basis and record the frequency, total time, and volume of MPI operations per process. This can lead to grossly misleading cumulative information for applications that make use of MPI features for partitioning the processes into different communicators. We present a novel MPI profiler, mpisee, for communicator-centric profiling that separates and records collective and point-to-point communication information per communicator in the application. We discuss the implementation of mpisee which makes significant use of the MPI attribute mechanism. We evaluate our…

Cartesian Collective Communication

Jesper Larsson Träff, Sascha Hunold

Computer Networks and Communications · · cited 9× · DOI ↗ · more like this ·

We introduce Cartesian Collective Communication as sparse, collective communication defined on processes (processors) organized into d-dimensional tori or meshes. Processes specify local neighborhoods, e.g., stencil patterns, by lists of relative Cartesian coordinate offsets. The Cartesian collective operations perform data exchanges (and reductions) over the set of all neighborhoods such that each process communicates with the processes in its local neighborhood. The key requirement is that local neighborhoods must be structurally identical (isomorphic). This makes it possible for processes to compute correct, deadlock-free, efficient communication schedules for the collective operations locally without any interaction with other processes. Cartesian Collective Communication substantially extends collective neighborhood communication on Cartesian communicators as defined by the MPI…

Algorithm Selection of MPI Collectives Using Machine Learning Techniques

Sascha Hunold, Alexandra Carpen-Amarie

Hardware and Architecture · · cited 9× · DOI ↗ · more like this ·

Autotuning is a well established method to improve software performance for a given system, and it is especially important in High Performance Computing. The goal of autotuning is to find the best possible algorithm and its best parameter settings for a given instance. Autotuning can also be applied to MPI libraries, such as Open MPI or Intel MPI. These MPI libraries provide numerous parameters that allow users to adapt them to a given system. Some of these tunable parameters enable users to select a specific algorithm that should be used internally by an MPI collective operation. For the purpose of automatically tuning MPI collectives on a given system, the Intel MPI library is shipped with mpitune. The drawback of tools like mpitune is that results can only be applied to cases (e.g., number of processes, message size) for which the tool has performed the optimization. To overcome this…

Combining Object-Oriented Design and SOA with Remote Objects over Web Services

Marvin Ferber, Thomas Rauber, Sascha Hunold

Information Systems · ERef Bayreuth (University of Bayreuth) · · cited 9× · DOI ↗ · more like this ·

Current approaches of accessing stateful resources via SOAP Web services do not provide a standardized way to use program objects (classes). In this article, we show how the interface of an object-oriented class can be expressed using WSDL. This approach enables a program object to be used in a distributed environment by accessing its Web service interface. The set of Web services associated with a class define a Remote Object over Web Service (ROWS). A ROWS object can be used to facilitate the simultaneous use of the concepts of Service Oriented Architectures (SOA) and Distributed Object Architectures (DOA). In this context, we show how ROWS objects can be utilized in BPEL. Beyond that, we present a ROWS implementation that is suitable to serve as a standalone distributed object middleware. In a case study, we show how classes that were written in Java can be made accessible remotely…

Dynamic scheduling of multi-processor tasks on clusters of clusters

Sascha Hunold, Thomas Rauber, Gudula Rünger

Computer Networks and Communications · ERef Bayreuth (University of Bayreuth) · · cited 9× · DOI ↗ · more like this ·

In this article we tackle the problem of scheduling a dynamically generated DAG of multi-processor tasks (M-tasks). At first, we outline the need of such a scheduling approach in the context of TGrid. TGrid is an M-task runtime system for heterogeneous clusters. Then, we propose a dynamic scheduling algorithm called reuse processors algorithm (RePA). The main objective of RePA is to reduce the communication and redistribution costs by trying to map child tasks to processors which are assigned to parent tasks (reuse processors). The algorithm is implemented using the SimGrid toolkit and is evaluated by comparing the makespan of the schedules produced by RePA and M-HEFT.

A Survey on Reproducibility in Parallel Computing

Sascha Hunold

cs.DC · · cited 8× · arXiv ↗ · more like this ·

We summarize the results of a survey on reproducibility in parallel computing, which was conducted during the Euro-Par conference in August 2015. The survey form was handed out to all participants of the conference and the workshops. The questionnaire, which specifically targeted the parallel computing community, contained questions in four different categories: general questions on reproducibility, the current state of reproducibility, the reproducibility of the participants' own papers, and questions about the participants' familiarity with tools, software, or open-source software licenses used for reproducible research.