arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Ariful Azad

75 papers · 2007–2026

2007: 22008: 02009: 12010: 12011: 12012: 22013: 02014: 12015: 52016: 52017: 32018: 22019: 32020: 132021: 62022: 52023: 42024: 82025: 52026: 8
2007peak 13 in 20202026

Frequent co-authors

Papers

showing 1–25 of 75 RSS

Exploiting Multiple Levels of Parallelism in Sparse Matrix-Matrix Multiplication

Ariful Azad, Grey Ballard, Aydin Buluc +5 more

cs.DC · · cited 114× · arXiv ↗ · more like this ·

Sparse matrix-matrix multiplication (or SpGEMM) is a key primitive for many high-performance graph algorithms as well as for some linear solvers, such as algebraic multigrid. The scaling of existing parallel implementations of SpGEMM is heavily bound by communication. Even though 3D (or 2.5D) algorithms have been proposed and theoretically analyzed in the flat MPI model on Erdos-Renyi matrices, those algorithms had not been implemented in practice and their complexities had not been analyzed for the general case. In this work, we present the first ever implementation of the 3D SpGEMM formulation that also exploits multiple (intra-node and inter-node) levels of parallelism, achieving significant speedups over the state-of-the-art publicly available codes at all levels of concurrencies. We extensively evaluate our implementation and identify bottlenecks that should be subject to further…

Integrated Model, Batch and Domain Parallelism in Training Neural Networks

Amir Gholami, Ariful Azad, Peter Jin +2 more

cs.LG · · cited 88× · arXiv ↗ · more like this ·

We propose a new integrated method of exploiting model, batch and domain parallelism for the training of deep neural networks (DNNs) on large distributed-memory computers using minibatch stochastic gradient descent (SGD). Our goal is to find an efficient parallelization strategy for a fixed batch size using $P$ processes. Our method is inspired by the communication-avoiding algorithms in numerical linear algebra. We see $P$ processes as logically divided into a $P_r \times P_c$ grid where the $P_r$ dimension is implicitly responsible for model/domain parallelism and the $P_c$ dimension is implicitly responsible for batch parallelism. In practice, the integrated matrix-based parallel algorithm encapsulates these types of parallelism automatically. We analyze the communication complexity and analytically demonstrate that the lowest communication costs are often achieved neither with pure…

A work-efficient parallel sparse matrix-sparse vector multiplication algorithm

Ariful Azad, Aydin Buluc

cs.DC · · cited 61× · arXiv ↗ · more like this ·

We design and develop a work-efficient multithreaded algorithm for sparse matrix-sparse vector multiplication (SpMSpV) where the matrix, the input vector, and the output vector are all sparse. SpMSpV is an important primitive in the emerging GraphBLAS standard and is the workhorse of many graph algorithms including breadth-first search, bipartite graph matching, and maximal independent set. As thread counts increase, existing multithreaded SpMSpV algorithms can spend more time accessing the sparse matrix data structure than doing arithmetic. Our shared-memory parallel SpMSpV algorithm is work efficient in the sense its total work is proportional to the number of arithmetic operations required. The key insight is to avoid each thread individually scan the list of matrix columns. Our algorithm is simple to implement and operates on existing column-based sparse matrix formats. It performs…

High-performance sparse matrix-matrix products on Intel KNL and multicore architectures

Yusuke Nagasaka, Satoshi Matsuoka, Ariful Azad +1 more

cs.DC · · cited 55× · arXiv ↗ · more like this ·

Sparse matrix-matrix multiplication (SpGEMM) is a computational primitive that is widely used in areas ranging from traditional numerical applications to recent big data analysis and machine learning. Although many SpGEMM algorithms have been proposed, hardware specific optimizations for multi- and many-core processors are lacking and a detailed analysis of their performance under various use cases and matrices is not available. We firstly identify and mitigate multiple bottlenecks with memory management and thread scheduling on Intel Xeon Phi (Knights Landing or KNL). Specifically targeting multi- and many-core processors, we develop a hash-table-based algorithm and optimize a heap-based shared-memory SpGEMM algorithm. We examine their performance together with other publicly available codes. Different from the literature, our evaluation also includes use cases that are representative…

Communication-Avoiding Parallel Sparse-Dense Matrix-Matrix Multiplication

Penporn Koanantakool, Ariful Azad, Aydın Buluç +4 more

Computer Networks and Communications · eScholarship (California Digital Library) · · cited 45× · DOI ↗ · more like this ·

Multiplication of a sparse matrix with a dense matrix is a building block of an increasing number of applications in many areas such as machine learning and graph algorithms. However, most previous work on parallel matrix multiplication considered only both dense or both sparse matrix operands. This paper analyzes the communication lower bounds and compares the communication costs of various classic parallel algorithms in the context of sparse-dense matrix-matrix multiplication. We also present new communication-avoiding algorithms based on a 1D decomposition, called 1.5D, which - while suboptimal in dense-dense and sparse-sparse cases - outperform the 2D and 3D variants both theoretically and in practice for sparse-dense multiplication. Our analysis separates one-time costs from per iteration costs in an iterative machine learning context. Experiments demonstrate speedups up to 100x…

GNNShap: Scalable and Accurate GNN Explanation using Shapley Values

Selahattin Akkas, Ariful Azad

cs.LG · · cited 41× · arXiv ↗ · more like this ·

Graph neural networks (GNNs) are popular machine learning models for graphs with many applications across scientific domains. However, GNNs are considered black box models, and it is challenging to understand how the model makes predictions. Game theoric Shapley value approaches are popular explanation methods in other domains but are not well-studied for graphs. Some studies have proposed Shapley value based GNN explanations, yet they have several limitations: they consider limited samples to approximate Shapley values; some mainly focus on small and large coalition sizes, and they are an order of magnitude slower than other explanation methods, making them inapplicable to even moderate-size graphs. In this work, we propose GNNShap, which provides explanations for edges since they provide more natural explanations for graphs and more fine-grained explanations. We overcome the…

The Reverse Cuthill-McKee Algorithm in Distributed-Memory

Ariful Azad, Mathias Jacquelin, Aydin Buluc +1 more

cs.DC · · cited 38× · arXiv ↗ · more like this ·

Ordering vertices of a graph is key to minimize fill-in and data structure size in sparse direct solvers, maximize locality in iterative solvers, and improve performance in graph algorithms. Except for naturally parallelizable ordering methods such as nested dissection, many important ordering methods have not been efficiently mapped to distributed-memory architectures. In this paper, we present the first-ever distributed-memory implementation of the reverse Cuthill-McKee (RCM) algorithm for reducing the profile of a sparse matrix. Our parallelization uses a two-dimensional sparse matrix decomposition. We achieve high performance by decomposing the problem into a small number of primitives and utilizing optimized implementations of these primitives. Our implementation shows strong scaling up to 1024 cores for smaller matrices and up to 4096 cores for larger matrices.

Combinatorial BLAS 2.0: Scaling combinatorial algorithms on distributed-memory systems

Ariful Azad, Oguz Selvitopi, Md Taufique Hussain +2 more

cs.DC · · cited 36× · arXiv ↗ · more like this ·

Combinatorial algorithms such as those that arise in graph analysis, modeling of discrete systems, bioinformatics, and chemistry, are often hard to parallelize. The Combinatorial BLAS library implements key computational primitives for rapid development of combinatorial algorithms in distributed-memory systems. During the decade since its first introduction, the Combinatorial BLAS library has evolved and expanded significantly. This paper details many of the key technical features of Combinatorial BLAS version 2.0, such as communication avoidance, hierarchical parallelism via in-node multithreading, accelerator support via GPU kernels, generalized semiring support, implementations of key data structures and functions, and scalable distributed I/O operations for human-readable files. Our paper also presents several rules of thumb for choosing the right data structures and functions in…

Bandwidth-Optimized Parallel Algorithms for Sparse Matrix-Matrix Multiplication using Propagation Blocking

Zhixiang Gu, Jose Moreira, David Edelsohn +1 more

cs.DC · · cited 32× · arXiv ↗ · more like this ·

Sparse matrix-matrix multiplication (SpGEMM) is a widely used kernel in various graph, scientific computing and machine learning algorithms. It is well known that SpGEMM is a memory-bound operation, and its peak performance is expected to be bound by the memory bandwidth. Yet, existing algorithms fail to saturate the memory bandwidth, resulting in suboptimal performance under the Roofline model. In this paper we characterize existing SpGEMM algorithms based on their memory access patterns and develop practical lower and upper bounds for SpGEMM performance. We then develop an SpGEMM algorithm based on outer product matrix multiplication. The newly developed algorithm called PB-SpGEMM saturates memory bandwidth by using the propagation blocking technique and by performing in-cache sorting and merging. For many practical matrices, PB-SpGEMM runs 20%-50% faster than the state-of-the-art…

Computing Maximum Cardinality Matchings in Parallel on Bipartite Graphs via Tree-Grafting

Ariful Azad, Aydın Buluç, Alex Pothen

Computer Vision and Pattern Recognition · IEEE Transactions on Parallel and Distributed Systems · · cited 32× · DOI ↗ · more like this ·

It is difficult to obtain high performance when computing matchings on parallel processors because matching algorithms explicitly or implicitly search for paths in the graph, and when these paths become long, there is little concurrency. In spite of this limitation, we present a new algorithm and its shared-memory parallelization that achieves good performance and scalability in computing maximum cardinality matchings in bipartite graphs. Our algorithm searches for augmenting paths via specialized breadth-first searches (BFS) from multiple source vertices, hence creating more parallelism than single source algorithms. Algorithms that employ multiple-source searches cannot discard a search tree once no augmenting path is discovered from the tree, unlike algorithms that rely on single-source searches. We describe a novel tree-grafting method that eliminates most of the redundant edge…

Multithreaded Algorithms for Maxmum Matching in Bipartite Graphs

Ariful Azad, Mahantesh Halappanavar, Sivasankaran Rajamanickam +3 more

Computer Vision and Pattern Recognition · · cited 32× · DOI ↗ · more like this ·

We design, implement, and evaluate algorithms for computing a matching of maximum cardinality in a bipartite graph on multicore and massively multithreaded computers. As computers with larger numbers of slower cores dominate the commodity processor market, the design of multithreaded algorithms to solve large matching problems becomes a necessity. Recent work on serial algorithms for the matching problem has shown that their performance is sensitive to the order in which the vertices are processed for matching. In a multithreaded environment, imposing a serial order in which vertices are considered for matching would lead to loss of concurrency and performance. But this raises the question: {\em Would parallel matching algorithms on multithreaded machines improve performance over a serial algorithm?}We answer this question in the affirmative. We report efficient multithreaded…

Evaluation of Graph Analytics Frameworks Using the GAP Benchmark Suite

Ariful Azad, Mohsen Mahmoudi Aznaveh, Scott Beamer +9 more

Computer Vision and Pattern Recognition · eScholarship (California Digital Library) · · cited 25× · DOI ↗ · more like this ·

Graphs play a key role in data analytics. Graphs and the software systems used to work with them are highly diverse. Algorithms interact with hardware in different ways and which graph solution works best on a given platform changes with the structure of the graph. This makes it difficult to decide which graph programming framework is the best for a given situation. In this paper, we try to make sense of this diverse landscape. We evaluate five different frameworks for graph analytics: SuiteS-parse GraphBLAS, Galois, the NWGraph library, the Graph Kernel Collection, and GraphIt. We use the GAP Benchmark Suite to evaluate each framework. GAP consists of 30 tests: six graph algorithms (breadth-first search, single-source shortest path, PageRank, betweenness centrality, connected components, and triangle counting) on five graphs. The GAP Benchmark Suite includes high-performance reference…

Distributed-Memory Algorithms for Maximum Cardinality Matching in Bipartite Graphs

Ariful Azad, Aydın Buluç

Computer Vision and Pattern Recognition · · cited 25× · DOI ↗ · more like this ·

We design and implement scalable distributed-memory algorithms for maximum cardinality matching in bipartite graphs. Computing matchings on distributed-memory supercomputers is challenged by the irregular andasynchronous data access patterns in graph searches and the difficulty in processing long pathspassing through multiple processors. We address these challenges by developing an algorithm based on matrix algebra. We employ bulk-synchronous matrix algebraic modules to implement graph searches, and Remote Memory Access (RMA) operations to map asynchronous light-weight graph accesses. On real matrices, our algorithm achieves up to 18x speedup when we go from 24 cores to 2048cores of a Cray XC30 supercomputer. Even higher speedups are obtained on larger synthetically generated graphs where ouralgorithms show good scaling on up to 12,000 cores.

LACC: A Linear-Algebraic Algorithm for Finding Connected Components in Distributed Memory

Ariful Azad, Aydın Buluç

Computer Vision and Pattern Recognition · · cited 23× · DOI ↗ · more like this ·

Finding connected components is one of the most widely used operations on a graph. Optimal serial algorithms for the problem have been known for half a century, and many competing parallel algorithms have been proposed over the last several decades under various different models of parallel computation. This paper presents a parallel connected-components algorithm that can run on distributed-memory computers. Our algorithm uses linear algebraic primitives and is based on a PRAM algorithm by Awerbuch and Shiloach. We show that the resulting algorithm, named LACC for Linear Algebraic Connected Components, outperforms competitors by a factor of up to 12× for small to medium scale graphs. For large graphs with more than 50B edges, LACC scales to 4K nodes (262K cores) of a Cray XC40 supercomputer and outperforms previous algorithms by a significant margin. This remarkable performance is…

The Parallelism Motifs of Genomic Data Analysis

Katherine Yelick, Aydin Buluc, Muaaz Awan +11 more

cs.DC · · cited 20× · arXiv ↗ · more like this ·

Genomic data sets are growing dramatically as the cost of sequencing continues to decline and small sequencing devices become available. Enormous community databases store and share this data with the research community, but some of these genomic data analysis problems require large scale computational platforms to meet both the memory and computational requirements. These applications differ from scientific simulations that dominate the workload on high end parallel systems today and place different requirements on programming support, software libraries, and parallel architectural design. For example, they involve irregular communication patterns such as asynchronous updates to shared data structures. We consider several problems in high performance genomics analysis, including alignment, profiling, clustering, and assembly for both single genomes and metagenomes. We identify some of…

Codesign Lessons Learned from Implementing Graph Matching on Multithreaded Architectures

Mahantesh Halappanavar, Alex Pothen, Ariful Azad +3 more

Computer Vision and Pattern Recognition · Computer · · cited 18× · DOI ↗ · more like this ·

Executing irregular, data-intensive workloads on multithreaded architectures can result in performance losses and scalability problems. Codesigning algorithms and architectures can realize high performance on irregular applications. A codesign study reveals four key lessons learned from implementing matching algorithms on various platforms.

Relational Learning Improves Prediction of Mortality in COVID-19 in the Intensive Care Unit

Tingyi Wanyan, Akhil Vaid, Jessica K. De Freitas +6 more

Artificial Intelligence · IEEE Transactions on Big Data · · cited 17× · DOI ↗ · more like this ·

Traditional Machine Learning (ML) models have had limited success in predicting Coronoavirus-19 (COVID-19) outcomes using Electronic Health Record (EHR) data partially due to not effectively capturing the inter-connectivity patterns between various data modalities. In this work, we propose a novel framework that utilizes relational learning based on a heterogeneous graph model (HGM) for predicting mortality at different time windows in COVID-19 patients within the intensive care unit (ICU). We utilize the EHRs of one of the largest and most diverse patient populations across five hospitals in major health system in New York City. In our model, we use an LSTM for processing time varying patient data and apply our proposed relational learning strategy in the final output layer along with other static features. Here, we replace the traditional softmax layer with a Skip-Gram relational…

EXAGRAPH: Graph and combinatorial methods for enabling exascale applications

Seher Acer, Ariful Azad, Erik G. Boman +9 more

Computer Vision and Pattern Recognition · The International Journal of High Performance Computing Applications · · cited 16× · DOI ↗ · more like this ·

Combinatorial algorithms in general and graph algorithms in particular play a critical enabling role in numerous scientific applications. However, the irregular memory access nature of these algorithms makes them one of the hardest algorithmic kernels to implement on parallel systems. With tens of billions of hardware threads and deep memory hierarchies, the exascale computing systems in particular pose extreme challenges in scaling graph algorithms. The codesign center on combinatorial algorithms, ExaGraph, was established to design and develop methods and techniques for efficient implementation of key combinatorial (graph) algorithms chosen from a diverse set of exascale applications. Algebraic and combinatorial methods have a complementary role in the advancement of computational science and engineering, including playing an enabling role on each other. In this paper, we survey the…

A Parallel Tree Grafting Algorithm for Maximum Cardinality Matching in Bipartite Graphs

Ariful Azad, Aydın Buluç, Alex Pothen

Computer Vision and Pattern Recognition · · cited 13× · DOI ↗ · more like this ·

In computing matching's in graphs on parallel processors, it is challenging to achieve high performance because these algorithms rely on searching for paths in the graph, and when these paths become long, there is little concurrency. We present a new algorithm and its shared-memory parallelization for computing maximum cardinality matching's in bipartite graphs. Our algorithm searches for augmenting paths via specialized breadth-first searches (BFS) from multiple source vertices, hence creating more parallelism than single source algorithms. Unfortunately, algorithms that employ multiple-source searches cannot discard a search tree once no augmenting path is discovered from the tree, unlike algorithms that rely on single-source searches. We describe a novel tree-grafting method that eliminates most of the redundant edge traversals resulting from this property of multiple-source…

FastSV: A Distributed-Memory Connected Component Algorithm with Fast Convergence

Yongzhe Zhang, Ariful Azad, Zhenjiang Hu

cs.DS · · cited 9× · arXiv ↗ · more like this ·

This paper presents a new distributed-memory algorithm called FastSV for finding connected components in an undirected graph. Our algorithm simplifies the classic Shiloach-Vishkin algorithm and employs several novel and efficient hooking strategies for faster convergence. We map different steps of FastSV to linear algebraic operations and implement them with the help of scalable graph libraries. FastSV uses sparse operations to avoid redundant work and optimized MPI communication to avoid bottlenecks. The resultant algorithm shows high-performance and scalability as it can find the connected components of a hyperlink graph with over 134B edges in 30 seconds using 262K cores on a Cray XC40 supercomputer. FastSV outperforms the state-of-the-art algorithm by an average speedup of 2.21x (max 4.27x) on a variety of real-world graphs.

Two-Face: Combining Collective and One-Sided Communication for Efficient Distributed SpMM

Charles Block, Gerasimos Gerogiannis, Charith Mendis +2 more

Artificial Intelligence · · cited 8× · DOI ↗ · more like this ·

Sparse matrix dense matrix multiplication (SpMM) is commonly used in applications ranging from scientific computing to graph neural networks. Typically, when SpMM is executed in a distributed platform, communication costs dominate. Such costs depend on how communication is scheduled. If it is scheduled in a sparsity-unaware manner, such as with collectives, execution is often inefficient due to unnecessary data transfers. On the other hand, if communication is scheduled in a fine-grained sparsity-aware manner, communicating only the necessary data, execution can also be inefficient due to high software overhead.

Supervised pretraining through contrastive categorical positive samplings to improve COVID-19 mortality prediction

Tingyi Wanyan, Mingquan Lin, Eyal Klang +9 more

Artificial Intelligence · PubMed · · cited 8× · DOI ↗ · more like this ·

Clinical EHR data is naturally heterogeneous, where it contains abundant sub-phenotype. Such diversity creates challenges for outcome prediction using a machine learning model since it leads to high intra-class variance. To address this issue, we propose a supervised pre-training model with a unique embedded k-nearest-neighbor positive sampling strategy. We demonstrate the enhanced performance value of this framework theoretically and show that it yields highly competitive experimental results in predicting patient mortality in real-world COVID-19 EHR data with a total of over 7,000 patients admitted to a large, urban health system. Our method achieves a better AUROC prediction score of 0.872, which outperforms the alternative pre-training models and traditional machine learning methods. Additionally, our method performs much better when the training data size is small (345 training…

Bootstrapping Your Own Positive Sample: Contrastive Learning With Electronic Health Record Data

Tingyi Wanyan, Jing Zhang, Ying Ding +3 more

cs.LG · · cited 8× · arXiv ↗ · more like this ·

Electronic Health Record (EHR) data has been of tremendous utility in Artificial Intelligence (AI) for healthcare such as predicting future clinical events. These tasks, however, often come with many challenges when using classical machine learning models due to a myriad of factors including class imbalance and data heterogeneity (i.e., the complex intra-class variances). To address some of these research gaps, this paper leverages the exciting contrastive learning framework and proposes a novel contrastive regularized clinical classification model. The contrastive loss is found to substantially augment EHR-based prediction: it effectively characterizes the similar/dissimilar patterns (by its "push-and-pull" form), meanwhile mitigating the highly skewed class distribution by learning more balanced feature spaces (as also echoed by recent findings). In particular, when naively exporting…

A Scalable Method for Readable Tree Layouts

Kathryn Gray, Mingwei Li, Reyan Ahmed +4 more

cs.CG · · cited 6× · arXiv ↗ · more like this ·

Large tree structures are ubiquitous and real-world relational datasets often have information associated with nodes (e.g., labels or other attributes) and edges (e.g., weights or distances) that need to be communicated to the viewers. Yet, scalable, easy to read tree layouts are difficult to achieve. We consider tree layouts to be readable if they meet some basic requirements: node labels should not overlap, edges should not cross, edge lengths should be preserved, and the output should be compact. There are many algorithms for drawing trees, although very few take node labels or edge lengths into account, and none optimizes all requirements above. With this in mind, we propose a new scalable method for readable tree layouts. The algorithm guarantees that the layout has no edge crossings and no label overlaps, and optimizes one of the remaining aspects: desired edge lengths and…

Security Attack Detection using Genetic Algorithm (GA) in Policy Based Network

Ariful Islam, Ariful Azad, Khurshid Alam +1 more

Computer Networks and Communications · · cited 6× · DOI ↗ · more like this ·

We present a modern approach of detecting the computer network security attacks (commonly termed as "hacks") by using an artificial intelligence method known as a genetic algorithm, which is a variant of stochastic beam search, which can be applied to a Policy Based Network. For this experiment, we develop the corresponding form of genetic algorithm and the form of fitness function to detect security attacks. Then we test the algorithm by a simulation program in TC (Turbo C) to gauge its effectiveness and the side effects of its execution. Finally, we give the indication for the necessary change in policy framework to deploy the functionality.