arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Benjamin Y. Cho

7 papers · 2013–2023

2013: 12014: 02015: 12016: 02017: 12018: 02019: 12020: 22021: 02022: 02023: 1
2013peak 2 in 20202023

Frequent co-authors

Papers

showing 1–7 of 7 RSS

MCM-GPU

Akhil Arunkumar, Evgeny Bolotin, Benjamin Cho +6 more

Hardware and Architecture · · cited 160× · DOI ↗ · more like this ·

Historically, improvements in GPU-based high performance computing have been tightly coupled to transistor scaling. As Moore's law slows down, and the number of transistors per die no longer grows at historical rates, the performance curve of single monolithic GPUs will ultimately plateau. However, the need for higher performing GPUs continues to exist in many domains. To address this need, in this paper we demonstrate that package-level integration of multiple GPU modules to build larger logical GPUs can enable continuous performance scaling beyond Moore's law. Specifically, we propose partitioning GPUs into easily manufacturable basic GPU Modules (GPMs), and integrating them on package using high bandwidth and power efficient signaling technologies. We lay out the details and evaluate the feasibility of a basic Multi-Chip-Module GPU (MCM-GPU) design. We then propose three…

Near Data Acceleration with Concurrent Host Access

Benjamin Y. Cho, Yongkee Kwon, Sangkug Lym +1 more

cs.AR · · cited 50× · arXiv ↗ · more like this ·

Near-data accelerators (NDAs) that are integrated with main memory have the potential for significant power and performance benefits. Fully realizing these benefits requires the large available memory capacity to be shared between the host and the NDAs in a way that permits both regular memory access by some applications and accelerating others with an NDA, avoids copying data, enables collaborative processing, and simultaneously offers high performance for both host and NDA. We identify and solve new challenges in this context: mitigating row-locality interference from host to NDAs, reducing read/write-turnaround overhead caused by fine-grain interleaving of host and NDA requests, architecting a memory layout that supports the locality required for NDAs and sophisticated address interleaving for host performance, and supporting both packetized and traditional memory interfaces. We…

Accelerating Bandwidth-Bound Deep Learning Inference with Main-Memory Accelerators

Benjamin Y. Cho, Jeageun Jung, Mattan Erez

cs.AR · · cited 38× · arXiv ↗ · more like this ·

DL inference queries play an important role in diverse internet services and a large fraction of datacenter cycles are spent on processing DL inference queries. Specifically, the matrix-matrix multiplication (GEMM) operations of fully-connected MLP layers dominate many inference tasks. We find that the GEMM operations for datacenter DL inference tasks are memory bandwidth bound, contrary to common assumptions: (1) strict query latency constraints force small-batch operation, which limits reuse and increases bandwidth demands; and (2) large and colocated models require reading the large weight matrices from main memory, again requiring high bandwidth without offering reuse opportunities. We demonstrate the large potential of accelerating these small-batch GEMMs with processing in the main CPU memory. We develop a novel GEMM execution flow and corresponding memory-side address-generation…

XSD: Accelerating MapReduce by Harnessing the GPU inside an SSD

Benjamin Y. Cho, Won Seob Jeong, Doohwan Oh +1 more

Computer Networks and Communications · · cited 26× · OpenAlex ↗ · more like this ·

Considerable research has been conducted recently on near-data processing techniques as real-world tasks increasingly involve large-scale and high-dimensional data sets. The advent of solid-state drives (SSDs) has spurred further research because of their processing capability and high internal bandwidth. However, the data processing capability of conventional SSD systems have not been impressive. In particular, they lack the parallel processing abilities that are crucial for data-centric workloads and that are needed to exploit the high internal bandwidth of the SSD. To overcome these shortcomings, we propose a new SSD architecture that integrates a graphics processing unit (GPU). We provide API sets based on the MapReduce framework that allow users to express parallelism in their application, and that exploit the parallelism provided by the embedded GPU. For better performance and…

Training with Multi-Layer Embeddings for Model Reduction

Benjamin Ghaemmaghami, Zihao Deng, Benjamin Cho +4 more

cs.LG · · cited 5× · arXiv ↗ · more like this ·

Modern recommendation systems rely on real-valued embeddings of categorical features. Increasing the dimension of embedding vectors improves model accuracy but comes at a high cost to model size. We introduce a multi-layer embedding training (MLET) architecture that trains embeddings via a sequence of linear layers to derive superior embedding accuracy vs. model size trade-off. Our approach is fundamentally based on the ability of factorized linear layers to produce superior embeddings to that of a single linear layer. We focus on the analysis and implementation of a two-layer scheme. Harnessing the recent results in dynamics of backpropagation in linear neural networks, we explain the ability to get superior multi-layer embeddings via their tendency to have lower effective rank. We show that substantial advantages are obtained in the regime where the width of the hidden layer is much…

Network Variation and Fault Tolerant Performance Acceleration in Mobile Devices with Simultaneous Remote Execution

Keun‐Soo Kim, Benjamin Y. Cho, Won Woo Ro +1 more

Computer Networks and Communications · IEEE Transactions on Computers · · cited 1× · DOI ↗ · more like this ·

As mobile applications provide increasingly richer features to end users, it has become imperative to overcome the constraints of a resource-limited mobile hardware. Remote execution is one promising technique to resolve this important problem. Using this technique, the computation intensive part of the workload is migrated to resource-rich servers, and then once the computation is completed, the results can be returned to the client devices. To enable this operation, strong wireless connectivity is required. However, unstable wireless connections are the staple of real-life. This makes performance unpredictable, sometimes offsetting the benefits brought by this technique and leading to performance degradation. To address this problem, in this paper, we present a Simultaneous Remote Execution (SRE) model for mobile devices. Our SRE model performs concurrent executions both locally and…

Enhancing Cross-Category Learning in Recommendation Systems with Multi-Layer Embedding Training

Zihao Deng, Benjamin Ghaemmaghami, Ashish Kumar Singh +4 more

cs.LG · · cited 0× · arXiv ↗ · more like this ·

Modern DNN-based recommendation systems rely on training-derived embeddings of sparse features. Input sparsity makes obtaining high-quality embeddings for rarely-occurring categories harder as their representations are updated infrequently. We demonstrate a training-time technique to produce superior embeddings via effective cross-category learning and theoretically explain its surprising effectiveness. The scheme, termed the multi-layer embeddings training (MLET), trains embeddings using factorization of the embedding layer, with an inner dimension higher than the target embedding dimension. For inference efficiency, MLET converts the trained two-layer embedding into a single-layer one thus keeping inference-time model size unchanged. Empirical superiority of MLET is puzzling as its search space is not larger than that of the single-layer embedding. The strong dependence of MLET on the…