arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Enda Yu

11 papers · 2009–2026

2009: 12010: 02011: 02012: 02013: 02014: 02015: 02016: 02017: 02018: 02019: 02020: 02021: 12022: 02023: 12024: 12025: 22026: 5
2009peak 5 in 20262026

Frequent co-authors

Papers

showing 1–11 of 11 RSS

Communication Optimization Algorithms for Distributed Deep Learning Systems: A Survey

Enda Yu, Dezun Dong, Xiangke Liao

Computer Vision and Pattern Recognition · IEEE Transactions on Parallel and Distributed Systems · · cited 12× · DOI ↗ · more like this ·

Deep learning's widespread adoption in various fields has made distributed training across multiple computing nodes essential. However, frequent communication between nodes can significantly slow down training speed, creating a bottleneck in distributed training. To address this issue, researchers are focusing on communication optimization algorithms for distributed deep learning systems. In this paper, we propose a standard that systematically classifies all communication optimization algorithms based on mathematical modeling, which is not achieved by existing surveys in the field. We categorize existing works into four categories based on the optimization strategies of communication: communication masking, communication compression, communication frequency reduction, and hybrid optimization. Finally, we discuss potential future challenges and research directions in the field of…

CD-SGD: Distributed Stochastic Gradient Descent with Compression and Delay Compensation

Enda Yu, Dezun Dong, Yemao Xu +2 more

cs.LG · · cited 6× · arXiv ↗ · more like this ·

Communication overhead is the key challenge for distributed training. Gradient compression is a widely used approach to reduce communication traffic. When combining with parallel communication mechanism method like pipeline, gradient compression technique can greatly alleviate the impact of communication overhead. However, there exists two problems of gradient compression technique to be solved. Firstly, gradient compression brings in extra computation cost, which will delay the next training iteration. Secondly, gradient compression usually leads to the decrease of convergence accuracy.

Human action recognition with extremities as semantic posture representation

Enda Yu, J.K. Aggarwal

Computer Vision and Pattern Recognition · 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · · cited 6× · DOI ↗ · more like this ·

In this paper, we present an approach for human action recognition with extremities as a compact semantic posture representation. First, we develop a variable star skeleton representation (VSS) in order to accurately find human extremities from contours. Earlier, Fujiyoshi and Lipton proposed an image skeletonization technique with the center of mass as a single star for rapid motion analysis. Yu and Aggarwal used the highest contour point as the second star in their application for fence climbing detection. We implement VSS and earlier algorithms and compare their performance over a set of 1000 frames from 50 sequences of persons climbing fences to analyze the characteristic of each representation. Our results show that VSS performs the best. Second, we build feature vectors out of detected extremities for hidden Markov model (HMM) based human action recognition. On the data set of…

PDScope: Breaking the Prefill-Decode Seesaw for Memory-Bound MoE Inference

Enda Yu, Dezun Dong, Zhaoning Zhang +6 more

cs.DC · · cited 1× · arXiv ↗ · more like this ·

Local deployment of MoE-based LLMs on CPU-GPU nodes is attractive for cost and privacy reasons, yet limited GPU memory forces most experts into CPU memory. Prefill is then compute-bound and benefits from aggressive GPU loading, while decode is I/O-bound and requires conservative PCIe transfers. These demands cannot be amortized under low batching---typical of local assistants and document tools---forcing phase-agnostic schedulers into a seesaw: favoring prefill's aggressive loading injects unmaskable I/O that stalls decode, while favoring decode's conservative transfers starves prefill's GPU of work. We observe that expert routing is predictable in phase-specific ways, allowing us to decouple the two phases and optimize prediction, scheduling, and execution independently for each phase. PDScope predicts expert activations to guide placement and prefetching, adapts these policies to each…

HiNa-MoE: High-Performance, Non-Intrusive MoE Inference on CPUs with Matrix Engines

Weiling Yang, Junwen Zhang, Dezun Dong +4 more

cs.DC · · cited 0× · arXiv ↗ · more like this ·

Mixture-of-Experts (MoE) inference is increasingly deployed in local and on-premise environments, where expert parameters often exceed GPU memory capacity. In latency-sensitive, low-concurrency settings, repeatedly staging routed-expert weights from CPU memory to the GPU can be prohibitive, leaving routed-expert feed-forward networks (FFNs) on the critical path of multi-socket CPUs. Existing CPU accelerations often rely on intrusive, hardware- or topology-specific requirements, such as AMX-specific weight layouts or manual NUMA-aware placement. These requirements reduce portability and complicate integration with standard CPU-GPU offloading pipelines. We present HiNa-MoE, a high-performance, non-intrusive operator library for MoE inference on CPUs with Intel AMX. HiNa-MoE (1) exploits AMX with an optimized micro-kernel that keeps expert weights in standard layouts and instead fuses…

Physically Partitioned KVCache Format for CPU--GPU Load Balancing in MoE Inference

Enda Yu, Dezun Dong, Xiangke Liao

cs.DC · · cited 0× · arXiv ↗ · more like this ·

Single-GPU long-context inference with Mixture-of-Experts (MoE) models requires spilling the key-value cache (KVCache) to CPU memory. The spilled KV serves two complementary purposes---transferring to the GPU for attention computation, or computing in-place on the CPU---which demand opposing physical states. The optimal split between them varies with workload, yet existing KVCache abstractions offer only storage semantics over a monolithic object of a single physical state, and cannot express dynamic load balancing. We propose InplaceKVCache, the first KVCache abstraction whose format fixes each byte's physical residency at write time, so that the CPU--GPU load balance can be adjusted without moving data after placement. It realizes this as a four-region layout along two dimensions---device affinity and access pattern---turning load balancing into pure scheduling. Built on this…

Liberating GPU with cocktails: Enhancing gradient compression for distributed deep learning

Enda Yu, Zhe Bai, Dezun Dong +1 more

Artificial Intelligence · Computer Networks · · cited 0× · DOI ↗ · more like this ·

Gradient compression is a crucial technology for mitigating communication bottlenecks in distributed deep learning (DDL). However, achieving significant performance improvements with gradient compression remains challenging. In practical applications, gradient compression faces several challenges: (1) it fails to effectively optimize the initial overhead of small-scale tensor communication; (2) compression operations might compete with tensor computations for GPU resources, thereby delaying gradient transmission; (3) it can potentially cause accuracy issues that require careful handling. To maximize the benefits of gradient compression and address these challenges, we propose the Cocktails system. Additionally, to circumvent contention for GPU compute and memory resources, Cocktails offloads compression operations to the CPU and employs a model backup technique that decouples read…

LLMEmu: A Lightweight Performance Emulator for High-Fidelity Distributed LLM Training

Siyuan Yang, Enda Yu, Pingjing Lu +1 more

Hardware and Architecture · ACM SIGMETRICS Performance Evaluation Review · · cited 0× · DOI ↗ · more like this ·

The prohibitive cost of training trillion-parameter large language models (LLMs) necessitates low-cost emulation tools for distributed system optimization. In modern large-scale clusters, communication often becomes the primary bottleneck to scalability. However, existing emulators, such as vTrain and ASTRA-Sim, overlook dynamic network factors that significantly impact performance at scale, resulting in limited emulation accuracy. This work offers an efficient and reliable tool for training system optimization and parallel strategy exploration, considerably lowering the barrier to large-scale AI research. We present LLMEmu, a distributed training emulator that combines real kernel profiling and actual communication execution. First, computation is profiled through real CUDA kernel traces on GPU nodes to construct an operator-level latency lookup table, enabling GPUlike execution on CPU…

Nezha: Breaking Multi-Rail Network Barriers for Distributed DNN Training

Enda Yu, Dezun Dong, Xiangke Liao

cs.DC · · cited 0× · arXiv ↗ · more like this ·

In distributed deep learning, communication remains a critical bottleneck. While modern hardware advances rapidly, over 60 percent of production HPC systems still rely on legacy infrastructure (V100 GPUs, multi-plane Ethernet/InfiniBand), necessitating communication optimization without hardware upgrades. Existing approaches face three key limitations: (1) static single-rail binding underutilizes multi-rail bandwidth, (2) protocol heterogeneity (TCP-RDMA coexistence) causes synchronization delays, and (3) mainstream libraries (NCCL/MPI) lack cross-protocol coordination. We present Nezha, the first protocol-agnostic system for multi-rail networks. Our contributions include: (1) Hardware-agnostic cross-protocol coordination: A unified abstraction enabling seamless collaboration between in-network computing (SHARP), adaptive RDMA (GLEX), and TCP, achieving 1.7 to 4.3 times lower latency…

LayerScope: Predictive Cross-Layer Scheduling for Efficient Multi-Batch MoE Inference on Legacy Servers

Enda Yu, Dezun Dong, Zhaoning Zhang +6 more

cs.LG · · arXiv ↗ · more like this ·

Mixture-of-Experts (MoE) models face memory and PCIe latency bottlenecks when deployed on commodity hardware. Offloading expert weights to CPU memory results in PCIe transfer latency that exceeds GPU computation by several folds. We present PreScope, a prediction-driven expert scheduling system that addresses three key challenges: inaccurate activation prediction, PCIe bandwidth competition, and cross-device scheduling complexity. Our solution includes: 1) Learnable Layer-Aware Predictor (LLaPor) that captures layer-specific expert activation patterns; 2) Prefetch-Aware Cross-Layer Scheduling (PreSched) that generates globally optimal plans balancing prefetching costs and loading overhead; 3) Asynchronous I/O Optimizer (AsyncIO) that decouples I/O from computation, eliminating waiting bubbles. PreScope achieves 141% higher throughput and 74.6% lower latency than state-of-the-art…