arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Computer Science · Computer Networks and Communications

An Optimization Pass for Training Speed-Up and Strategy Search in 3D Parallelism

Ryubu Hosoki, Kento Sato, Toshio Endo, Julien Bigot, E. Audit

See on the map

Abstract

Large deep neural networks (DNNs) have made remarkable progress across various applications. Training a large model requires enormous computational and memory capacity; thus, it requires large-scale parallelization and distribution.

More like this