DeepSpeed
≈ 1 min read
Names in other languages
No names in other languages are held for this entry yet.
Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.
DeepSpeed is an open-source optimization library for the distributed training and inference of deep learning models using PyTorch.
DeepSpeed is an open-source optimization library for the distributed training and inference of deep learning models using PyTorch. ==Library== The library is designed to reduce computing power and memory use and to train large distributed models with better parallelism on existing computer hardware. DeepSpeed is optimized for low latency, high throughput training. It includes the Zero Redundancy Optimizer (ZeRO) for training models with 1 trillion or more parameters. Features include mixed precision training, single-GPU, multi-GPU, and multi-node training as well as custom model parallelism. The DeepSpeed source code is licensed under Apache License and available on GitHub. The team claimed to achieve up to a 6.2x throughput improvement, 2.8x faster convergence, and 4.6x less communication. ==See also== Comparison of deep learning software Deep learning Machine learning TensorFlow ==References== ==Further reading== ==External links== AI at Scale - Microsoft Research GitHub - microsoft/DeepSpeed ZeRO & DeepSpeed: New system optimizations enable training models with over 100 billion parameters - Microsoft Research
Text derived from https://en.wikipedia.org/w/index.php?oldid=1377654878 under CC BY-SA 4.0; modifications have been made.