DeepSpeed: Accelerating large-scale model inference and training

miro.medium.com/v2/resize:fit:2000/1*WkGUbKgwpsihJ

DeepSpeed - Make distributed training easy, efficient, and effective

DeepSpeed ZeRO++: A leap in speed for LLM and chat model training

DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training

LLM Inference Unveiled: Survey and Roofline Model Insights

SW/HW Co-optimization Strategy for LLMs — Part 2 (Software), by Liz Li

AI at Scale: Timeline - Microsoft Research

AI at Scale: Timeline - Microsoft Research

miro.medium.com/v2/resize:fit:1400/1*DafLIAEn1yQAx

Training Causal Language Models on SDSC's Gaudi-based Voyager

N] Improvement on model's inference from DeepSpeed team. [D] How is Jax compared? : r/MachineLearning

miro.medium.com/v2/resize:fit:1400/1*EDndx6q1g7C_d

Training your own ChatGPT-like model