Distributed communication for GPUs (part 1)
Introduction to distributed communication for GPUs.
Posts
Authenticating AWS with EKS
How to authenticate EKS workloads with AWS services?
Choosing a batch size and provider for LLM training
Notes on choosing appropriate batch size and compute for training LLMs
Ultra-scale Playbook - Deepspeed ZeRO
Notes on training LLMs using sharding strategies
Ultra-scale Playbook - Data Parallelism
Notes on training LLMs using data parallelism strategy