Archive

Ultrascale Playbook - Expert Parallelism

Ultrascale Playbook - Context Parallelism

Ultrascale Playbook - Tensor and Sequence Parallelism

Ultrascale Playbook - Pipeline Parallelism

Python in practice

Pydata MCR talk on training LLMs

Pydantic Logfire is awesome 🔥

Distributed communication for GPUs (part 2)

Distributed communication for GPUs (part 1)

Authenticating AWS with EKS

Choosing a batch size and provider for LLM training

Ultra-scale Playbook - ZeRO Sharding

Ultra-scale Playbook - Data Parallelism

Ultra-scale Playbook - Train on a single GPU

TIL this week

Hello World 👋