Ultrascale Playbook - Context Parallelism
Notes on training LLMs using context parallelism
Posts
Ultrascale Playbook - Tensor and Sequence Parallelism
Notes on training LLMs using tensor and sequence parallelism
Ultrascale Playbook - Pipeline Parallelism
Notes on training LLMs using pipeline parallelism
Python in practice
Learnings from reading the python code for semlib library
Pydata MCR talk on training LLMs
My talk on training LLMs at Pydata MCR