Welcome to my small corner of the internet where I share things I’m learning, building, and exploring.
- I work as MLOps Engineer at Fuzzy Labs
- You can find me on the following socials.
Data Parallelism Revisited - Implementations using PyTorch
Implement various data parallelism strategies using PyTorch
Ultrascale Playbook - Expert Parallelism
Notes on training LLMs using expert parallelism
Ultrascale Playbook - Context Parallelism
Notes on training LLMs using context parallelism
Ultrascale Playbook - Tensor and Sequence Parallelism
Notes on training LLMs using tensor and sequence parallelism
Ultrascale Playbook - Pipeline Parallelism
Notes on training LLMs using pipeline parallelism