Pipeline Parallelism Revisited - Implementations using PyTorch
Implementing and profiling pipeline parallelism techniques using PyTorch
Posts
TIL - Cloud VMs, Unikernels and hypervisors
Anatomy of different layers of abstractions for a cloud compute
ZeRO Sharding Revisited - Implementations using PyTorch
Implement ZeRO sharding strategies using PyTorch
Python in practice - Config driven design
Patterns in python code design
Data Parallelism Revisited - Implementations using PyTorch
Implement various data parallelism strategies using PyTorch