-
Notifications
You must be signed in to change notification settings - Fork 4.7k
Open
Labels
enhancementNew feature or requestNew feature or request
Description
This issue aims to track the progress, bugs, and requests regarding the support of torch.compile.
@tjruwase
Tasks
- ZeRO3 support (Enable torch.compile with ZeRO (Experimental) #4878, we currently break the graph to make communication collectives work)
- Pipeline parallel support ([REQUEST] torch.compile + DeepSpeed #4677)
- Activation checkpointing with Megatron-DeepSpeed (Enable torch.compile Megatron-DeepSpeed#322)
- Pass HuggingFace CI tests
PRs
- Enable torch.compile with ZeRO (Experimental) #4878: The initial proposal of the official
torch.compilesupport
Issues
tjruwase, stas00, loadams, yundai424, oraluben and 3 more
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request