Home / Skills / distributed-llm-pretraining-torchtitan
distributed-llm-pretraining-torchtitan
Orchestra-Research/AI-Research-SKILLsSkill
Provides PyTorch-native distributed LLM pretraining using torchtitan with 4D parallelism (FSDP2, TP, PP, CP). Use when pretraining Llama 3.1, DeepSeek V3, or custom models at scale from 8 to 512+ GPUs with Float8, torch.compile, and distributed checkpointing.
Why this score
This repo hosts 10 skills, so its 13,721 stars are split between them (about 1,372 each) for scoring.
Popularity score: 77/100 from public GitHub signals only. It shows how widely used and maintained this is, not how well it works. Not included yet: the fake-star check. The security scan is shown separately and doesn't change the score. How we rank
Security
Scanned Oct 20, 2026 at commit 773a52944ba4, with Cisco's open-source skill-scanner.
What its code does:
- Uses dependencies without a pinned version A later version of a dependency could change what runs.SKILL.md and 2 more
A scan lists patterns in the code. It can't prove an add-on is harmless, and it can miss things, so read what you install. How we scan
Alternatives
Other makers' add-ons that do the same job, next to this one.
| Name | Score | Stars | Context | Updated |
|---|---|---|---|---|
| This onedistributed-llm-pretraining-torchtitan Orchestra-Research/AI-Research-SKILLs · Skill | 77 | ★ 14k | ≈75 always | 25d ago |
| distributed sql awslabs/mcp · Skill Both about distributed | 82 | ★ 9.8k | ≈58 always | 16d ago |
| distributed-search proyecto26/system-design-skills · Skill Both about distributed | 48 | ★ 93 | ≈132 always | 5mo ago |
| designing-distributed-system-tests shenli/distributed-system-testing · Skill Both about distributed | 58 | ★ 231 | ≈159 always | 4mo ago |