evaluation
Evaluates accuracy of quantized or unquantized LLMs using NeMo Evaluator Launcher (NEL). Triggers on "evaluate model", "benchmark accuracy", "run MMLU", "evaluate quantized model", "run nel". Handles deployment, config generation, and evaluation execution. Always load when validating completed runs or reporting evaluation scores, including standalone launching-evals analysis. Not for quantizing models (use ptq), standalone serving (use deployment), or comparing completed baseline-vs-quantized results (use compare-results).
Why this score
This repo hosts 10 skills, so its 5,374 stars are split between them (about 537 each) for scoring.
Popularity score: 77/100 from public GitHub signals only. It shows how widely used and maintained this is, not how well it works. Not included yet: the fake-star check. The security scan is shown separately and doesn't change the score. How we rank
Security
Scanned Oct 21, 2026 at commit f1d68fd4dd50, with Cisco's open-source skill-scanner.
What its code does:
- Runs shell commands Common for dev tools. Read the commands it runs.references/launcher-workflow.md:109
A scan lists patterns in the code. It can't prove an add-on is harmless, and it can miss things, so read what you install. How we scan
Alternatives
Other makers' add-ons that do the same job, next to this one.
| Name | Score | Stars | Context | Updated |
|---|---|---|---|---|
| This oneevaluation NVIDIA/Model-Optimizer · Skill | 77 | ★ 5.4k | ≈135 always | 16d ago |
| run-evaluation allenai/vla-evaluation-harness · Skill Same job: ai engineering | 72 | ★ 650 | ≈34 always | 20d ago |
| nat-evaluation NVIDIA/NeMo-Agent-Toolkit · Skill Same job: ai engineering | 75 | ★ 2.7k | ≈51 always | 17d ago |
| advanced-evaluation muratcankoylan/Agent-Skills-for-Context-Engineering · Skill Same job: ai engineering | 82 | ★ 18k | ≈59 always | 24d ago |
| agent-self-evaluation affaan-m/ECC · Skill Both about evaluation and accuracy | 96 | ★ 284k | ≈71 always | 16d ago |
| analyzing-offline-evaluations PostHog/ai-plugin · Skill Both about evaluation and compare | 49 | ★ 93 | ≈26 always | 16d ago |