Home / Skills / evaluating-llms-harness
evaluating-llms-harness
Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.
Why this score
This repo hosts 10 skills, so its 1,341 stars are split between them (about 134 each) for scoring.
Popularity score: 70/100 from public GitHub signals only. It shows how widely used and maintained this is, not how well it works. Not included yet: the fake-star check. The security scan is shown separately and doesn't change the score. How we rank
Security
Scanned Oct 20, 2026 at commit 584fd06f295b, with Cisco's open-source skill-scanner.
What its code does:
- Runs shell commands Common for dev tools. Read the commands it runs.SKILL.md:187 and 2 more
- Uses dependencies without a pinned version A later version of a dependency could change what runs.SKILL.md and 2 more
A scan lists patterns in the code. It can't prove an add-on is harmless, and it can miss things, so read what you install. How we scan
Alternatives
Other makers' add-ons that do the same job, next to this one.
| Name | Score | Stars | Context | Updated |
|---|---|---|---|---|
| This oneevaluating-llms-harness OpenRaiser/NanoResearch · Skill | 70 | ★ 1.3k | ≈83 always | 26d ago |
| harness revfactory/harness · Skill Both about harness | 83 | ★ 9.2k | ≈80 always | 1mo ago |
| harness-eval HKUDS/OpenHarness · Skill Both about harness and eval | 80 | ★ 16k | ≈74 always | 5mo ago |
| harness-creator walkinglabs/learn-harness-engineering · Skill Both about harness | 86 | ★ 21k | ≈27 always | 1mo ago |
| integrate-harness a5c-ai/babysitter · Skill Both about harness | 66 | ★ 1.9k | ≈54 always | 2mo ago |
| devtu-benchmark-harness mims-harvard/ToolUniverse · Skill Both about harness and benchmark | 72 | ★ 1.7k | ≈60 always | 16d ago |