iris
iris-eval/mcp-serverPlugin
The agent eval standard for MCP. Score every agent output for quality, safety, and cost. 12 built-in eval rules cover completeness, relevance, safety (PII detection, prompt injection), and cost thresholds. Log traces with hierarchical spans, evaluate outputs inline, and track costs across all your agents. No SDK, no code changes — add one line to your MCP config. Open source, MIT licensed.
Why this score
Popularity score: 56/100 from public GitHub signals only. It shows how widely used and maintained this is, not how well it works. Not included yet: the fake-star check. The security scan is shown separately and doesn't change the score. How we rank
Security
Scanned Nov 4, 2026 at commit b83394823c50, first 300 source files (it's a large repo), with Cisco's open-source skill-scanner.
What its code does:
- Reads environment variables and sends network requests with them API clients do this to send your key to the service. Check which service it talks to.scripts/post-launch-snapshot.sh:56, scripts/track10-v040-verify.sh:29
- Runs shell commands Common for dev tools. Read the commands it runs.scripts/check-product-claims.sh:28, scripts/claims/capture-tests.mjs:16, scripts/claims/generate.mjs:64 and 14 more
- Downloads content and runs it Whatever it downloads can change after we scanned it.scripts/post-launch-snapshot.sh:19, scripts/track10-v040-verify.sh:28
Web addresses in its code:
- iris-eval.comdashboard/src/components/command/commands.ts:190
- ghcr.ioscripts/post-launch-snapshot.sh:55
- api.npmjs.orgscripts/post-launch-snapshot.sh:19
- dev.toscripts/devto-crosspost.mjs:83
- api.anthropic.comsrc/eval/llm-judge/client.ts:105
- api.openai.comsrc/eval/llm-judge/client.ts:175
- doi.orgsrc/eval/citation-verify/resolve.ts:288
- fonts.googleapis.comsrc/dashboard/server.ts:60
- fonts.gstatic.comsrc/dashboard/server.ts:62
- mcp.soscripts/post-launch-snapshot.sh:51
- mcpservers.orgscripts/post-launch-snapshot.sh:52
- pulsemcp.comscripts/post-launch-snapshot.sh:50
- registry.modelcontextprotocol.ioscripts/post-launch-snapshot.sh:34
- token.actions.githubusercontent.comscripts/track9-rc-verify.sh:50
These are the sites named in the code, so it's where your data could go. It isn't a complete list: code can build addresses while it runs, and a few of these may be links rather than calls. Whatever you send also goes to the AI model you use.
A scan lists patterns in the code. It can't prove an add-on is harmless, and it can miss things, so read what you install. How we scan
Alternatives
Other makers' add-ons that do the same job, next to this one.
| Name | Score | Stars | Context | Updated |
|---|---|---|---|---|
| This oneiris iris-eval/mcp-server · Plugin | 56 | ★ 9 | not measured | yesterday |
| iris-development redis/agent-skills · Skill Both about iris and sdk | 57 | ★ 167 | ≈107 always | 2mo ago |