mirror of
https://github.com/wshobson/agents
synced 2026-06-21 14:13:58 +00:00
dependabot[bot]
07413134b0
Bumps the python-minor-and-patch group in /plugins/plugin-eval with 5 updates: | Package | From | To | | --- | --- | --- | | [pydantic](https://github.com/pydantic/pydantic) | `2.12.5` | `2.13.4` | | [typer](https://github.com/fastapi/typer) | `0.24.1` | `0.25.1` | | [claude-agent-sdk](https://github.com/anthropics/claude-agent-sdk-python) | `0.1.50` | `0.2.85` | | [anthropic](https://github.com/anthropics/anthropic-sdk-python) | `0.87.0` | `0.104.1` | | [ruff](https://github.com/astral-sh/ruff) | `0.15.7` | `0.15.14` | Updates `pydantic` from 2.12.5 to 2.13.4 - [Release notes](https://github.com/pydantic/pydantic/releases) - [Changelog](https://github.com/pydantic/pydantic/blob/main/HISTORY.md) - [Commits](https://github.com/pydantic/pydantic/compare/v2.12.5...v2.13.4) Updates `typer` from 0.24.1 to 0.25.1 - [Release notes](https://github.com/fastapi/typer/releases) - [Changelog](https://github.com/fastapi/typer/blob/master/docs/release-notes.md) - [Commits](https://github.com/fastapi/typer/compare/0.24.1...0.25.1) Updates `claude-agent-sdk` from 0.1.50 to 0.2.85 - [Release notes](https://github.com/anthropics/claude-agent-sdk-python/releases) - [Changelog](https://github.com/anthropics/claude-agent-sdk-python/blob/main/CHANGELOG.md) - [Commits](https://github.com/anthropics/claude-agent-sdk-python/compare/v0.1.50...v0.2.85) Updates `anthropic` from 0.87.0 to 0.104.1 - [Release notes](https://github.com/anthropics/anthropic-sdk-python/releases) - [Changelog](https://github.com/anthropics/anthropic-sdk-python/blob/main/CHANGELOG.md) - [Commits](https://github.com/anthropics/anthropic-sdk-python/compare/v0.87.0...v0.104.1) Updates `ruff` from 0.15.7 to 0.15.14 - [Release notes](https://github.com/astral-sh/ruff/releases) - [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md) - [Commits](https://github.com/astral-sh/ruff/compare/0.15.7...0.15.14) --- updated-dependencies: - dependency-name: pydantic dependency-version: 2.13.4 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: python-minor-and-patch - dependency-name: typer dependency-version: 0.25.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: python-minor-and-patch - dependency-name: claude-agent-sdk dependency-version: 0.2.85 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: python-minor-and-patch - dependency-name: anthropic dependency-version: 0.104.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: python-minor-and-patch - dependency-name: ruff dependency-version: 0.15.14 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: python-minor-and-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
plugin-eval
Three-layer quality evaluation framework for Claude Code plugins.
Quick Start
cd plugins/plugin-eval
uv sync
# Evaluate a skill (static only, instant)
uv run plugin-eval score path/to/skill --depth quick
# Evaluate with LLM judge (~30s)
uv run plugin-eval score path/to/skill --depth standard
# Full certification (all layers, ~5 min)
uv run plugin-eval certify path/to/skill
Layers
- Static Analysis — Structural checks, anti-pattern detection. Instant, free.
- LLM Judge — Semantic evaluation (triggering, orchestration, output, scope). ~30s, 4 calls.
- Monte Carlo — Statistical reliability via 50–100 simulated runs. ~2–5 min.
Commands
| CLI | Claude Code | Description |
|---|---|---|
plugin-eval score |
/eval |
Score a plugin or skill |
plugin-eval certify |
/certify |
Full certification with badge |
plugin-eval compare |
/compare |
Head-to-head comparison |
plugin-eval init |
— | Build corpus for Elo ranking |
Documentation
See docs/plugin-eval.md for the full reference: layers, dimensions, scoring formula, anti-patterns, statistical methods, and project structure.