Seth Hobson 38cb914673 feat(plugin-eval): implement Layer 3 Monte Carlo simulation with statistical analysis ...

Adds MonteCarloAnalyzer with SimResult/MonteCarloConfig dataclasses, run_simulation
async helper, and _compute_statistics using Wilson score CI, bootstrap CI,
Clopper-Pearson CI, and coefficient of variation. Wires MC layer into evaluate_skill
for Depth.DEEP and Depth.THOROUGH runs (50 and 100 runs respectively).

2026-03-25 17:52:15 -04:00

..

__init__.py

feat(plugin-eval): scaffold project with uv, ruff, ty

2026-03-25 17:31:00 -04:00

conftest.py

feat(plugin-eval): scaffold project with uv, ruff, ty

2026-03-25 17:31:00 -04:00

test_cli.py

feat(plugin-eval): add reporter (JSON/MD/HTML) and Typer CLI with score/certify/compare/init commands

2026-03-25 17:45:34 -04:00

test_corpus.py

feat(plugin-eval): implement Elo ranking system and corpus management

2026-03-25 17:48:18 -04:00

test_elo.py

feat(plugin-eval): implement Elo ranking system and corpus management

2026-03-25 17:48:18 -04:00

test_engine.py

feat(plugin-eval): implement eval engine with composite scoring and layer blending

2026-03-25 17:42:11 -04:00

test_judge.py

feat(plugin-eval): implement Layer 2 LLM judge with Agent SDK and model tiering

2026-03-25 17:48:53 -04:00

test_models.py

feat(plugin-eval): add Pydantic data models for all eval layers

2026-03-25 17:33:32 -04:00

test_monte_carlo.py

feat(plugin-eval): implement Layer 3 Monte Carlo simulation with statistical analysis

2026-03-25 17:52:15 -04:00

test_parser.py

feat(plugin-eval): add markdown/frontmatter parser for skills, agents, plugins

2026-03-25 17:36:23 -04:00

test_reporter.py

feat(plugin-eval): add reporter (JSON/MD/HTML) and Typer CLI with score/certify/compare/init commands

2026-03-25 17:45:34 -04:00

test_static.py

feat(plugin-eval): implement Layer 1 static analysis with anti-pattern detection

2026-03-25 17:39:58 -04:00

test_stats.py

feat(plugin-eval): add statistics module (bootstrap, Wilson, Clopper-Pearson, kappa)

2026-03-25 17:34:17 -04:00