mirror of
https://github.com/trailofbits/skills
synced 2026-06-21 14:12:00 +00:00
fix(fp-check): scope completeness hooks to skill/agent frontmatter (#143)
The plugin-level hooks/hooks.json registered Stop and SubagentStop
hooks with matcher "*". Because plugin hooks are always-on, both fired
on every (sub)agent stop in any session where fp-check was merely
installed — burning a 30s LLM turn and injecting stop-feedback even
when fp-check was never used, then no-opping via the prompt's own
fallback.
Root-cause fix: move the hooks into skill/agent frontmatter, which per
the Claude Code docs is "scoped to the component's lifecycle and only
run when that component is active":
- Overall verification-completeness Stop hook -> fp-check SKILL.md
frontmatter. Fires only when the fp-check skill is loaded.
- Per-subagent output checks -> each agent's own frontmatter as a Stop
hook, which auto-converts to SubagentStop. data-flow-analyzer checks
Phase 1, exploitability-verifier Phase 2, poc-builder Phase 4. Each
fires only when that specific agent runs, so the single SubagentStop
prompt's "identify which agent type" branching is no longer needed.
- Delete plugins/fp-check/hooks/hooks.json.
This fixes both hooks (not just Stop) and keeps full coverage including
inline fp-check runs, unlike deleting the Stop hook outright. Hook
prompts and the {"ok"/"reason"} response schema are carried over
unchanged from the current plugin hooks.
Bump 1.0.2 -> 1.0.3.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -375,7 +375,7 @@
|
||||
},
|
||||
{
|
||||
"name": "fp-check",
|
||||
"version": "1.0.2",
|
||||
"version": "1.0.3",
|
||||
"description": "Systematic false positive verification for security bug analysis with mandatory gate reviews",
|
||||
"author": {
|
||||
"name": "Maciej Domanski"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "fp-check",
|
||||
"version": "1.0.2",
|
||||
"version": "1.0.3",
|
||||
"description": "Systematic false positive verification for security bug analysis with mandatory gate reviews",
|
||||
"author": {
|
||||
"name": "Maciej Domanski"
|
||||
|
||||
@@ -7,6 +7,24 @@ tools:
|
||||
- Read
|
||||
- Grep
|
||||
- Glob
|
||||
hooks:
|
||||
Stop:
|
||||
- matcher: "*"
|
||||
hooks:
|
||||
- type: prompt
|
||||
timeout: 30
|
||||
prompt: |
|
||||
You are an output completeness checker for the fp-check data-flow-analyzer subagent, which is about to stop. Verify it produced complete structured output for its assigned Phase 1 work.
|
||||
|
||||
Required sections:
|
||||
- Phase 1.1: Trust boundary map with source, path, sink, and validation points (each with file:line references)
|
||||
- Phase 1.2: API contract analysis (built-in protections identified or absence noted)
|
||||
- Phase 1.3: Environment protection analysis (each classified as 'prevents entirely' or 'raises bar')
|
||||
- Phase 1.4: Cross-reference analysis (similar patterns, test coverage, history)
|
||||
- Phase 1 Conclusion with evidence
|
||||
|
||||
If required sections are missing, respond with JSON: {"ok": false, "reason": "<specific gaps>"}
|
||||
If all required sections are present with evidence, respond with JSON: {"ok": true}
|
||||
---
|
||||
|
||||
# Data Flow Analyzer
|
||||
|
||||
@@ -7,6 +7,24 @@ tools:
|
||||
- Read
|
||||
- Grep
|
||||
- Glob
|
||||
hooks:
|
||||
Stop:
|
||||
- matcher: "*"
|
||||
hooks:
|
||||
- type: prompt
|
||||
timeout: 30
|
||||
prompt: |
|
||||
You are an output completeness checker for the fp-check exploitability-verifier subagent, which is about to stop. Verify it produced complete structured output for its assigned Phase 2 work.
|
||||
|
||||
Required sections:
|
||||
- Phase 2.1: Attacker control analysis with control level (full/partial/none) and evidence
|
||||
- Phase 2.2: Mathematical bounds proof with step-by-step algebra, OR explicit 'N/A — not a bounds issue'
|
||||
- Phase 2.3: Race condition feasibility analysis, OR explicit 'N/A — not a concurrency issue'
|
||||
- Phase 2.4: Adversarial analysis synthesizing 2.1-2.3
|
||||
- Phase 2 Conclusion with evidence
|
||||
|
||||
If required sections are missing, respond with JSON: {"ok": false, "reason": "<specific gaps>"}
|
||||
If all required sections are present with evidence, respond with JSON: {"ok": true}
|
||||
---
|
||||
|
||||
# Exploitability Verifier
|
||||
|
||||
@@ -10,6 +10,25 @@ tools:
|
||||
- Grep
|
||||
- Glob
|
||||
- Bash
|
||||
hooks:
|
||||
Stop:
|
||||
- matcher: "*"
|
||||
hooks:
|
||||
- type: prompt
|
||||
timeout: 30
|
||||
prompt: |
|
||||
You are an output completeness checker for the fp-check poc-builder subagent, which is about to stop. Verify it produced complete structured output for its assigned Phase 4 work.
|
||||
|
||||
Required sections:
|
||||
- Phase 4.1: Pseudocode PoC with data flow diagram (always required)
|
||||
- Phase 4.2: Executable PoC or explicit skip with justification
|
||||
- Phase 4.3: Unit test PoC or explicit skip with justification
|
||||
- Phase 4.4: Negative PoC showing exploit preconditions
|
||||
- Phase 4.5: Verification that PoCs demonstrate the vulnerability
|
||||
- Phase 4 Conclusion
|
||||
|
||||
If required sections are missing, respond with JSON: {"ok": false, "reason": "<specific gaps>"}
|
||||
If all required sections are present with evidence, respond with JSON: {"ok": true}
|
||||
---
|
||||
|
||||
# PoC Builder
|
||||
|
||||
@@ -1,29 +0,0 @@
|
||||
{
|
||||
"description": "Enforce fp-check verification completeness — prevent premature verdicts and incomplete agent output",
|
||||
"hooks": {
|
||||
"Stop": [
|
||||
{
|
||||
"matcher": "*",
|
||||
"hooks": [
|
||||
{
|
||||
"type": "prompt",
|
||||
"prompt": "You are a verification completeness checker for the fp-check false positive analysis skill. The agent is about to stop. Check whether the verification process was completed properly.\n\nScan the conversation for evidence of ALL of the following for EVERY bug that was being verified:\n\n1. Phase 1 (Data Flow Analysis): Trust boundaries mapped, API contracts checked, environment protections analyzed, cross-references checked\n2. Phase 2 (Exploitability Verification): Attacker control confirmed or denied, mathematical bounds analyzed (or marked N/A), race conditions analyzed (or marked N/A), adversarial analysis completed\n3. Phase 3 (Impact Assessment): Real security impact vs operational robustness distinguished, primary controls vs defense-in-depth distinguished\n4. Phase 4 (PoC Creation): Pseudocode PoC created, executable/unit test PoCs created or explicitly skipped with justification, negative PoC created, PoC verification completed\n5. Phase 5 (Devil's Advocate): All 13 challenge questions addressed\n6. Gate Review: All 6 gates (Process, Reachability, Real Impact, PoC Validation, Math Bounds, Environment) evaluated with pass/fail\n7. Verdict: Each bug has a TRUE POSITIVE or FALSE POSITIVE verdict with evidence\n\nIf ANY bug is missing ANY of these phases or the gate review, respond with JSON: {\"ok\": false, \"reason\": \"<specific gaps>\"}\nIf all bugs have complete verification with verdicts, respond with JSON: {\"ok\": true}\nIf the conversation is not about fp-check verification at all, respond with JSON: {\"ok\": true}",
|
||||
"timeout": 30
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"SubagentStop": [
|
||||
{
|
||||
"matcher": "*",
|
||||
"hooks": [
|
||||
{
|
||||
"type": "prompt",
|
||||
"prompt": "You are an output completeness checker for fp-check verification agents. A subagent is about to stop. Check whether it produced complete structured output for its assigned phases.\n\nIdentify which agent type this is from the conversation context and verify completeness:\n\n**data-flow-analyzer** must include:\n- Phase 1.1: Trust boundary map with source, path, sink, and validation points (each with file:line references)\n- Phase 1.2: API contract analysis (built-in protections identified or absence noted)\n- Phase 1.3: Environment protection analysis (each classified as 'prevents entirely' or 'raises bar')\n- Phase 1.4: Cross-reference analysis (similar patterns, test coverage, history)\n- Phase 1 Conclusion with evidence\n\n**exploitability-verifier** must include:\n- Phase 2.1: Attacker control analysis with control level (full/partial/none) and evidence\n- Phase 2.2: Mathematical bounds proof with step-by-step algebra, OR explicit 'N/A — not a bounds issue'\n- Phase 2.3: Race condition feasibility analysis, OR explicit 'N/A — not a concurrency issue'\n- Phase 2.4: Adversarial analysis synthesizing 2.1-2.3\n- Phase 2 Conclusion with evidence\n\n**poc-builder** must include:\n- Phase 4.1: Pseudocode PoC with data flow diagram (always required)\n- Phase 4.2: Executable PoC or explicit skip with justification\n- Phase 4.3: Unit test PoC or explicit skip with justification\n- Phase 4.4: Negative PoC showing exploit preconditions\n- Phase 4.5: Verification that PoCs demonstrate the vulnerability\n- Phase 4 Conclusion\n\nIf the agent is not an fp-check agent, respond with JSON: {\"ok\": true}\nIf required sections are missing, respond with JSON: {\"ok\": false, \"reason\": \"<specific gaps>\"}\nIf all required sections are present with evidence, respond with JSON: {\"ok\": true}",
|
||||
"timeout": 30
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -2,6 +2,28 @@
|
||||
name: fp-check
|
||||
description: "Systematically verifies suspected security bugs to eliminate false positives. Produces TRUE POSITIVE or FALSE POSITIVE verdicts with documented evidence for each bug."
|
||||
allowed-tools: Read Grep Glob LSP Bash Task Write Edit AskUserQuestion TaskCreate TaskUpdate TaskList TaskGet
|
||||
hooks:
|
||||
Stop:
|
||||
- matcher: "*"
|
||||
hooks:
|
||||
- type: prompt
|
||||
timeout: 30
|
||||
prompt: |
|
||||
You are a verification completeness checker for the fp-check false positive analysis skill. The agent is about to stop. Check whether the verification process was completed properly.
|
||||
|
||||
Scan the conversation for evidence of ALL of the following for EVERY bug that was being verified:
|
||||
|
||||
1. Phase 1 (Data Flow Analysis): Trust boundaries mapped, API contracts checked, environment protections analyzed, cross-references checked
|
||||
2. Phase 2 (Exploitability Verification): Attacker control confirmed or denied, mathematical bounds analyzed (or marked N/A), race conditions analyzed (or marked N/A), adversarial analysis completed
|
||||
3. Phase 3 (Impact Assessment): Real security impact vs operational robustness distinguished, primary controls vs defense-in-depth distinguished
|
||||
4. Phase 4 (PoC Creation): Pseudocode PoC created, executable/unit test PoCs created or explicitly skipped with justification, negative PoC created, PoC verification completed
|
||||
5. Phase 5 (Devil's Advocate): All 13 challenge questions addressed
|
||||
6. Gate Review: All 6 gates (Process, Reachability, Real Impact, PoC Validation, Math Bounds, Environment) evaluated with pass/fail
|
||||
7. Verdict: Each bug has a TRUE POSITIVE or FALSE POSITIVE verdict with evidence
|
||||
|
||||
If ANY bug is missing ANY of these phases or the gate review, respond with JSON: {"ok": false, "reason": "<specific gaps>"}
|
||||
If all bugs have complete verification with verdicts, respond with JSON: {"ok": true}
|
||||
If the conversation is not about fp-check verification at all, respond with JSON: {"ok": true}
|
||||
---
|
||||
|
||||
# False Positive Check
|
||||
|
||||
Reference in New Issue
Block a user