Files
github-spec-kit/src/specify_cli/integrations/base.py
T
Copilot a00e679918 Add workflow engine with catalog system (#2158)
* Initial plan

* Add workflow engine with step registry, expression engine, catalog system, and CLI commands

Agent-Logs-Url: https://github.com/github/spec-kit/sessions/72a7bb5d-071f-4d67-a507-7e1abae2384d

Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com>

* Add comprehensive tests for workflow engine (94 tests)

Agent-Logs-Url: https://github.com/github/spec-kit/sessions/72a7bb5d-071f-4d67-a507-7e1abae2384d

Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com>

* Address review feedback: do-while condition preservation and URL scheme validation

Agent-Logs-Url: https://github.com/github/spec-kit/sessions/72a7bb5d-071f-4d67-a507-7e1abae2384d

Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com>

* Address review feedback, add CLI dispatch, interactive gates, and docs

Review comments (7/7):
- Add explanatory comment to empty except block
- Implement workflow catalog download with cleanup on failure
- Add input type coercion for number/boolean/enum
- Fix example workflow to remove non-existent output references
- Fix while_loop and if_then condition defaults (string 'false' → bool False)
- Fix resume step index tracking with step_offset parameter

CLI dispatch:
- Add build_exec_args() and dispatch_command() to IntegrationBase
- Override for Claude (skills: /speckit-specify), Gemini (-m flag),
  Codex (codex exec), Copilot (--agent speckit.specify)
- CommandStep invokes installed commands by name via integration CLI
- Add PromptStep for arbitrary inline prompts (10th step type)
- Stream CLI output live to terminal (no silent blocking)
- Remove timeout when streaming (user can Ctrl+C)
- Ctrl+C saves state as PAUSED for clean resume

Interactive gates:
- Gate steps prompt [1] approve [2] reject in TTY
- Fall back to PAUSED in non-interactive environments
- Resume re-executes the gate for interactive prompting

Documentation:
- workflows/README.md — user guide
- workflows/ARCHITECTURE.md — internals with Mermaid diagrams
- workflows/PUBLISHING.md — catalog submission guide

Tests: 94 → 122 workflow tests, 1362 total (all passing)

* Fix ruff lint errors: unused imports, f-string placeholders, undefined name

* Address second review: registry-backed validation, shell failures, loop/fan-out execution, URL validation

- VALID_STEP_TYPES now queries STEP_REGISTRY dynamically
- Shell step returns FAILED on non-zero exit code
- Persist workflow YAML in run directory for reliable resume
- Resume loads from run copy, falls back to installed workflow
- Engine iterates while/do-while loops up to max_iterations
- Engine expands fan-out per item with context.item
- HTTPS URL validation for catalog workflow installs (HTTP allowed for localhost)
- Fix catalog merge priority docstring (lower number wins)
- Fix dispatch_command docstring (no build_exec_args_for_command)
- Gate on_reject=retry pauses for re-prompt on resume
- Update docs to 10 step types, add prompt step to tables and README

* Potential fix for pull request finding 'Empty except'

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>

* Address third review: fan-out IDs, catalog guards, shell coercion, docs

- Fan-out generates unique per-item step IDs and collects results
- Catalog merge skips non-dict workflow entries (malformed data guard)
- Shell step coerces run_cmd to str after expression evaluation
- urlopen timeout=30 for catalog workflow installs
- yaml.dump with sort_keys=False, allow_unicode=True for catalog configs
- Document streaming timeout as intentionally unbounded (user Ctrl+C)
- Document --allow-all-tools as required for non-interactive + future enhancement
- Update test docstring and PUBLISHING.md to 10 step types with prompt

* Validate final URL after redirects in catalog fetch

urlopen follows redirects, so validate the response URL against the
same HTTPS/localhost rules to prevent redirect-based downgrade attacks.

* Address fourth review: filter arg eval, tags normalization, install redirect check

- Filter arguments now evaluated via _evaluate_simple_expression() so
  default(42) returns int not string
- Tags normalized: non-list/non-string values handled gracefully
- Install URL redirect validation (same as catalog fetch)
- Remove unused 'skipped' variable in catalog config parsing
- Author 'github' → 'GitHub' in example workflow
- Document nested step resume limitation (re-runs parent step)

* Add explanatory comment to empty except ValueError block

* Address fifth review: expression parsing, fan-out output, URL install, gate options

- Move string literal parsing before operator detection in expressions
  so quoted strings with operators (e.g. 'a in b') are not mis-parsed
- Fan-out: remove max_concurrency from persisted output, fix docstring
  to reflect sequential execution
- workflow add: support URL sources with HTTPS/redirect validation,
  validate workflow ID is non-empty before writing files
- Deduplicate local install logic via _validate_and_install_local()
- Remove 'edit' gate option from speckit workflow (not implemented)

* Add comments to empty except ValueError blocks in URL install

* Address sixth review: operator precedence, fan_in cleanup, registry resilience, docs

- Fix or/and operator precedence (or parsed first = lower precedence)
- Restore context.fan_in after fan-in step completes
- Catch JSONDecodeError in registry load for corrupted files
- Replace print() with on_step_start callback (library-safe)
- Gate validation warns when on_reject set but no reject option
- Shell step: document shell=True security tradeoff
- README: sdd-pipeline → speckit, parallel → sequential for fan-out
- ARCHITECTURE.md: parallel → fan-out/fan-in in diagram

* Address seventh review: string literal before pipe, type annotations, validate on install

- Move string literal check above pipe filter parsing so 'a | b' works
- Fix type annotations: input_values list[str] | None, run_id str | None
- Run validate_workflow() before installing from local path/URL
- Remove duplicate string literal check from expression parser

* Address eighth review: fan-out namespaced IDs, early return, catalog validation

- Fan-out per-item step IDs use _fanout_{step_id}_{base}_{idx} namespace
  to avoid collisions with user-defined step IDs
- Early return after fan-out loop when state is paused/failed/aborted
- Catalog installs parse + validate downloaded YAML before registering,
  using definition metadata instead of catalog entry for registry

* Address ninth review: populate catalog, fix indentation, priority, README

- Add speckit workflow entry to catalog.json so it's discoverable
- Fix shell step output dict indentation
- Catalog add_catalog priority derived from max existing + 1
- README Quick Start clarified with install + local file examples

* Address tenth review: max_iterations validation, catalog config guard, version alignment

- Validate max_iterations is int >= 1 in while and do-while steps
- Guard add_catalog against corrupted config (non-dict/non-list)
- Align speckit_version requirement to >=0.6.1 (current package version)
- Fan-out template validation uses separate seen_ids set to avoid
  false duplication errors with user-defined step IDs

* Address eleventh review: command step fails without CLI, ID mismatch warning, state persistence

- Command step returns FAILED when CLI not installed (was silent COMPLETED)
- Catalog install warns on workflow ID vs catalog key mismatch
- Engine persists state.save() before returning on unknown step type
- Update tests to expect FAILED for command steps without CLI
- Integration tests use shell steps for CLI-independent execution

* Address twelfth review: type annotations, version examples, streaming docs, requires

- Fix workflow_search type annotations (str | None)
- PUBLISHING.md: speckit_version >=0.15.0 → >=0.6.1
- Document that exit_code is captured and referenceable by later steps
- Mark requires as declared-but-not-enforced (planned enhancement)
- Note full stdout/stderr capture as planned enhancement

* Enforce catalog key matches workflow ID (fail instead of warn)

* Bundle speckit workflow: auto-install during specify init

- Add workflows/speckit to pyproject.toml force-include for wheel builds
- Add _locate_bundled_workflow() helper (mirrors _locate_bundled_extension)
- Auto-install speckit workflow during specify init (after git extension)
- Update all integration file inventory tests to expect workflow files

* Address fourteenth review: prompt fails without CLI, resolved step data, fan-out normalization

- PromptStep returns FAILED when CLI not installed (was silent COMPLETED)
- Engine step_data prefers resolved values from step output
- Fan-out normalizes output.results=[] for empty item lists
- subprocess.run inherits stdout/stderr (no explicit sys.stdout)
- Registry tests use issubset for extensibility

* Address fifteenth review: fan_in docstring, gate defaults, validation guards, reserved prefix

- FanInStep docstring: aggregate-only, no blocking semantics
- FanInStep: guard output_config as dict, handle None
- Gate validate: use same default options as execute
- Validate inputs is dict and steps is list before iterating
- Reserve _fanout_ prefix in step ID validation
- PUBLISHING.md: remove unenforced checklist items, add _fanout_ note

* Address sixteenth review: docs regex, fan_in try/finally, hyphenated dot-path keys

- PUBLISHING.md: update ID regex docs to match implementation (single-char OK)
- FanInStep: wrap expression evaluation in try/finally for context.fan_in
- Expression dot-path: allow hyphens in keys before list index (e.g. run-tests[0])

* Make speckit workflow integration-agnostic, document Copilot CLI requirement

- Workflow integration selectable via input (default: claude)
- Each command step uses {{ inputs.integration }} instead of hardcoded copilot
- Copilot docstring documents CLI requirement for workflow dispatch
- Added install_url for Copilot CLI docs

* Address seventeenth review: project checks, catalog robustness

- Add .specify/ project check to workflow run/resume/status/search/info
- remove_catalog validates config shape (dict + list) before indexing
- _fetch_single_catalog validates response is a dict
- _get_merged_workflows raises when all catalogs fail to fetch
- add_catalog guards against non-dict catalog entries in config

* Address eighteenth review: condition coercion, gate abort result, while default, cache guard, resume log

- evaluate_condition treats plain 'false'/'true' strings as booleans
- Gate abort returns StepResult(FAILED) instead of raising exception
  so step output is persisted in state for inspection
- while_loop max_iterations optional (default 10), validation aligned
- Catalog cache fallback catches invalid JSON gracefully
- resume() appends workflow_finished log entry like execute()

* Address nineteenth review: allow-all-tools opt-in, empty catalogs, abort dead code, while docstring

- --allow-all-tools controlled by SPECKIT_ALLOW_ALL_TOOLS env var (default: 1)
  Set to 0 to disable automatic tool approval for Copilot CLI
- Empty catalogs list falls back to built-in defaults (not an error)
- Remove unreachable WorkflowAbortError catches from execute/resume
  (gate abort now returns StepResult(FAILED) instead of raising)
- while_loop docstring updated: max_iterations is optional (default 10)

* Address twentieth review: gate abort maps to ABORTED status, do-while max_iterations optional

- Engine detects output.aborted from gate step and sets RunStatus.ABORTED
  (was unreachable — gate abort returned FAILED but status was always FAILED)
- do-while max_iterations now optional (default 10), aligned with while_loop
- do-while docstring and validation updated accordingly

* Coerce default_options to dict, align bundled workflow ID regex with validator

* Gate validates string options, prompt uses resolved integration, loop normalizes max_iterations

* Use parentId:childId convention for nested step IDs

- Fan-out per-item IDs use parentId:templateId:index (e.g. parallel:impl:0)
- Reserve ':' in user step IDs (validation rejects)
- Replaces _fanout_ prefix with cleaner namespacing
- Expressions like {{ steps.parallel:impl:0.output.file }} work naturally

* Validate workflow version is semantic versioning (X.Y.Z)

* Schema version validation, strict semver, load_workflow docstring, preserve max_concurrency

- Validate schema_version is '1.0' (reject unknown future schemas)
- Strict semver regex: ^\d+\.\d+\.\d+$ (rejects 1.0.0beta etc.)
- load_workflow docstring: 'parsed' not 'validated'
- Keep max_concurrency in fan-out output (was dropped)
- do_while docstring: engine re-evaluates step_config condition
- ARCHITECTURE.md: document nested resume limitation

* Path traversal prevention, loop step ID namespacing

- RunState validates run_id is alphanumeric+hyphens (no path separators)
- workflow_add validates catalog source doesn't escape workflows_dir
- Loop iterations namespace nested step IDs as parentId:childId:iteration
  so multiple iterations don't overwrite each other in context/state

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
2026-04-14 10:11:56 -05:00

1215 lines
43 KiB
Python

"""Base classes for AI-assistant integrations.
Provides:
- ``IntegrationOption`` — declares a CLI option an integration accepts.
- ``IntegrationBase`` — abstract base every integration must implement.
- ``MarkdownIntegration`` — concrete base for standard Markdown-format
integrations (the common case — subclass, set three class attrs, done).
- ``TomlIntegration`` — concrete base for TOML-format integrations
(Gemini, Tabnine — subclass, set three class attrs, done).
- ``SkillsIntegration`` — concrete base for integrations that install
commands as agent skills (``speckit-<name>/SKILL.md`` layout).
"""
from __future__ import annotations
import re
import shutil
from abc import ABC
from dataclasses import dataclass
from pathlib import Path
from typing import TYPE_CHECKING, Any
if TYPE_CHECKING:
from .manifest import IntegrationManifest
# ---------------------------------------------------------------------------
# IntegrationOption
# ---------------------------------------------------------------------------
@dataclass(frozen=True)
class IntegrationOption:
"""Declares an option that an integration accepts via ``--integration-options``.
Attributes:
name: The flag name (e.g. ``"--commands-dir"``).
is_flag: ``True`` for boolean flags (``--skills``).
required: ``True`` if the option must be supplied.
default: Default value when not supplied (``None`` → no default).
help: One-line description shown in ``specify integrate info``.
"""
name: str
is_flag: bool = False
required: bool = False
default: Any = None
help: str = ""
# ---------------------------------------------------------------------------
# IntegrationBase — abstract base class
# ---------------------------------------------------------------------------
class IntegrationBase(ABC):
"""Abstract base class every integration must implement.
Subclasses must set the following class-level attributes:
* ``key`` — unique identifier, matches actual CLI tool name
* ``config`` — dict compatible with ``AGENT_CONFIG`` entries
* ``registrar_config`` — dict compatible with ``CommandRegistrar.AGENT_CONFIGS``
And may optionally set:
* ``context_file`` — path (relative to project root) of the agent
context/instructions file (e.g. ``"CLAUDE.md"``)
"""
# -- Must be set by every subclass ------------------------------------
key: str = ""
"""Unique integration key — should match the actual CLI tool name."""
config: dict[str, Any] | None = None
"""Metadata dict matching the ``AGENT_CONFIG`` shape."""
registrar_config: dict[str, Any] | None = None
"""Registration dict matching ``CommandRegistrar.AGENT_CONFIGS`` shape."""
# -- Optional ---------------------------------------------------------
context_file: str | None = None
"""Relative path to the agent context file (e.g. ``CLAUDE.md``)."""
# -- Public API -------------------------------------------------------
@classmethod
def options(cls) -> list[IntegrationOption]:
"""Return options this integration accepts. Default: none."""
return []
def build_exec_args(
self,
prompt: str,
*,
model: str | None = None,
output_json: bool = True,
) -> list[str] | None:
"""Build CLI arguments for non-interactive execution.
Returns a list of command-line tokens that will execute *prompt*
non-interactively using this integration's CLI tool, or ``None``
if the integration does not support CLI dispatch.
Subclasses for CLI-based integrations should override this.
"""
return None
def build_command_invocation(self, command_name: str, args: str = "") -> str:
"""Build the native slash-command invocation for a Spec Kit command.
The CLI tools discover and execute commands from installed files
on disk. This method builds the invocation string the CLI
expects — e.g. ``"/speckit.specify my-feature"`` for markdown
agents or ``"/speckit-specify my-feature"`` for skills agents.
*command_name* may be a full dotted name like
``"speckit.specify"`` or a bare stem like ``"specify"``.
"""
stem = command_name
if "." in stem:
stem = stem.rsplit(".", 1)[-1]
invocation = f"/speckit.{stem}"
if args:
invocation = f"{invocation} {args}"
return invocation
def dispatch_command(
self,
command_name: str,
args: str = "",
*,
project_root: Path | None = None,
model: str | None = None,
timeout: int = 600,
stream: bool = True,
) -> dict[str, Any]:
"""Dispatch a Spec Kit command through this integration's CLI.
By default this builds a slash-command invocation with
``build_command_invocation()`` and passes that prompt to
``build_exec_args()`` to construct the CLI command line.
Integrations with custom dispatch behavior can override
``build_command_invocation()``, ``build_exec_args()``, or
``dispatch_command()`` directly.
When *stream* is ``True`` (the default), stdout and stderr are
piped directly to the terminal so the user sees live output.
When ``False``, output is captured and returned in the dict.
Returns a dict with ``exit_code``, ``stdout``, and ``stderr``.
Raises ``NotImplementedError`` if the integration does not
support CLI dispatch.
"""
import subprocess
prompt = self.build_command_invocation(command_name, args)
# When streaming to the terminal, request text output so the
# user sees readable output instead of raw JSONL events.
exec_args = self.build_exec_args(
prompt, model=model, output_json=not stream
)
if exec_args is None:
msg = (
f"Integration {self.key!r} does not support CLI dispatch. "
f"Override build_exec_args() to enable it."
)
raise NotImplementedError(msg)
cwd = str(project_root) if project_root else None
if stream:
# No timeout when streaming — the user sees live output and
# can Ctrl+C at any time. The timeout parameter is only
# applied in the captured (non-streaming) branch below.
try:
result = subprocess.run(
exec_args,
text=True,
cwd=cwd,
)
except KeyboardInterrupt:
return {
"exit_code": 130,
"stdout": "",
"stderr": "Interrupted by user",
}
return {
"exit_code": result.returncode,
"stdout": "",
"stderr": "",
}
result = subprocess.run(
exec_args,
capture_output=True,
text=True,
cwd=cwd,
timeout=timeout,
)
return {
"exit_code": result.returncode,
"stdout": result.stdout,
"stderr": result.stderr,
}
# -- Primitives — building blocks for setup() -------------------------
def shared_commands_dir(self) -> Path | None:
"""Return path to the shared command templates directory.
Checks ``core_pack/commands/`` (wheel install) first, then
``templates/commands/`` (source checkout). Returns ``None``
if neither exists.
"""
import inspect
pkg_dir = Path(inspect.getfile(IntegrationBase)).resolve().parent.parent
for candidate in [
pkg_dir / "core_pack" / "commands",
pkg_dir.parent.parent / "templates" / "commands",
]:
if candidate.is_dir():
return candidate
return None
def shared_templates_dir(self) -> Path | None:
"""Return path to the shared page templates directory.
Contains ``vscode-settings.json``, ``spec-template.md``, etc.
Checks ``core_pack/templates/`` then ``templates/``.
"""
import inspect
pkg_dir = Path(inspect.getfile(IntegrationBase)).resolve().parent.parent
for candidate in [
pkg_dir / "core_pack" / "templates",
pkg_dir.parent.parent / "templates",
]:
if candidate.is_dir():
return candidate
return None
def list_command_templates(self) -> list[Path]:
"""Return sorted list of command template files from the shared directory."""
cmd_dir = self.shared_commands_dir()
if not cmd_dir or not cmd_dir.is_dir():
return []
return sorted(f for f in cmd_dir.iterdir() if f.is_file() and f.suffix == ".md")
def command_filename(self, template_name: str) -> str:
"""Return the destination filename for a command template.
*template_name* is the stem of the source file (e.g. ``"plan"``).
Default: ``speckit.{template_name}.md``. Subclasses override
to change the extension or naming convention.
"""
return f"speckit.{template_name}.md"
def commands_dest(self, project_root: Path) -> Path:
"""Return the absolute path to the commands output directory.
Derived from ``config["folder"]`` and ``config["commands_subdir"]``.
Raises ``ValueError`` if ``config`` or ``folder`` is missing.
"""
if not self.config:
raise ValueError(
f"{type(self).__name__}.config is not set; integration "
"subclasses must define a non-empty 'config' mapping."
)
folder = self.config.get("folder")
if not folder:
raise ValueError(
f"{type(self).__name__}.config is missing required 'folder' entry."
)
subdir = self.config.get("commands_subdir", "commands")
return project_root / folder / subdir
# -- File operations — granular primitives for setup() ----------------
@staticmethod
def copy_command_to_directory(
src: Path,
dest_dir: Path,
filename: str,
) -> Path:
"""Copy a command template to *dest_dir* with the given *filename*.
Creates *dest_dir* if needed. Returns the absolute path of the
written file. The caller can post-process the file before
recording it in the manifest.
"""
dest_dir.mkdir(parents=True, exist_ok=True)
dst = dest_dir / filename
shutil.copy2(src, dst)
return dst
@staticmethod
def record_file_in_manifest(
file_path: Path,
project_root: Path,
manifest: IntegrationManifest,
) -> None:
"""Hash *file_path* and record it in *manifest*.
*file_path* must be inside *project_root*.
"""
rel = file_path.resolve().relative_to(project_root.resolve())
manifest.record_existing(rel)
@staticmethod
def write_file_and_record(
content: str,
dest: Path,
project_root: Path,
manifest: IntegrationManifest,
) -> Path:
"""Write *content* to *dest*, hash it, and record in *manifest*.
Creates parent directories as needed. Writes bytes directly to
avoid platform newline translation (CRLF on Windows). Any
``\r\n`` sequences in *content* are normalised to ``\n`` before
writing. Returns *dest*.
"""
dest.parent.mkdir(parents=True, exist_ok=True)
normalized = content.replace("\r\n", "\n")
dest.write_bytes(normalized.encode("utf-8"))
rel = dest.resolve().relative_to(project_root.resolve())
manifest.record_existing(rel)
return dest
def integration_scripts_dir(self) -> Path | None:
"""Return path to this integration's bundled ``scripts/`` directory.
Looks for a ``scripts/`` sibling of the module that defines the
concrete subclass (not ``IntegrationBase`` itself).
Returns ``None`` if the directory doesn't exist.
"""
import inspect
cls_file = inspect.getfile(type(self))
scripts = Path(cls_file).resolve().parent / "scripts"
return scripts if scripts.is_dir() else None
def install_scripts(
self,
project_root: Path,
manifest: IntegrationManifest,
) -> list[Path]:
"""Copy integration-specific scripts into the project.
Copies files from this integration's ``scripts/`` directory to
``.specify/integrations/<key>/scripts/`` in the project. Shell
scripts are made executable. All copied files are recorded in
*manifest*.
Returns the list of files created.
"""
scripts_src = self.integration_scripts_dir()
if not scripts_src:
return []
created: list[Path] = []
scripts_dest = project_root / ".specify" / "integrations" / self.key / "scripts"
scripts_dest.mkdir(parents=True, exist_ok=True)
for src_script in sorted(scripts_src.iterdir()):
if not src_script.is_file():
continue
dst_script = scripts_dest / src_script.name
shutil.copy2(src_script, dst_script)
if dst_script.suffix == ".sh":
dst_script.chmod(dst_script.stat().st_mode | 0o111)
self.record_file_in_manifest(dst_script, project_root, manifest)
created.append(dst_script)
return created
@staticmethod
def process_template(
content: str,
agent_name: str,
script_type: str,
arg_placeholder: str = "$ARGUMENTS",
) -> str:
"""Process a raw command template into agent-ready content.
Performs the same transformations as the release script:
1. Extract ``scripts.<script_type>`` value from YAML frontmatter
2. Replace ``{SCRIPT}`` with the extracted script command
3. Extract ``agent_scripts.<script_type>`` and replace ``{AGENT_SCRIPT}``
4. Strip ``scripts:`` and ``agent_scripts:`` sections from frontmatter
5. Replace ``{ARGS}`` and ``$ARGUMENTS`` with *arg_placeholder*
6. Replace ``__AGENT__`` with *agent_name*
7. Rewrite paths: ``scripts/`` → ``.specify/scripts/`` etc.
"""
# 1. Extract script command from frontmatter
script_command = ""
script_pattern = re.compile(
rf"^\s*{re.escape(script_type)}:\s*(.+)$", re.MULTILINE
)
# Find the scripts: block
in_scripts = False
for line in content.splitlines():
if line.strip() == "scripts:":
in_scripts = True
continue
if in_scripts and line and not line[0].isspace():
in_scripts = False
if in_scripts:
m = script_pattern.match(line)
if m:
script_command = m.group(1).strip()
break
# 2. Replace {SCRIPT}
if script_command:
content = content.replace("{SCRIPT}", script_command)
# 3. Extract agent_script command
agent_script_command = ""
in_agent_scripts = False
for line in content.splitlines():
if line.strip() == "agent_scripts:":
in_agent_scripts = True
continue
if in_agent_scripts and line and not line[0].isspace():
in_agent_scripts = False
if in_agent_scripts:
m = script_pattern.match(line)
if m:
agent_script_command = m.group(1).strip()
break
if agent_script_command:
content = content.replace("{AGENT_SCRIPT}", agent_script_command)
# 4. Strip scripts: and agent_scripts: sections from frontmatter
lines = content.splitlines(keepends=True)
output_lines: list[str] = []
in_frontmatter = False
skip_section = False
dash_count = 0
for line in lines:
stripped = line.rstrip("\n\r")
if stripped == "---":
dash_count += 1
if dash_count == 1:
in_frontmatter = True
else:
in_frontmatter = False
skip_section = False
output_lines.append(line)
continue
if in_frontmatter:
if stripped in ("scripts:", "agent_scripts:"):
skip_section = True
continue
if skip_section:
if line[0:1].isspace():
continue # skip indented content under scripts/agent_scripts
skip_section = False
output_lines.append(line)
content = "".join(output_lines)
# 5. Replace {ARGS} and $ARGUMENTS
content = content.replace("{ARGS}", arg_placeholder)
content = content.replace("$ARGUMENTS", arg_placeholder)
# 6. Replace __AGENT__
content = content.replace("__AGENT__", agent_name)
# 7. Rewrite paths — delegate to the shared implementation in
# CommandRegistrar so extension-local paths are preserved and
# boundary rules stay consistent across the codebase.
from specify_cli.agents import CommandRegistrar
content = CommandRegistrar.rewrite_project_relative_paths(content)
return content
def setup(
self,
project_root: Path,
manifest: IntegrationManifest,
parsed_options: dict[str, Any] | None = None,
**opts: Any,
) -> list[Path]:
"""Install integration command files into *project_root*.
Returns the list of files created. Copies raw templates without
processing. Integrations that need placeholder replacement
(e.g. ``{SCRIPT}``, ``__AGENT__``) should override ``setup()``
and call ``process_template()`` in their own loop — see
``CopilotIntegration`` for an example.
"""
templates = self.list_command_templates()
if not templates:
return []
project_root_resolved = project_root.resolve()
if manifest.project_root != project_root_resolved:
raise ValueError(
f"manifest.project_root ({manifest.project_root}) does not match "
f"project_root ({project_root_resolved})"
)
dest = self.commands_dest(project_root).resolve()
try:
dest.relative_to(project_root_resolved)
except ValueError as exc:
raise ValueError(
f"Integration destination {dest} escapes "
f"project root {project_root_resolved}"
) from exc
created: list[Path] = []
for src_file in templates:
dst_name = self.command_filename(src_file.stem)
dst_file = self.copy_command_to_directory(src_file, dest, dst_name)
self.record_file_in_manifest(dst_file, project_root, manifest)
created.append(dst_file)
return created
def teardown(
self,
project_root: Path,
manifest: IntegrationManifest,
*,
force: bool = False,
) -> tuple[list[Path], list[Path]]:
"""Uninstall integration files from *project_root*.
Delegates to ``manifest.uninstall()`` which only removes files
whose hash still matches the recorded value (unless *force*).
Returns ``(removed, skipped)`` file lists.
"""
return manifest.uninstall(project_root, force=force)
# -- Convenience helpers for subclasses -------------------------------
def install(
self,
project_root: Path,
manifest: IntegrationManifest,
parsed_options: dict[str, Any] | None = None,
**opts: Any,
) -> list[Path]:
"""High-level install — calls ``setup()`` and returns created files."""
return self.setup(project_root, manifest, parsed_options=parsed_options, **opts)
def uninstall(
self,
project_root: Path,
manifest: IntegrationManifest,
*,
force: bool = False,
) -> tuple[list[Path], list[Path]]:
"""High-level uninstall — calls ``teardown()``."""
return self.teardown(project_root, manifest, force=force)
# ---------------------------------------------------------------------------
# MarkdownIntegration — covers ~20 standard agents
# ---------------------------------------------------------------------------
class MarkdownIntegration(IntegrationBase):
"""Concrete base for integrations that use standard Markdown commands.
Subclasses only need to set ``key``, ``config``, ``registrar_config``
(and optionally ``context_file``). Everything else is inherited.
``setup()`` processes command templates (replacing ``{SCRIPT}``,
``{ARGS}``, ``__AGENT__``, rewriting paths) and installs
integration-specific scripts (``update-context.sh`` / ``.ps1``).
"""
def build_exec_args(
self,
prompt: str,
*,
model: str | None = None,
output_json: bool = True,
) -> list[str] | None:
if not self.config or not self.config.get("requires_cli"):
return None
args = [self.key, "-p", prompt]
if model:
args.extend(["--model", model])
if output_json:
args.extend(["--output-format", "json"])
return args
def setup(
self,
project_root: Path,
manifest: IntegrationManifest,
parsed_options: dict[str, Any] | None = None,
**opts: Any,
) -> list[Path]:
templates = self.list_command_templates()
if not templates:
return []
project_root_resolved = project_root.resolve()
if manifest.project_root != project_root_resolved:
raise ValueError(
f"manifest.project_root ({manifest.project_root}) does not match "
f"project_root ({project_root_resolved})"
)
dest = self.commands_dest(project_root).resolve()
try:
dest.relative_to(project_root_resolved)
except ValueError as exc:
raise ValueError(
f"Integration destination {dest} escapes "
f"project root {project_root_resolved}"
) from exc
dest.mkdir(parents=True, exist_ok=True)
script_type = opts.get("script_type", "sh")
arg_placeholder = (
self.registrar_config.get("args", "$ARGUMENTS")
if self.registrar_config
else "$ARGUMENTS"
)
created: list[Path] = []
for src_file in templates:
raw = src_file.read_text(encoding="utf-8")
processed = self.process_template(
raw, self.key, script_type, arg_placeholder
)
dst_name = self.command_filename(src_file.stem)
dst_file = self.write_file_and_record(
processed, dest / dst_name, project_root, manifest
)
created.append(dst_file)
created.extend(self.install_scripts(project_root, manifest))
return created
# ---------------------------------------------------------------------------
# TomlIntegration — TOML-format agents (Gemini, Tabnine)
# ---------------------------------------------------------------------------
class TomlIntegration(IntegrationBase):
"""Concrete base for integrations that use TOML command format.
Mirrors ``MarkdownIntegration`` closely: subclasses only need to set
``key``, ``config``, ``registrar_config`` (and optionally
``context_file``). Everything else is inherited.
``setup()`` processes command templates through the same placeholder
pipeline as ``MarkdownIntegration``, then converts the result to
TOML format (``description`` key + ``prompt`` multiline string).
"""
def build_exec_args(
self,
prompt: str,
*,
model: str | None = None,
output_json: bool = True,
) -> list[str] | None:
if not self.config or not self.config.get("requires_cli"):
return None
args = [self.key, "-p", prompt]
if model:
args.extend(["-m", model])
if output_json:
args.extend(["--output-format", "json"])
return args
def command_filename(self, template_name: str) -> str:
"""TOML commands use ``.toml`` extension."""
return f"speckit.{template_name}.toml"
@staticmethod
def _extract_description(content: str) -> str:
"""Extract the ``description`` value from YAML frontmatter.
Parses the YAML frontmatter so block scalar descriptions (``|``
and ``>``) keep their YAML semantics instead of being treated as
raw text.
"""
import yaml
frontmatter_text, _ = TomlIntegration._split_frontmatter(content)
if not frontmatter_text:
return ""
try:
frontmatter = yaml.safe_load(frontmatter_text) or {}
except yaml.YAMLError:
return ""
if not isinstance(frontmatter, dict):
return ""
description = frontmatter.get("description", "")
if isinstance(description, str):
return description
return ""
@staticmethod
def _split_frontmatter(content: str) -> tuple[str, str]:
"""Split YAML frontmatter from the remaining content.
Returns ``("", content)`` when no complete frontmatter block is
present. The body is preserved exactly as written so prompt text
keeps its intended formatting.
"""
if not content.startswith("---"):
return "", content
lines = content.splitlines(keepends=True)
if not lines or lines[0].rstrip("\r\n") != "---":
return "", content
frontmatter_end = -1
for i, line in enumerate(lines[1:], start=1):
if line.rstrip("\r\n") == "---":
frontmatter_end = i
break
if frontmatter_end == -1:
return "", content
frontmatter = "".join(lines[1:frontmatter_end])
body = "".join(lines[frontmatter_end + 1 :])
return frontmatter, body
@staticmethod
def _render_toml_string(value: str) -> str:
"""Render *value* as a TOML string literal.
Uses a basic string for single-line values, multiline basic
strings for values containing newlines, and falls back to a
literal string or escaped basic string when delimiters appear in
the content.
"""
if "\n" not in value and "\r" not in value:
escaped = value.replace("\\", "\\\\").replace('"', '\\"')
return f'"{escaped}"'
escaped = value.replace("\\", "\\\\")
if '"""' not in escaped:
if escaped.endswith('"'):
return '"""\n' + escaped + '\\\n"""'
return '"""\n' + escaped + '"""'
if "'''" not in value and not value.endswith("'"):
return "'''\n" + value + "'''"
return (
'"'
+ (
value.replace("\\", "\\\\")
.replace('"', '\\"')
.replace("\n", "\\n")
.replace("\r", "\\r")
.replace("\t", "\\t")
)
+ '"'
)
@staticmethod
def _render_toml(description: str, body: str) -> str:
"""Render a TOML command file from description and body.
Uses multiline basic strings (``\"\"\"``) with backslashes
escaped, matching the output of the release script. Falls back
to multiline literal strings (``'''``) if the body contains
``\"\"\"``, then to an escaped basic string as a last resort.
The body is ``rstrip("\\n")``'d before rendering, so the TOML
value preserves content without forcing a trailing newline. As a
result, multiline delimiters appear on their own line only when
the rendered value itself ends with a newline.
"""
toml_lines: list[str] = []
if description:
toml_lines.append(
f"description = {TomlIntegration._render_toml_string(description)}"
)
toml_lines.append("")
body = body.rstrip("\n")
toml_lines.append(f"prompt = {TomlIntegration._render_toml_string(body)}")
return "\n".join(toml_lines) + "\n"
def setup(
self,
project_root: Path,
manifest: IntegrationManifest,
parsed_options: dict[str, Any] | None = None,
**opts: Any,
) -> list[Path]:
templates = self.list_command_templates()
if not templates:
return []
project_root_resolved = project_root.resolve()
if manifest.project_root != project_root_resolved:
raise ValueError(
f"manifest.project_root ({manifest.project_root}) does not match "
f"project_root ({project_root_resolved})"
)
dest = self.commands_dest(project_root).resolve()
try:
dest.relative_to(project_root_resolved)
except ValueError as exc:
raise ValueError(
f"Integration destination {dest} escapes "
f"project root {project_root_resolved}"
) from exc
dest.mkdir(parents=True, exist_ok=True)
script_type = opts.get("script_type", "sh")
arg_placeholder = (
self.registrar_config.get("args", "{{args}}")
if self.registrar_config
else "{{args}}"
)
created: list[Path] = []
for src_file in templates:
raw = src_file.read_text(encoding="utf-8")
description = self._extract_description(raw)
processed = self.process_template(
raw, self.key, script_type, arg_placeholder
)
_, body = self._split_frontmatter(processed)
toml_content = self._render_toml(description, body)
dst_name = self.command_filename(src_file.stem)
dst_file = self.write_file_and_record(
toml_content, dest / dst_name, project_root, manifest
)
created.append(dst_file)
created.extend(self.install_scripts(project_root, manifest))
return created
# ---------------------------------------------------------------------------
# YamlIntegration — YAML-format agents (Goose)
# ---------------------------------------------------------------------------
class YamlIntegration(IntegrationBase):
"""Concrete base for integrations that use YAML recipe format.
Mirrors ``TomlIntegration`` closely: subclasses only need to set
``key``, ``config``, ``registrar_config`` (and optionally
``context_file``). Everything else is inherited.
``setup()`` processes command templates through the same placeholder
pipeline as ``MarkdownIntegration``, then converts the result to
YAML recipe format (version, title, description, prompt block scalar).
"""
def command_filename(self, template_name: str) -> str:
"""YAML commands use ``.yaml`` extension."""
return f"speckit.{template_name}.yaml"
@staticmethod
def _extract_frontmatter(content: str) -> dict[str, Any]:
"""Extract frontmatter as a dict from YAML frontmatter block."""
import yaml
if not content.startswith("---"):
return {}
lines = content.splitlines(keepends=True)
if not lines or lines[0].rstrip("\r\n") != "---":
return {}
frontmatter_end = -1
for i, line in enumerate(lines[1:], start=1):
if line.rstrip("\r\n") == "---":
frontmatter_end = i
break
if frontmatter_end == -1:
return {}
frontmatter_text = "".join(lines[1:frontmatter_end])
try:
fm = yaml.safe_load(frontmatter_text) or {}
except yaml.YAMLError:
return {}
return fm if isinstance(fm, dict) else {}
@staticmethod
def _split_frontmatter(content: str) -> tuple[str, str]:
"""Split YAML frontmatter from the remaining body content."""
if not content.startswith("---"):
return "", content
lines = content.splitlines(keepends=True)
if not lines or lines[0].rstrip("\r\n") != "---":
return "", content
frontmatter_end = -1
for i, line in enumerate(lines[1:], start=1):
if line.rstrip("\r\n") == "---":
frontmatter_end = i
break
if frontmatter_end == -1:
return "", content
frontmatter = "".join(lines[1:frontmatter_end])
body = "".join(lines[frontmatter_end + 1 :])
return frontmatter, body
@staticmethod
def _human_title(identifier: str) -> str:
"""Convert an identifier to a human-readable title.
Strips a leading ``speckit.`` prefix and replaces ``.``, ``-``,
and ``_`` with spaces before title-casing.
"""
text = identifier
if text.startswith("speckit."):
text = text[len("speckit.") :]
return text.replace(".", " ").replace("-", " ").replace("_", " ").title()
@staticmethod
def _render_yaml(title: str, description: str, body: str, source_id: str) -> str:
"""Render a YAML recipe file from title, description, and body.
Produces a Goose-compatible recipe with a literal block scalar
for the prompt content. Uses ``yaml.safe_dump()`` for the
header fields to ensure proper escaping.
"""
import yaml
header = {
"version": "1.0.0",
"title": title,
"description": description,
"author": {"contact": "spec-kit"},
"extensions": [{"type": "builtin", "name": "developer"}],
"activities": ["Spec-Driven Development"],
}
header_yaml = yaml.safe_dump(
header,
sort_keys=False,
allow_unicode=True,
default_flow_style=False,
).strip()
# Indent each line for YAML block scalar
indented = "\n".join(f" {line}" for line in body.split("\n"))
lines = [header_yaml, "prompt: |", indented, "", f"# Source: {source_id}"]
return "\n".join(lines) + "\n"
def setup(
self,
project_root: Path,
manifest: IntegrationManifest,
parsed_options: dict[str, Any] | None = None,
**opts: Any,
) -> list[Path]:
templates = self.list_command_templates()
if not templates:
return []
project_root_resolved = project_root.resolve()
if manifest.project_root != project_root_resolved:
raise ValueError(
f"manifest.project_root ({manifest.project_root}) does not match "
f"project_root ({project_root_resolved})"
)
dest = self.commands_dest(project_root).resolve()
try:
dest.relative_to(project_root_resolved)
except ValueError as exc:
raise ValueError(
f"Integration destination {dest} escapes "
f"project root {project_root_resolved}"
) from exc
dest.mkdir(parents=True, exist_ok=True)
script_type = opts.get("script_type", "sh")
arg_placeholder = (
self.registrar_config.get("args", "{{args}}")
if self.registrar_config
else "{{args}}"
)
created: list[Path] = []
for src_file in templates:
raw = src_file.read_text(encoding="utf-8")
fm = self._extract_frontmatter(raw)
description = fm.get("description", "")
if not isinstance(description, str):
description = str(description) if description is not None else ""
title = fm.get("title", "") or fm.get("name", "")
if not isinstance(title, str):
title = str(title) if title is not None else ""
if not title:
title = self._human_title(src_file.stem)
processed = self.process_template(
raw, self.key, script_type, arg_placeholder
)
_, body = self._split_frontmatter(processed)
yaml_content = self._render_yaml(
title, description, body, f"templates/commands/{src_file.name}"
)
dst_name = self.command_filename(src_file.stem)
dst_file = self.write_file_and_record(
yaml_content, dest / dst_name, project_root, manifest
)
created.append(dst_file)
created.extend(self.install_scripts(project_root, manifest))
return created
# ---------------------------------------------------------------------------
# SkillsIntegration — skills-format agents (Codex, Kimi, Agy)
# ---------------------------------------------------------------------------
class SkillsIntegration(IntegrationBase):
"""Concrete base for integrations that install commands as agent skills.
Skills use the ``speckit-<name>/SKILL.md`` directory layout following
the `agentskills.io <https://agentskills.io/specification>`_ spec.
Subclasses set ``key``, ``config``, ``registrar_config`` (and
optionally ``context_file``) like any integration. They may also
override ``options()`` to declare additional CLI flags (e.g.
``--skills``, ``--migrate-legacy``).
``setup()`` processes each shared command template into a
``speckit-<name>/SKILL.md`` file with skills-oriented frontmatter.
"""
def build_exec_args(
self,
prompt: str,
*,
model: str | None = None,
output_json: bool = True,
) -> list[str] | None:
if not self.config or not self.config.get("requires_cli"):
return None
args = [self.key, "-p", prompt]
if model:
args.extend(["--model", model])
if output_json:
args.extend(["--output-format", "json"])
return args
def skills_dest(self, project_root: Path) -> Path:
"""Return the absolute path to the skills output directory.
Derived from ``config["folder"]`` and the configured
``commands_subdir`` (defaults to ``"skills"``).
Raises ``ValueError`` when ``config`` or ``folder`` is missing.
"""
if not self.config:
raise ValueError(f"{type(self).__name__}.config is not set.")
folder = self.config.get("folder")
if not folder:
raise ValueError(
f"{type(self).__name__}.config is missing required 'folder' entry."
)
subdir = self.config.get("commands_subdir", "skills")
return project_root / folder / subdir
def build_command_invocation(self, command_name: str, args: str = "") -> str:
"""Skills use ``/speckit-<stem>`` (hyphenated directory name)."""
stem = command_name
if "." in stem:
stem = stem.rsplit(".", 1)[-1]
invocation = f"/speckit-{stem}"
if args:
invocation = f"{invocation} {args}"
return invocation
def setup(
self,
project_root: Path,
manifest: IntegrationManifest,
parsed_options: dict[str, Any] | None = None,
**opts: Any,
) -> list[Path]:
"""Install command templates as agent skills.
Creates ``speckit-<name>/SKILL.md`` for each shared command
template. Each SKILL.md has normalised frontmatter containing
``name``, ``description``, ``compatibility``, and ``metadata``.
"""
import yaml
templates = self.list_command_templates()
if not templates:
return []
project_root_resolved = project_root.resolve()
if manifest.project_root != project_root_resolved:
raise ValueError(
f"manifest.project_root ({manifest.project_root}) does not match "
f"project_root ({project_root_resolved})"
)
skills_dir = self.skills_dest(project_root).resolve()
try:
skills_dir.relative_to(project_root_resolved)
except ValueError as exc:
raise ValueError(
f"Skills destination {skills_dir} escapes "
f"project root {project_root_resolved}"
) from exc
script_type = opts.get("script_type", "sh")
arg_placeholder = (
self.registrar_config.get("args", "$ARGUMENTS")
if self.registrar_config
else "$ARGUMENTS"
)
created: list[Path] = []
for src_file in templates:
raw = src_file.read_text(encoding="utf-8")
# Derive the skill name from the template stem
command_name = src_file.stem # e.g. "plan"
skill_name = f"speckit-{command_name.replace('.', '-')}"
# Parse frontmatter for description
frontmatter: dict[str, Any] = {}
if raw.startswith("---"):
parts = raw.split("---", 2)
if len(parts) >= 3:
try:
fm = yaml.safe_load(parts[1])
if isinstance(fm, dict):
frontmatter = fm
except yaml.YAMLError:
pass
# Process body through the standard template pipeline
processed_body = self.process_template(
raw, self.key, script_type, arg_placeholder
)
# Strip the processed frontmatter — we rebuild it for skills.
# Preserve leading whitespace in the body to match release ZIP
# output byte-for-byte (the template body starts with \n after
# the closing ---).
if processed_body.startswith("---"):
parts = processed_body.split("---", 2)
if len(parts) >= 3:
processed_body = parts[2]
# Select description — use the original template description
# to stay byte-for-byte identical with release ZIP output.
description = frontmatter.get("description", "")
if not description:
description = f"Spec Kit: {command_name} workflow"
# Build SKILL.md with manually formatted frontmatter to match
# the release packaging script output exactly (double-quoted
# values, no yaml.safe_dump quoting differences).
def _quote(v: str) -> str:
escaped = v.replace("\\", "\\\\").replace('"', '\\"')
return f'"{escaped}"'
skill_content = (
f"---\n"
f"name: {_quote(skill_name)}\n"
f"description: {_quote(description)}\n"
f"compatibility: {_quote('Requires spec-kit project structure with .specify/ directory')}\n"
f"metadata:\n"
f" author: {_quote('github-spec-kit')}\n"
f" source: {_quote('templates/commands/' + src_file.name)}\n"
f"---\n"
f"{processed_body}"
)
# Write speckit-<name>/SKILL.md
skill_dir = skills_dir / skill_name
skill_file = skill_dir / "SKILL.md"
dst = self.write_file_and_record(
skill_content, skill_file, project_root, manifest
)
created.append(dst)
created.extend(self.install_scripts(project_root, manifest))
return created