diff --git a/skills/memory-ci-capture/SKILL.md b/skills/memory-ci-capture/SKILL.md new file mode 100644 index 00000000..e2db7adb --- /dev/null +++ b/skills/memory-ci-capture/SKILL.md @@ -0,0 +1,56 @@ +--- +name: memory-ci-capture +description: Synthesize GitHub delivery context into a concise Basic Memory project update. Use in CI after `bm ci collect` prepares a ProjectUpdateContext; return only structured AgentSynthesis JSON for `bm ci publish`. +--- + +# Memory CI Capture + +Turn a meaningful GitHub delivery moment into project memory. GitHub records the +mechanics. Basic Memory remembers what changed and why. + +## Inputs + +Read the `ProjectUpdateContext` JSON produced by `bm ci collect` at +`.github/basic-memory/project-update-context.json`. Treat it as the immutable +source of truth for repository, event type, PR number, workflow run, SHA, source +URL, timestamps, and deployment environment. + +Do not invent tests, deploy checks, linked issues, product impact, or decisions. +If evidence is absent, say so briefly in `verification` or leave the field empty. + +## Output + +Return only JSON matching the `AgentSynthesis` shape: + +```json +{ + "summary": "What changed.", + "why_it_matters": "Why this update matters for future humans and agents.", + "user_facing_changes": [], + "internal_changes": [], + "verification": [], + "follow_ups": [], + "decision_candidates": [], + "task_candidates": [] +} +``` + +## Synthesis Rules + +- Prefer a short explanation over a commit-by-commit changelog. +- Preserve intent, changed behavior, source links, verification evidence present + in the context, and concrete follow-ups. +- Put explicit product or architecture decisions in `decision_candidates` only + when the source context clearly contains them. +- Put future work in `task_candidates` only when it is concrete enough to act on. +- Keep the tone factual and useful. This is project memory, not marketing copy. + +## Event Guidance + +For merged pull requests, focus on why the PR existed, what area changed, what +issues it advanced or closed, and what verification evidence appears in the +context. + +For production deploys, focus on what reached production, the deployed SHA, +environment, workflow run, and verification evidence. Do not overclaim success +beyond the workflow and source facts. diff --git a/src/basic_memory/ci/README.md b/src/basic_memory/ci/README.md new file mode 100644 index 00000000..0f447bbb --- /dev/null +++ b/src/basic_memory/ci/README.md @@ -0,0 +1,181 @@ +# Basic Memory CI + +Basic Memory CI turns meaningful GitHub delivery moments into durable +`project_update` notes in Basic Memory. + +GitHub records the mechanics: pull requests, workflow runs, SHAs, URLs, labels, +and timestamps. The agent reads those facts and writes a short synthesis of what +changed and why it matters. The Basic Memory CLI owns authentication, schema +guidance, idempotency, and publishing. + +The product voice is: + +> GitHub records the mechanics. Basic Memory remembers what changed and why. + +## Flow + +```text +GitHub event +-> workflow eligibility filter +-> bm ci collect +-> Codex Action reads ProjectUpdateContext + memory-ci-capture prompt +-> Codex writes AgentSynthesis JSON +-> bm ci publish +-> Basic Memory project_update note +``` + +The v0 workflow is GitHub-only and uses `openai/codex-action@v1` as the +synthesis runner. The Codex step runs in read-only mode and does not receive the +Basic Memory API key. + +## Setup CI/CD + +Use `bm ci setup` from the GitHub repository root. The command installs the +workflow/config/prompt files and seeds the Basic Memory schema notes. + +For the common cloud path: + +```bash +bm cloud api-key save bmc_... +bm ci setup --project --workspace --cloud --yes +``` + +Prefer `--project-id ` when the same project name exists in more +than one workspace: + +```bash +bm ci setup --project --project-id --cloud --yes +``` + +Then review and commit the generated files: + +```text +.github/workflows/basic-memory.yml +.github/basic-memory/config.yml +.github/basic-memory/memory-ci-capture.md +``` + +Add these GitHub repository secrets: + +- `OPENAI_API_KEY`: used only by `openai/codex-action`. +- `BASIC_MEMORY_API_KEY`: mapped to `BASIC_MEMORY_CLOUD_API_KEY` only for + `bm ci publish`. + +Add this optional GitHub repository variable only when using a non-default cloud +host: + +- `BASIC_MEMORY_CLOUD_HOST` + +Configure production deploy capture in `.github/basic-memory/config.yml`: + +```yaml +project: +workspace: +deploy_workflows: + - Deploy Production +production_environments: + - production +``` + +The generated workflow also uses `deploy_workflows` to populate +`on.workflow_run.workflows`. If you change deploy workflow names later, rerun +`bm ci setup --force` or update both `.github/basic-memory/config.yml` and +`.github/workflows/basic-memory.yml` together. + +The generated workflow installs `basic-memory` from PyPI. When dogfooding an +unreleased `bm ci` change, temporarily edit the workflow install step to install +the branch or commit that contains the CI commands. + +After the files and secrets are in place, verify the first run by merging a test +PR or completing a configured production deploy workflow. The Auto BM workflow +should create or update a `project_update` note under: + +```text +project-updates/github/// +``` + +Failures in the Auto BM workflow fail only the project-update capture workflow; +they do not roll back the original merge or deploy. + +## Commands + +`bm ci setup` + +Installs the repository automation files: + +- `.github/workflows/basic-memory.yml` +- `.github/basic-memory/config.yml` +- `.github/basic-memory/memory-ci-capture.md` + +It also seeds the canonical Basic Memory schema notes when they do not already +exist: + +- `ProjectUpdate` +- `GitHubPullRequestUpdate` +- `GitHubProductionDeployUpdate` + +`bm ci collect` + +Reads the current GitHub event payload and normalizes it into +`ProjectUpdateContext`. This command decides whether the event is eligible. +Merged pull requests and configured successful production deploy workflow runs +are eligible. Routine CI runs, failed deploys, and unmerged PR closures are +no-ops. In v0, collection is intentionally limited to the GitHub event payload; +GitHub API enrichment for file lists, checks, reviews, or commit lists can be +added later without changing the publishing boundary. + +`bm ci agent-schema` + +Writes the optional `AgentSynthesis` JSON schema used by the generated workflow +as a CI guardrail. This schema is not a Basic Memory domain schema and is not +committed by setup. + +`bm ci publish` + +Combines deterministic GitHub facts with the agent synthesis and upserts a +Basic Memory `project_update` note. Agent-supplied identity fields are ignored; +source identity comes from `ProjectUpdateContext`. + +## Auth Boundary + +The generated workflow needs these secrets: + +- `OPENAI_API_KEY` +- `BASIC_MEMORY_API_KEY` + +`OPENAI_API_KEY` is passed only to the Codex Action. `BASIC_MEMORY_API_KEY` is +mapped to `BASIC_MEMORY_CLOUD_API_KEY` only for the publish step, where the +Basic Memory client uses it as `cloud_api_key`. The publish step also passes +`--cloud` so CI writes to the configured cloud project. + +When `.github/basic-memory/config.yml` includes `workspace` and no `project_id`, +CI routes the project as `/`. Use a workspace slug there, or +prefer `project_id` when project names collide across workspaces. + +## Idempotency + +Project updates are upserted by stable GitHub identity: + +- PR merge: + `github:/:pull_request_merged:` +- Production deploy: + `github:/:production_deploy_succeeded::` + +The default note folder is: + +```text +project-updates/github/// +``` + +## Module Responsibilities + +`project_updates.py` contains the v0 domain model and rendering helpers: + +- `ProjectUpdateConfig`: non-secret repo configuration. +- `ProjectUpdateContext`: normalized immutable GitHub facts. +- `AgentSynthesis`: agent-authored summary fields. +- `ProjectUpdateNote`: final Basic Memory note payload. +- workflow, prompt, and schema-note seed rendering. + +The CLI command group lives in `basic_memory.cli.commands.ci` and performs file +installation, event collection, schema seeding, and publish orchestration. diff --git a/src/basic_memory/ci/__init__.py b/src/basic_memory/ci/__init__.py new file mode 100644 index 00000000..532ce2af --- /dev/null +++ b/src/basic_memory/ci/__init__.py @@ -0,0 +1 @@ +"""CI helpers for Basic Memory project update capture.""" diff --git a/src/basic_memory/ci/project_updates.py b/src/basic_memory/ci/project_updates.py new file mode 100644 index 00000000..8b0e2220 --- /dev/null +++ b/src/basic_memory/ci/project_updates.py @@ -0,0 +1,645 @@ +"""Project update capture helpers for GitHub Actions. + +Auto BM treats GitHub as the immutable source layer, the agent as the synthesis +layer, and Basic Memory as the durable project-memory layer. +""" + +from __future__ import annotations + +import json +import re +import subprocess +from pathlib import Path +from typing import Any, Literal + +import yaml +from pydantic import BaseModel, ConfigDict, Field, field_validator + + +PULL_REQUEST_MERGED = "pull_request_merged" +PRODUCTION_DEPLOY_SUCCEEDED = "production_deploy_succeeded" +DEFAULT_NOTE_FOLDER_TEMPLATE = "project-updates/github/{owner}/{repo}" +DEFAULT_CONFIG_PATH = ".github/basic-memory/config.yml" +DEFAULT_WORKFLOW_PATH = ".github/workflows/basic-memory.yml" +DEFAULT_PROMPT_PATH = ".github/basic-memory/memory-ci-capture.md" +DEFAULT_CONTEXT_PATH = ".github/basic-memory/project-update-context.json" + + +class ProjectUpdateConfig(BaseModel): + """Non-secret Auto BM repository configuration.""" + + project: str | None = None + project_id: str | None = None + workspace: str | None = None + deploy_workflows: list[str] = Field(default_factory=lambda: ["Deploy Production"]) + production_environments: list[str] = Field(default_factory=lambda: ["production"]) + note_folder: str = DEFAULT_NOTE_FOLDER_TEMPLATE + codex_model: str | None = None + codex_effort: str | None = None + + @field_validator("deploy_workflows", "production_environments") + @classmethod + def _non_empty_list(cls, value: list[str]) -> list[str]: + cleaned = [item.strip() for item in value if item.strip()] + if not cleaned: + raise ValueError("must contain at least one non-empty value") + return cleaned + + +class ProjectUpdateContext(BaseModel): + """Normalized facts collected from a GitHub event payload.""" + + eligible: bool + source: Literal["github"] = "github" + source_event: str | None = None + skip_reason: str | None = None + repo: str | None = None + repo_url: str | None = None + source_url: str | None = None + occurred_at: str | None = None + idempotency_key: str | None = None + sha: str | None = None + pr_number: int | None = None + workflow_run_id: str | None = None + environment: str | None = None + title: str | None = None + body: str | None = None + author: str | None = None + labels: list[str] = Field(default_factory=list) + linked_issues: list[str] = Field(default_factory=list) + changed_files_count: int | None = None + + +class AgentSynthesis(BaseModel): + """Agent-authored synthesis for a project update.""" + + model_config = ConfigDict(extra="ignore") + + summary: str + why_it_matters: str + user_facing_changes: list[str] = Field(default_factory=list) + internal_changes: list[str] = Field(default_factory=list) + verification: list[str] = Field(default_factory=list) + follow_ups: list[str] = Field(default_factory=list) + decision_candidates: list[str] = Field(default_factory=list) + task_candidates: list[str] = Field(default_factory=list) + + @field_validator( + "summary", + "why_it_matters", + ) + @classmethod + def _required_text(cls, value: str) -> str: + stripped = value.strip() + if not stripped: + raise ValueError("must not be empty") + return stripped + + +class ProjectUpdateNote(BaseModel): + """Final note payload for the Basic Memory writer.""" + + title: str + directory: str + content: str + metadata: dict[str, Any] + tags: list[str] + + +class SchemaSeedSpec(BaseModel): + """Basic Memory schema note seed.""" + + title: str + entity: str + content: str + metadata: dict[str, Any] + + +def parse_github_remote(remote_url: str) -> tuple[str, str]: + """Parse an HTTPS or SSH GitHub remote into owner/repo.""" + patterns = ( + r"^https://github\.com/(?P[^/]+)/(?P[^/]+?)(?:\.git)?/?$", + r"^git@github\.com:(?P[^/]+)/(?P[^/]+?)(?:\.git)?$", + r"^ssh://git@github\.com/(?P[^/]+)/(?P[^/]+?)(?:\.git)?$", + ) + for pattern in patterns: + match = re.match(pattern, remote_url.strip()) + if match: + return match.group("owner"), match.group("repo") + raise ValueError(f"Expected a GitHub remote, got: {remote_url}") + + +def detect_github_repo(cwd: Path) -> tuple[str, str]: + """Detect the GitHub origin for a repository checkout.""" + result = subprocess.run( + ["git", "config", "--get", "remote.origin.url"], + cwd=cwd, + check=False, + capture_output=True, + text=True, + ) + if result.returncode != 0 or not result.stdout.strip(): + raise ValueError("No remote.origin.url found; run this from a GitHub repository checkout") + return parse_github_remote(result.stdout.strip()) + + +def load_project_update_config(path: Path) -> ProjectUpdateConfig: + """Load Auto BM repository config from YAML.""" + if not path.exists(): + return ProjectUpdateConfig() + raw = yaml.safe_load(path.read_text(encoding="utf-8")) or {} + if not isinstance(raw, dict): + raise ValueError(f"{path} must contain a YAML object") + return ProjectUpdateConfig.model_validate(raw) + + +def write_project_update_config(path: Path, config: ProjectUpdateConfig) -> None: + """Write repository config YAML.""" + path.parent.mkdir(parents=True, exist_ok=True) + data = config.model_dump(exclude_none=True) + path.write_text(yaml.safe_dump(data, sort_keys=False), encoding="utf-8") + + +def _load_event_payload(event_path: Path) -> dict[str, Any]: + try: + payload = json.loads(event_path.read_text(encoding="utf-8")) + except FileNotFoundError as exc: + raise ValueError(f"GitHub event payload not found: {event_path}") from exc + except json.JSONDecodeError as exc: + raise ValueError(f"GitHub event payload is not valid JSON: {event_path}") from exc + if not isinstance(payload, dict): + raise ValueError("GitHub event payload must be a JSON object") + return payload + + +def collect_project_update_context( + *, + event_name: str, + event_path: Path, + config: ProjectUpdateConfig, +) -> ProjectUpdateContext: + """Normalize a GitHub Actions event into a project update context.""" + payload = _load_event_payload(event_path) + if event_name == "pull_request": + return _collect_pull_request_context(payload) + if event_name == "workflow_run": + return _collect_workflow_run_context(payload, config) + return ProjectUpdateContext( + eligible=False, + skip_reason=f"unsupported GitHub event: {event_name}", + ) + + +def _repo_fields(payload: dict[str, Any]) -> tuple[str | None, str | None]: + repo = payload.get("repository") + if not isinstance(repo, dict): + return None, None + full_name = repo.get("full_name") + html_url = repo.get("html_url") + return ( + full_name if isinstance(full_name, str) else None, + html_url if isinstance(html_url, str) else None, + ) + + +def _label_names(labels: Any) -> list[str]: + if not isinstance(labels, list): + return [] + names: list[str] = [] + for label in labels: + if isinstance(label, dict) and isinstance(label.get("name"), str): + names.append(label["name"]) + return names + + +def _linked_issues(*texts: str | None) -> list[str]: + seen: set[str] = set() + issues: list[str] = [] + for text in texts: + if not text: + continue + for match in re.finditer(r"(?\d+)\b", text): + issue = f"#{match.group('number')}" + if issue not in seen: + seen.add(issue) + issues.append(issue) + return issues + + +def _collect_pull_request_context(payload: dict[str, Any]) -> ProjectUpdateContext: + pr = payload.get("pull_request") + if not isinstance(pr, dict): + return ProjectUpdateContext(eligible=False, skip_reason="pull request payload missing") + + if payload.get("action") != "closed": + return ProjectUpdateContext( + eligible=False, skip_reason="pull request action was not closed" + ) + + if pr.get("merged") is not True: + return ProjectUpdateContext( + eligible=False, + skip_reason="pull request was closed without merging", + ) + + repo, repo_url = _repo_fields(payload) + number = pr.get("number") + title = pr.get("title") if isinstance(pr.get("title"), str) else None + body = pr.get("body") if isinstance(pr.get("body"), str) else None + source_url = pr.get("html_url") if isinstance(pr.get("html_url"), str) else None + sha = pr.get("merge_commit_sha") if isinstance(pr.get("merge_commit_sha"), str) else None + occurred_at = pr.get("merged_at") if isinstance(pr.get("merged_at"), str) else None + author = None + user = pr.get("user") + if isinstance(user, dict) and isinstance(user.get("login"), str): + author = user["login"] + + idempotency_key = None + if repo and isinstance(number, int): + idempotency_key = f"github:{repo}:{PULL_REQUEST_MERGED}:{number}" + + return ProjectUpdateContext( + eligible=True, + source_event=PULL_REQUEST_MERGED, + repo=repo, + repo_url=repo_url, + source_url=source_url, + occurred_at=occurred_at, + idempotency_key=idempotency_key, + sha=sha, + pr_number=number if isinstance(number, int) else None, + title=title, + body=body, + author=author, + labels=_label_names(pr.get("labels")), + linked_issues=_linked_issues(title, body), + changed_files_count=( + pr["changed_files"] if isinstance(pr.get("changed_files"), int) else None + ), + ) + + +def _collect_workflow_run_context( + payload: dict[str, Any], + config: ProjectUpdateConfig, +) -> ProjectUpdateContext: + run = payload.get("workflow_run") + if not isinstance(run, dict): + return ProjectUpdateContext(eligible=False, skip_reason="workflow run payload missing") + + conclusion = run.get("conclusion") + if conclusion != "success": + return ProjectUpdateContext( + eligible=False, + skip_reason=f"workflow conclusion was {conclusion or 'missing'}", + ) + + workflow_name = run.get("name") + if workflow_name not in config.deploy_workflows: + return ProjectUpdateContext( + eligible=False, + skip_reason=f"workflow '{workflow_name}' is not configured for project updates", + ) + + repo, repo_url = _repo_fields(payload) + run_id = run.get("id") + workflow_run_id = str(run_id) if run_id is not None else None + environment = config.production_environments[0] + source_url = run.get("html_url") if isinstance(run.get("html_url"), str) else None + sha = run.get("head_sha") if isinstance(run.get("head_sha"), str) else None + occurred_at = run.get("updated_at") if isinstance(run.get("updated_at"), str) else None + + idempotency_key = None + if repo and workflow_run_id: + idempotency_key = ( + f"github:{repo}:{PRODUCTION_DEPLOY_SUCCEEDED}:{environment}:{workflow_run_id}" + ) + + return ProjectUpdateContext( + eligible=True, + source_event=PRODUCTION_DEPLOY_SUCCEEDED, + repo=repo, + repo_url=repo_url, + source_url=source_url, + occurred_at=occurred_at, + idempotency_key=idempotency_key, + sha=sha, + workflow_run_id=workflow_run_id, + environment=environment, + title=str(workflow_name) if workflow_name is not None else None, + ) + + +def _owner_repo(repo: str) -> tuple[str, str]: + if "/" not in repo: + raise ValueError(f"Repository must be owner/repo, got: {repo}") + owner, repo_name = repo.split("/", 1) + return owner, repo_name + + +def _note_directory(context: ProjectUpdateContext, config: ProjectUpdateConfig | None) -> str: + if not context.repo: + raise ValueError("Project update context is missing repo") + owner, repo_name = _owner_repo(context.repo) + template = config.note_folder if config else DEFAULT_NOTE_FOLDER_TEMPLATE + return template.format(owner=owner, repo=repo_name, full_repo=context.repo) + + +def _note_title(context: ProjectUpdateContext) -> str: + if context.source_event == PULL_REQUEST_MERGED and context.pr_number: + title = context.title or "Merged pull request" + return f"PR #{context.pr_number}: {title}" + if context.source_event == PRODUCTION_DEPLOY_SUCCEEDED: + environment = context.environment or "production" + when = (context.occurred_at or "").split("T", 1)[0] or "unknown date" + return f"{environment.title()} deploy: {when}" + return "Project update" + + +def build_project_update_note( + *, + context: ProjectUpdateContext, + synthesis: AgentSynthesis, + config: ProjectUpdateConfig | None = None, +) -> ProjectUpdateNote: + """Build the final Basic Memory project update note.""" + if not context.eligible: + raise ValueError(f"Cannot build a note for an ineligible context: {context.skip_reason}") + if not context.source_event or not context.repo or not context.idempotency_key: + raise ValueError("Project update context is missing deterministic identity fields") + + metadata: dict[str, Any] = { + "source": "github", + "source_event": context.source_event, + "repo": context.repo, + "source_url": context.source_url, + "occurred_at": context.occurred_at, + "idempotency_key": context.idempotency_key, + "sha": context.sha, + "pr_number": context.pr_number, + "workflow_run_id": context.workflow_run_id, + "environment": context.environment, + } + metadata = {key: value for key, value in metadata.items() if value is not None} + + sections = [ + f"# {_note_title(context)}", + "## Summary", + synthesis.summary, + "## Why It Matters", + synthesis.why_it_matters, + ] + + _extend_list_section(sections, "User-Facing Changes", synthesis.user_facing_changes) + _extend_list_section(sections, "Internal Changes", synthesis.internal_changes) + _extend_list_section(sections, "Verification", synthesis.verification) + _extend_list_section(sections, "Follow-Ups", synthesis.follow_ups) + _extend_list_section(sections, "Decision Candidates", synthesis.decision_candidates) + _extend_list_section(sections, "Task Candidates", synthesis.task_candidates) + + source_links = [] + if context.source_url: + source_links.append(f"- Source: {context.source_url}") + if context.repo_url: + source_links.append(f"- Repository: {context.repo_url}") + if context.linked_issues: + source_links.append(f"- Linked issues: {', '.join(context.linked_issues)}") + if source_links: + sections.extend(["## Source Links", *source_links]) + + observations = [ + f"- [summary] {synthesis.summary}", + f"- [source] GitHub {context.source_event} in {context.repo}", + ] + sections.extend(["## Observations", *observations]) + + return ProjectUpdateNote( + title=_note_title(context), + directory=_note_directory(context, config), + content="\n\n".join(sections).strip() + "\n", + metadata=metadata, + tags=["github", "project-update", context.source_event], + ) + + +def _extend_list_section(sections: list[str], title: str, values: list[str]) -> None: + cleaned = [value.strip() for value in values if value.strip()] + if cleaned: + sections.extend([f"## {title}", *[f"- {value}" for value in cleaned]]) + + +def render_agent_synthesis_schema() -> str: + """Render the optional Codex structured-output schema guardrail.""" + schema = { + "$schema": "https://json-schema.org/draft/2020-12/schema", + "title": "AgentSynthesis", + "type": "object", + "additionalProperties": False, + "required": ["summary", "why_it_matters"], + "properties": { + "summary": {"type": "string", "minLength": 1}, + "why_it_matters": {"type": "string", "minLength": 1}, + "user_facing_changes": {"type": "array", "items": {"type": "string"}}, + "internal_changes": {"type": "array", "items": {"type": "string"}}, + "verification": {"type": "array", "items": {"type": "string"}}, + "follow_ups": {"type": "array", "items": {"type": "string"}}, + "decision_candidates": {"type": "array", "items": {"type": "string"}}, + "task_candidates": {"type": "array", "items": {"type": "string"}}, + }, + } + return json.dumps(schema, indent=2, sort_keys=True) + "\n" + + +def render_capture_prompt() -> str: + """Render the prompt contract used by the generated workflow.""" + return """# Memory CI Capture + +You turn GitHub delivery context into a concise project update synthesis for +Basic Memory. GitHub records the mechanics. Basic Memory remembers what changed +and why. + +## Inputs + +- Read `.github/basic-memory/project-update-context.json`. +- Treat GitHub payload fields as immutable facts. +- Do not invent tests, deployment status, issues, or user impact. + +## Output + +Return only JSON that matches the provided AgentSynthesis schema: + +- `summary`: what changed. +- `why_it_matters`: why this project update matters for future humans and agents. +- `user_facing_changes`: visible behavior or product changes. +- `internal_changes`: implementation, infrastructure, or operational changes. +- `verification`: checks, tests, deploy evidence, or explicit unknowns. +- `follow_ups`: concrete remaining work only. +- `decision_candidates`: explicit product or architecture decisions only. +- `task_candidates`: concrete future tasks only. + +Prefer source links and grounded phrasing. This is project memory, not marketing +copy and not a commit-by-commit changelog. +""" + + +def render_workflow(config: ProjectUpdateConfig) -> str: + """Render the generated GitHub Actions workflow.""" + workflow_names = json.dumps(config.deploy_workflows) + model_line = f" model: {config.codex_model}\n" if config.codex_model else "" + effort_line = f" effort: {config.codex_effort}\n" if config.codex_effort else "" + return f"""name: Basic Memory Project Updates + +on: + pull_request: + types: [closed] + workflow_run: + workflows: {workflow_names} + types: [completed] + +jobs: + project-update: + runs-on: ubuntu-latest + permissions: + contents: read + pull-requests: read + issues: read + actions: read + steps: + - name: Checkout repository + uses: actions/checkout@v6 + + - name: Set up Python + uses: actions/setup-python@v6 + with: + python-version: "3.12" + + - name: Install Basic Memory + run: | + python -m pip install --upgrade pip + pip install basic-memory + + - name: Collect project update context + id: collect + run: | + bm ci collect \\ + --config {DEFAULT_CONFIG_PATH} \\ + --output {DEFAULT_CONTEXT_PATH} + + - name: Stop when event is not eligible + if: steps.collect.outputs.eligible != 'true' + run: echo "Auto BM skipped: ${{{{ steps.collect.outputs.skip_reason }}}}" + + - name: Write Codex output schema + if: steps.collect.outputs.eligible == 'true' + run: | + bm ci agent-schema --output "${{{{ runner.temp }}}}/agent-synthesis.schema.json" + + - name: Synthesize project update with Codex + if: steps.collect.outputs.eligible == 'true' + uses: openai/codex-action@v1 + with: + openai-api-key: ${{{{ secrets.OPENAI_API_KEY }}}} + prompt-file: {DEFAULT_PROMPT_PATH} + output-file: ${{{{ runner.temp }}}}/agent-synthesis.json + output-schema-file: ${{{{ runner.temp }}}}/agent-synthesis.schema.json + sandbox: read-only + safety-strategy: drop-sudo +{model_line}{effort_line} + - name: Publish project update + if: steps.collect.outputs.eligible == 'true' + env: + BASIC_MEMORY_CLOUD_API_KEY: ${{{{ secrets.BASIC_MEMORY_API_KEY }}}} + BASIC_MEMORY_CI_CLOUD_HOST: ${{{{ vars.BASIC_MEMORY_CLOUD_HOST }}}} + run: | + if [ -n "$BASIC_MEMORY_CI_CLOUD_HOST" ]; then + export BASIC_MEMORY_CLOUD_HOST="$BASIC_MEMORY_CI_CLOUD_HOST" + fi + bm ci publish \\ + --cloud \\ + --config {DEFAULT_CONFIG_PATH} \\ + --context {DEFAULT_CONTEXT_PATH} \\ + --synthesis "${{{{ runner.temp }}}}/agent-synthesis.json" +""" + + +def schema_seed_specs() -> list[SchemaSeedSpec]: + """Return Basic Memory schema note seeds for Auto BM project updates.""" + return [ + _schema_seed( + title="ProjectUpdate", + entity="ProjectUpdate", + schema={ + "summary": "string, concise account of what changed", + "why_it_matters": "string, why this update matters", + "source": "string, source system such as github", + "source_event": ("string, pull_request_merged or production_deploy_succeeded"), + "repo": "string, owner/repository", + "source_url": "string, canonical source URL", + "occurred_at?": "string, ISO timestamp", + "idempotency_key": "string, stable source identity", + "sha?": "string, commit SHA", + "pr_number?": "integer, pull request number", + "workflow_run_id?": "string, GitHub Actions workflow run id", + "environment?": "string, deployment environment", + }, + body=( + "A ProjectUpdate preserves what changed in a project and why it matters. " + "GitHub records mechanics; Basic Memory keeps the durable narrative." + ), + ), + _schema_seed( + title="GitHubPullRequestUpdate", + entity="GitHubPullRequestUpdate", + schema={ + "intent": "string, purpose of the merged pull request", + "changed_area?(array)": "string, product or implementation areas touched", + "linked_issue?(array)": "string, issues closed or advanced", + "verification?(array)": "string, checks and tests observed", + "follow_up?(array)": "string, concrete remaining work", + }, + body=( + "Guidance for pull request project updates: preserve intent, changed " + "behavior, review tradeoffs, issue links, and verification. Do not " + "summarize commit by commit unless that is the clearest explanation." + ), + ), + _schema_seed( + title="GitHubProductionDeployUpdate", + entity="GitHubProductionDeployUpdate", + schema={ + "deployed_sha": "string, deployed commit SHA", + "environment": "string, production environment", + "workflow_run_id": "string, GitHub Actions workflow run id", + "verification?(array)": "string, deploy evidence and smoke checks", + "user_impact?(array)": "string, user-facing impact since previous deploy", + "rollback_note?": "string, rollback or mitigation note when known", + }, + body=( + "Guidance for production deploy project updates: preserve what actually " + "reached production, the deployed SHA, environment, workflow run, and " + "verification evidence. Do not overclaim beyond the source facts." + ), + ), + ] + + +def _schema_seed( + *, + title: str, + entity: str, + schema: dict[str, str], + body: str, +) -> SchemaSeedSpec: + metadata = { + "type": "schema", + "entity": entity, + "version": 1, + "schema": schema, + "settings": {"validation": "warn"}, + } + return SchemaSeedSpec( + title=title, + entity=entity, + content=f"# {title}\n\n{body}\n", + metadata=metadata, + ) diff --git a/src/basic_memory/cli/commands/__init__.py b/src/basic_memory/cli/commands/__init__.py index 5f80a149..fb0c187e 100644 --- a/src/basic_memory/cli/commands/__init__.py +++ b/src/basic_memory/cli/commands/__init__.py @@ -1,6 +1,6 @@ """CLI commands for basic-memory.""" -from . import status, db, doctor, import_memory_json, mcp, import_claude_conversations, orphans +from . import ci, status, db, doctor, import_memory_json, mcp, import_claude_conversations, orphans from . import ( import_claude_projects, import_chatgpt, @@ -13,6 +13,7 @@ from . import ( __all__ = [ "status", + "ci", "db", "doctor", "import_memory_json", diff --git a/src/basic_memory/cli/commands/ci.py b/src/basic_memory/cli/commands/ci.py new file mode 100644 index 00000000..f98b6919 --- /dev/null +++ b/src/basic_memory/cli/commands/ci.py @@ -0,0 +1,395 @@ +"""GitHub CI project update commands.""" + +from __future__ import annotations + +import json +import os +from pathlib import Path +from typing import Annotated, Any, Optional, cast + +import typer +from pydantic import ValidationError +from rich.console import Console + +from basic_memory.ci.project_updates import ( + DEFAULT_CONFIG_PATH, + DEFAULT_PROMPT_PATH, + DEFAULT_WORKFLOW_PATH, + AgentSynthesis, + ProjectUpdateConfig, + ProjectUpdateContext, + ProjectUpdateNote, + build_project_update_note, + collect_project_update_context, + detect_github_repo, + load_project_update_config, + render_agent_synthesis_schema, + render_capture_prompt, + render_workflow, + schema_seed_specs, + write_project_update_config, +) +from basic_memory.cli.app import app +from basic_memory.cli.commands.command_utils import run_with_cleanup +from basic_memory.cli.commands.routing import force_routing, validate_routing_flags +from basic_memory.mcp.tools import search_notes as mcp_search_notes +from basic_memory.mcp.tools import write_note as mcp_write_note + + +console = Console() +ci_app = typer.Typer(help="Capture GitHub delivery moments into Basic Memory") +app.add_typer(ci_app, name="ci") + + +@ci_app.command() +def setup( + project: Annotated[str, typer.Option(help="Basic Memory project for project updates")], + repo_root: Annotated[ + Path, + typer.Option("--repo-root", help="GitHub repository root to configure"), + ] = Path("."), + project_id: Annotated[ + Optional[str], + typer.Option("--project-id", help="Basic Memory project external_id"), + ] = None, + workspace: Annotated[ + Optional[str], + typer.Option("--workspace", help="Cloud workspace slug for generated config"), + ] = None, + deploy_workflow: Annotated[ + Optional[list[str]], + typer.Option("--deploy-workflow", help="Production deploy workflow name"), + ] = None, + environment: Annotated[ + Optional[list[str]], + typer.Option("--environment", help="Production environment name"), + ] = None, + force: bool = typer.Option(False, "--force", help="Overwrite generated Auto BM files"), + yes: bool = typer.Option(False, "--yes", help="Skip confirmation prompts"), + local: bool = typer.Option(False, "--local", help="Force local API routing for schema seeding"), + cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing for schema seeding"), +) -> None: + """Install the GitHub Actions workflow and seed project update schemas.""" + try: + validate_routing_flags(local, cloud) + repo_root = repo_root.resolve() + owner, repo = detect_github_repo(repo_root) + config = ProjectUpdateConfig( + project=project, + project_id=project_id, + workspace=workspace, + deploy_workflows=deploy_workflow or ["Deploy Production"], + production_environments=environment or ["production"], + ) + + if not yes: + confirmed = typer.confirm( + f"Install Auto BM for {owner}/{repo} and write updates to {project}?" + ) + if not confirmed: + raise typer.Exit(1) + + _write_generated_files(repo_root, config, force=force) + + with force_routing(local=local, cloud=cloud): + seeded = run_with_cleanup( + seed_project_update_schemas( + project=project, + project_id=project_id, + workspace=workspace, + ) + ) + + console.print("[green]Auto BM GitHub workflow installed[/green]") + console.print(f"Repository: {owner}/{repo}") + console.print(f"Project: {project}") + if seeded: + console.print(f"Seeded schemas: {', '.join(seeded)}") + else: + console.print("Schema notes already exist; nothing seeded") + console.print("\nAdd these GitHub secrets before enabling the workflow:") + console.print("- OPENAI_API_KEY") + console.print("- BASIC_MEMORY_API_KEY") + except ValueError as exc: + console.print(f"[red]{exc}[/red]") + raise typer.Exit(1) from exc + + +@ci_app.command() +def collect( + output: Annotated[ + Path, + typer.Option("--output", help="Where to write ProjectUpdateContext JSON"), + ], + config_path: Annotated[ + Path, + typer.Option("--config", help="Auto BM repository config path"), + ] = Path(DEFAULT_CONFIG_PATH), + event_name: Annotated[ + Optional[str], + typer.Option("--event-name", help="GitHub event name; defaults to GITHUB_EVENT_NAME"), + ] = None, + event_path: Annotated[ + Optional[Path], + typer.Option("--event-path", help="GitHub event payload; defaults to GITHUB_EVENT_PATH"), + ] = None, +) -> None: + """Normalize the current GitHub event into project update context JSON.""" + try: + effective_event_name = event_name or os.environ.get("GITHUB_EVENT_NAME") + if not effective_event_name: + raise ValueError("Missing event name. Pass --event-name or set GITHUB_EVENT_NAME.") + + event_path_value = event_path or ( + Path(os.environ["GITHUB_EVENT_PATH"]) if os.environ.get("GITHUB_EVENT_PATH") else None + ) + if event_path_value is None: + raise ValueError("Missing event payload. Pass --event-path or set GITHUB_EVENT_PATH.") + + config = load_project_update_config(config_path) + context = collect_project_update_context( + event_name=effective_event_name, + event_path=event_path_value, + config=config, + ) + output.parent.mkdir(parents=True, exist_ok=True) + output.write_text( + json.dumps(context.model_dump(mode="json"), indent=2, sort_keys=True) + "\n", + encoding="utf-8", + ) + _write_github_output("eligible", str(context.eligible).lower()) + _write_github_output("skip_reason", context.skip_reason or "") + console.print(f"Wrote project update context to {output}") + if not context.eligible: + console.print(f"Skipped: {context.skip_reason}") + except ValueError as exc: + console.print(f"[red]{exc}[/red]") + raise typer.Exit(1) from exc + + +@ci_app.command("agent-schema") +def agent_schema( + output: Annotated[ + Path, + typer.Option("--output", help="Where to write the temporary AgentSynthesis schema"), + ], +) -> None: + """Write the temporary Codex structured-output schema.""" + output.parent.mkdir(parents=True, exist_ok=True) + output.write_text(render_agent_synthesis_schema(), encoding="utf-8") + console.print(f"Wrote agent synthesis schema to {output}") + + +@ci_app.command() +def publish( + context_path: Annotated[ + Path, + typer.Option("--context", help="ProjectUpdateContext JSON from bm ci collect"), + ], + synthesis_path: Annotated[ + Path, + typer.Option("--synthesis", help="AgentSynthesis JSON produced by Codex"), + ], + config_path: Annotated[ + Path, + typer.Option("--config", help="Auto BM repository config path"), + ] = Path(DEFAULT_CONFIG_PATH), + local: bool = typer.Option( + False, "--local", help="Force local API routing (ignore cloud mode)" + ), + cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"), +) -> None: + """Publish an agent synthesis as an idempotent Basic Memory project update.""" + try: + validate_routing_flags(local, cloud) + config = load_project_update_config(config_path) + context = ProjectUpdateContext.model_validate(_read_json(context_path)) + if not context.eligible: + console.print(f"Auto BM skipped: {context.skip_reason}") + return + + synthesis = AgentSynthesis.model_validate(_read_json(synthesis_path)) + note = build_project_update_note(context=context, synthesis=synthesis, config=config) + + with force_routing(local=local, cloud=cloud): + result = run_with_cleanup( + publish_project_update_note(config=config, context=context, note=note) + ) + + console.print(json.dumps(result, indent=2, sort_keys=True, default=str)) + except (ValueError, ValidationError) as exc: + console.print(f"[red]{exc}[/red]") + raise typer.Exit(1) from exc + + +async def seed_project_update_schemas( + *, + project: str | None, + project_id: str | None = None, + workspace: str | None = None, +) -> list[str]: + """Seed Auto BM schema notes without overwriting customized schemas.""" + seeded: list[str] = [] + routed_project = _routed_project(project=project, project_id=project_id, workspace=workspace) + for spec in schema_seed_specs(): + existing = await mcp_search_notes( + query=None, + project=routed_project, + project_id=project_id, + metadata_filters={"type": "schema", "entity": spec.entity}, + output_format="json", + page_size=1, + ) + if _search_results(existing): + continue + + await mcp_write_note( + title=spec.title, + content=spec.content, + directory="schemas", + project=routed_project, + project_id=project_id, + note_type="schema", + metadata=spec.metadata, + overwrite=False, + output_format="json", + ) + seeded.append(spec.entity) + return seeded + + +async def publish_project_update_note( + *, + config: ProjectUpdateConfig, + context: ProjectUpdateContext, + note: ProjectUpdateNote, +) -> dict[str, Any]: + """Search by idempotency key and then upsert the deterministic note path.""" + routed_project = _routed_project( + project=config.project, + project_id=config.project_id, + workspace=config.workspace, + ) + existing = await mcp_search_notes( + query=None, + project=routed_project, + project_id=config.project_id, + metadata_filters={"type": "project_update", "idempotency_key": context.idempotency_key}, + output_format="json", + page_size=1, + ) + title, directory = _note_write_target( + existing, default_title=note.title, default_directory=note.directory + ) + result = await mcp_write_note( + title=title, + content=note.content, + directory=directory, + project=routed_project, + project_id=config.project_id, + tags=note.tags, + note_type="project_update", + metadata=note.metadata, + overwrite=True, + output_format="json", + ) + if isinstance(result, dict): + return result + return {"result": result} + + +def _note_write_target( + search_payload: object, + *, + default_title: str, + default_directory: str, +) -> tuple[str, str]: + """Use an existing idempotency match's path when one exists.""" + results = _search_results(search_payload) + if not results: + return default_title, default_directory + + match = results[0] + if not isinstance(match, dict): + return default_title, default_directory + + raw_title = match.get("title") + title = raw_title if isinstance(raw_title, str) else default_title + file_path = match.get("file_path") + if isinstance(file_path, str) and file_path.strip(): + parent = Path(file_path).parent.as_posix() + directory = "" if parent == "." else parent + return title, directory + + return title, default_directory + + +def _write_generated_files(repo_root: Path, config: ProjectUpdateConfig, *, force: bool) -> None: + files = { + repo_root / DEFAULT_WORKFLOW_PATH: render_workflow(config), + repo_root / DEFAULT_PROMPT_PATH: render_capture_prompt(), + } + config_path = repo_root / DEFAULT_CONFIG_PATH + _validate_generated_targets([*files, config_path], force=force) + for path, content in files.items(): + _write_generated_file(path, content, force=force) + write_project_update_config(config_path, config) + + +def _validate_generated_targets(paths: list[Path], *, force: bool) -> None: + if force: + return + for path in paths: + if path.exists(): + raise ValueError(f"{path} already exists; pass --force to overwrite") + + +def _write_generated_file(path: Path, content: str, *, force: bool) -> None: + if path.exists() and not force: + raise ValueError(f"{path} already exists; pass --force to overwrite") + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(content, encoding="utf-8") + + +def _read_json(path: Path) -> dict[str, Any]: + try: + payload = json.loads(path.read_text(encoding="utf-8")) + except FileNotFoundError as exc: + raise ValueError(f"JSON file not found: {path}") from exc + except json.JSONDecodeError as exc: + raise ValueError(f"Invalid JSON in {path}: {exc}") from exc + if not isinstance(payload, dict): + raise ValueError(f"{path} must contain a JSON object") + return payload + + +def _write_github_output(key: str, value: str) -> None: + output_path = os.environ.get("GITHUB_OUTPUT") + if not output_path: + return + with Path(output_path).open("a", encoding="utf-8") as handle: + handle.write(f"{key}={value}\n") + + +def _routed_project( + *, + project: str | None, + project_id: str | None, + workspace: str | None, +) -> str | None: + """Return a workspace-qualified project route when the config can enforce one.""" + if project_id or not project or not workspace or "/" in project: + return project + return f"{workspace.strip('/')}/{project}" + + +def _search_results(payload: object) -> list[Any]: + if isinstance(payload, dict): + payload_dict = cast(dict[str, Any], payload) + results = payload_dict.get("results") + if isinstance(results, list): + return results + nested = payload_dict.get("result") + if isinstance(nested, dict) and isinstance(nested.get("results"), list): + return nested["results"] + return [] diff --git a/src/basic_memory/cli/main.py b/src/basic_memory/cli/main.py index bd0d1cf9..de7e067d 100644 --- a/src/basic_memory/cli/main.py +++ b/src/basic_memory/cli/main.py @@ -16,6 +16,7 @@ def _version_only_invocation(argv: list[str]) -> bool: if not _version_only_invocation(sys.argv[1:]): # Register commands only when not short-circuiting for --version from basic_memory.cli.commands import ( # noqa: F401 # pragma: no cover + ci, cloud, db, doctor, diff --git a/tests/ci/test_project_updates.py b/tests/ci/test_project_updates.py new file mode 100644 index 00000000..dc407db6 --- /dev/null +++ b/tests/ci/test_project_updates.py @@ -0,0 +1,473 @@ +import json +from pathlib import Path + +import pytest +from pydantic import ValidationError + +from basic_memory.ci.project_updates import ( + AgentSynthesis, + ProjectUpdateConfig, + ProjectUpdateContext, + build_project_update_note, + collect_project_update_context, + detect_github_repo, + load_project_update_config, + parse_github_remote, + render_agent_synthesis_schema, + render_capture_prompt, + render_workflow, + schema_seed_specs, +) +from basic_memory.ci import project_updates + + +def _write_json(path: Path, payload: dict) -> Path: + path.write_text(json.dumps(payload), encoding="utf-8") + return path + + +def _pr_payload(*, merged: bool = True) -> dict: + return { + "action": "closed", + "repository": { + "full_name": "basicmachines-co/basic-memory", + "html_url": "https://github.com/basicmachines-co/basic-memory", + }, + "pull_request": { + "number": 123, + "title": "Remember project updates", + "body": "Adds Auto BM capture.\n\nCloses #77", + "html_url": "https://github.com/basicmachines-co/basic-memory/pull/123", + "merged": merged, + "merged_at": "2026-06-04T18:42:00Z" if merged else None, + "merge_commit_sha": "abc123", + "changed_files": 4, + "labels": [{"name": "feature"}, {"name": "ci"}], + "user": {"login": "octocat"}, + }, + } + + +def test_collect_merged_pull_request_context(tmp_path: Path) -> None: + event_path = _write_json(tmp_path / "event.json", _pr_payload()) + + context = collect_project_update_context( + event_name="pull_request", + event_path=event_path, + config=ProjectUpdateConfig(project="team-memory"), + ) + + assert context.eligible is True + assert context.source_event == "pull_request_merged" + assert context.repo == "basicmachines-co/basic-memory" + assert context.idempotency_key == "github:basicmachines-co/basic-memory:pull_request_merged:123" + assert context.pr_number == 123 + assert context.sha == "abc123" + assert context.labels == ["feature", "ci"] + assert context.linked_issues == ["#77"] + assert context.source_url == "https://github.com/basicmachines-co/basic-memory/pull/123" + + +def test_collect_handles_sparse_pull_request_payload(tmp_path: Path) -> None: + payload = { + "action": "closed", + "repository": {}, + "pull_request": { + "number": 123, + "merged": True, + "labels": "not-a-list", + }, + } + event_path = _write_json(tmp_path / "event.json", payload) + + context = collect_project_update_context( + event_name="pull_request", + event_path=event_path, + config=ProjectUpdateConfig(project="team-memory"), + ) + + assert context.eligible is True + assert context.repo is None + assert context.repo_url is None + assert context.labels == [] + assert context.linked_issues == [] + + +def test_collect_handles_missing_repository_payload(tmp_path: Path) -> None: + payload = { + "action": "closed", + "pull_request": { + "number": 123, + "merged": True, + }, + } + event_path = _write_json(tmp_path / "event.json", payload) + + context = collect_project_update_context( + event_name="pull_request", + event_path=event_path, + config=ProjectUpdateConfig(project="team-memory"), + ) + + assert context.eligible is True + assert context.repo is None + assert context.repo_url is None + + +def test_collect_rejects_missing_payload_shapes(tmp_path: Path) -> None: + pr_context = collect_project_update_context( + event_name="pull_request", + event_path=_write_json(tmp_path / "pr.json", {"action": "closed"}), + config=ProjectUpdateConfig(project="team-memory"), + ) + workflow_context = collect_project_update_context( + event_name="workflow_run", + event_path=_write_json(tmp_path / "workflow.json", {"action": "completed"}), + config=ProjectUpdateConfig(project="team-memory"), + ) + + assert pr_context.eligible is False + assert pr_context.skip_reason == "pull request payload missing" + assert workflow_context.eligible is False + assert workflow_context.skip_reason == "workflow run payload missing" + + +def test_collect_ignores_non_closed_pull_request_action(tmp_path: Path) -> None: + payload = _pr_payload() + payload["action"] = "opened" + event_path = _write_json(tmp_path / "event.json", payload) + + context = collect_project_update_context( + event_name="pull_request", + event_path=event_path, + config=ProjectUpdateConfig(project="team-memory"), + ) + + assert context.eligible is False + assert context.skip_reason == "pull request action was not closed" + + +def test_collect_ignores_closed_unmerged_pull_request(tmp_path: Path) -> None: + event_path = _write_json(tmp_path / "event.json", _pr_payload(merged=False)) + + context = collect_project_update_context( + event_name="pull_request", + event_path=event_path, + config=ProjectUpdateConfig(project="team-memory"), + ) + + assert context.eligible is False + assert context.skip_reason == "pull request was closed without merging" + + +def test_collect_successful_configured_production_deploy(tmp_path: Path) -> None: + payload = { + "action": "completed", + "repository": { + "full_name": "basicmachines-co/basic-memory-cloud", + "html_url": "https://github.com/basicmachines-co/basic-memory-cloud", + }, + "workflow_run": { + "id": 98765, + "name": "Deploy Production", + "conclusion": "success", + "html_url": "https://github.com/basicmachines-co/basic-memory-cloud/actions/runs/98765", + "head_sha": "def456", + "updated_at": "2026-06-04T19:10:00Z", + }, + } + event_path = _write_json(tmp_path / "event.json", payload) + + context = collect_project_update_context( + event_name="workflow_run", + event_path=event_path, + config=ProjectUpdateConfig( + project="cloud-memory", + deploy_workflows=["Deploy Production"], + production_environments=["production"], + ), + ) + + assert context.eligible is True + assert context.source_event == "production_deploy_succeeded" + assert context.workflow_run_id == "98765" + assert context.environment == "production" + assert context.idempotency_key == ( + "github:basicmachines-co/basic-memory-cloud:production_deploy_succeeded:production:98765" + ) + + +def test_collect_ignores_failed_or_unconfigured_deploy(tmp_path: Path) -> None: + payload = { + "action": "completed", + "repository": {"full_name": "basicmachines-co/basic-memory"}, + "workflow_run": {"id": 1, "name": "Tests", "conclusion": "failure"}, + } + event_path = _write_json(tmp_path / "event.json", payload) + + context = collect_project_update_context( + event_name="workflow_run", + event_path=event_path, + config=ProjectUpdateConfig(project="team-memory"), + ) + + assert context.eligible is False + assert context.skip_reason == "workflow conclusion was failure" + + +def test_collect_ignores_successful_unconfigured_deploy(tmp_path: Path) -> None: + payload = { + "action": "completed", + "repository": {"full_name": "basicmachines-co/basic-memory"}, + "workflow_run": {"id": 1, "name": "Tests", "conclusion": "success"}, + } + event_path = _write_json(tmp_path / "event.json", payload) + + context = collect_project_update_context( + event_name="workflow_run", + event_path=event_path, + config=ProjectUpdateConfig(project="team-memory"), + ) + + assert context.eligible is False + assert context.skip_reason == "workflow 'Tests' is not configured for project updates" + + +def test_collect_ignores_unsupported_event(tmp_path: Path) -> None: + event_path = _write_json(tmp_path / "event.json", {}) + + context = collect_project_update_context( + event_name="push", + event_path=event_path, + config=ProjectUpdateConfig(project="team-memory"), + ) + + assert context.eligible is False + assert context.skip_reason == "unsupported GitHub event: push" + + +def test_collect_rejects_missing_or_invalid_event_payload(tmp_path: Path) -> None: + with pytest.raises(ValueError, match="not found"): + collect_project_update_context( + event_name="pull_request", + event_path=tmp_path / "missing.json", + config=ProjectUpdateConfig(project="team-memory"), + ) + + invalid_json = tmp_path / "invalid.json" + invalid_json.write_text("{", encoding="utf-8") + with pytest.raises(ValueError, match="not valid JSON"): + collect_project_update_context( + event_name="pull_request", + event_path=invalid_json, + config=ProjectUpdateConfig(project="team-memory"), + ) + + list_json = tmp_path / "list.json" + list_json.write_text("[]", encoding="utf-8") + with pytest.raises(ValueError, match="JSON object"): + collect_project_update_context( + event_name="pull_request", + event_path=list_json, + config=ProjectUpdateConfig(project="team-memory"), + ) + + +def test_build_project_update_note_uses_deterministic_identity_fields(tmp_path: Path) -> None: + event_path = _write_json(tmp_path / "event.json", _pr_payload()) + context = collect_project_update_context( + event_name="pull_request", + event_path=event_path, + config=ProjectUpdateConfig(project="team-memory"), + ) + synthesis = AgentSynthesis.model_validate( + { + "summary": "Auto BM now records project updates.", + "why_it_matters": "Future agents can recover the delivery narrative.", + "repo": "evil/repo", + "source_event": "production_deploy_succeeded", + "verification": ["Unit tests cover event normalization."], + } + ) + + note = build_project_update_note(context=context, synthesis=synthesis) + + assert note.title == "PR #123: Remember project updates" + assert note.directory == "project-updates/github/basicmachines-co/basic-memory" + assert note.metadata["repo"] == "basicmachines-co/basic-memory" + assert note.metadata["source_event"] == "pull_request_merged" + assert note.metadata["idempotency_key"] == context.idempotency_key + assert "evil/repo" not in note.content + + +def test_build_project_update_note_for_production_deploy(tmp_path: Path) -> None: + payload = { + "action": "completed", + "repository": { + "full_name": "basicmachines-co/basic-memory-cloud", + "html_url": "https://github.com/basicmachines-co/basic-memory-cloud", + }, + "workflow_run": { + "id": 98765, + "name": "Deploy Production", + "conclusion": "success", + "html_url": "https://github.com/basicmachines-co/basic-memory-cloud/actions/runs/98765", + "head_sha": "def456", + "updated_at": "2026-06-04T19:10:00Z", + }, + } + context = collect_project_update_context( + event_name="workflow_run", + event_path=_write_json(tmp_path / "event.json", payload), + config=ProjectUpdateConfig( + project="cloud-memory", + deploy_workflows=["Deploy Production"], + production_environments=["production"], + ), + ) + synthesis = AgentSynthesis( + summary="Production deploy completed.", + why_it_matters="The latest project update reached users.", + ) + + note = build_project_update_note(context=context, synthesis=synthesis) + + assert note.title == "Production deploy: 2026-06-04" + assert note.metadata["workflow_run_id"] == "98765" + assert note.metadata["environment"] == "production" + assert "https://github.com/basicmachines-co/basic-memory-cloud/actions/runs/98765" in ( + note.content + ) + + +def test_build_project_update_note_rejects_invalid_context() -> None: + synthesis = AgentSynthesis( + summary="Auto BM records project updates.", + why_it_matters="Future agents can recover context.", + ) + with pytest.raises(ValueError, match="ineligible"): + build_project_update_note( + context=ProjectUpdateContext(eligible=False, skip_reason="not useful"), + synthesis=synthesis, + ) + + with pytest.raises(ValueError, match="deterministic identity"): + build_project_update_note( + context=ProjectUpdateContext( + eligible=True, + source_event="pull_request_merged", + repo="basicmachines-co/basic-memory", + ), + synthesis=synthesis, + ) + + +def test_agent_synthesis_requires_summary_and_why_it_matters() -> None: + with pytest.raises(ValidationError): + AgentSynthesis.model_validate({"summary": "Too thin"}) + + with pytest.raises(ValidationError): + AgentSynthesis.model_validate({"summary": " ", "why_it_matters": "Still too thin"}) + + +def test_project_update_config_requires_non_empty_lists() -> None: + with pytest.raises(ValueError, match="at least one"): + ProjectUpdateConfig(deploy_workflows=[" "]) + + +def test_render_workflow_invokes_codex_read_only_without_basic_memory_secret() -> None: + workflow = render_workflow( + ProjectUpdateConfig( + project="team-memory", + deploy_workflows=["Deploy Production"], + production_environments=["production"], + ) + ) + + assert "openai/codex-action@v1" in workflow + assert "sandbox: read-only" in workflow + assert "output-schema-file: ${{ runner.temp }}/agent-synthesis.schema.json" in workflow + assert "BASIC_MEMORY_CLOUD_API_KEY: ${{ secrets.BASIC_MEMORY_API_KEY }}" in workflow + assert "BASIC_MEMORY_CLOUD_HOST: ${{ vars.BASIC_MEMORY_CLOUD_HOST || '' }}" not in workflow + assert "BASIC_MEMORY_CI_CLOUD_HOST: ${{ vars.BASIC_MEMORY_CLOUD_HOST }}" in workflow + assert 'if [ -n "$BASIC_MEMORY_CI_CLOUD_HOST" ]' in workflow + assert "--context .github/basic-memory/project-update-context.json" in workflow + assert "--cloud \\" in workflow + codex_step = workflow.split("- name: Synthesize project update with Codex", 1)[1].split( + "- name: Publish project update", 1 + )[0] + assert "BASIC_MEMORY_API_KEY" not in codex_step + + +def test_render_capture_prompt_uses_workspace_context_path() -> None: + prompt = render_capture_prompt() + + assert ".github/basic-memory/project-update-context.json" in prompt + assert "${{ runner.temp }}" not in prompt + + +def test_render_agent_synthesis_schema_is_ci_guardrail_not_domain_schema() -> None: + schema = json.loads(render_agent_synthesis_schema()) + + assert schema["title"] == "AgentSynthesis" + assert "summary" in schema["required"] + assert "why_it_matters" in schema["required"] + assert "project_update" not in json.dumps(schema) + + +def test_schema_seed_specs_are_basic_memory_schema_notes() -> None: + specs = schema_seed_specs() + + assert {spec.entity for spec in specs} == { + "ProjectUpdate", + "GitHubPullRequestUpdate", + "GitHubProductionDeployUpdate", + } + assert all(spec.metadata["type"] == "schema" for spec in specs) + assert all(spec.metadata["settings"]["validation"] == "warn" for spec in specs) + + +def test_parse_github_remote_accepts_https_and_ssh() -> None: + assert parse_github_remote("https://github.com/basicmachines-co/basic-memory.git") == ( + "basicmachines-co", + "basic-memory", + ) + assert parse_github_remote("git@github.com:basicmachines-co/basic-memory.git") == ( + "basicmachines-co", + "basic-memory", + ) + + +def test_parse_github_remote_rejects_non_github_remote() -> None: + with pytest.raises(ValueError, match="GitHub remote"): + parse_github_remote("https://example.com/basicmachines-co/basic-memory.git") + + +def test_detect_github_repo_requires_origin_remote(tmp_path: Path) -> None: + with pytest.raises(ValueError, match="No remote.origin.url"): + detect_github_repo(tmp_path) + + +def test_load_project_update_config_handles_missing_and_invalid_yaml(tmp_path: Path) -> None: + assert load_project_update_config(tmp_path / "missing.yml") == ProjectUpdateConfig() + + invalid = tmp_path / "invalid.yml" + invalid.write_text("- not\n- an\n- object\n", encoding="utf-8") + with pytest.raises(ValueError, match="YAML object"): + load_project_update_config(invalid) + + +def test_private_note_helpers_reject_invalid_repo_shape() -> None: + context = ProjectUpdateContext(eligible=True, repo="not-owner-repo") + with pytest.raises(ValueError, match="owner/repo"): + project_updates._note_directory(context, ProjectUpdateConfig(project="team-memory")) + + missing_repo = ProjectUpdateContext(eligible=True) + with pytest.raises(ValueError, match="missing repo"): + project_updates._note_directory(missing_repo, ProjectUpdateConfig(project="team-memory")) + + +def test_private_note_title_uses_generic_fallback_for_unknown_event() -> None: + context = ProjectUpdateContext(eligible=True, source_event="unknown") + + assert project_updates._note_title(context) == "Project update" diff --git a/tests/cli/test_ci_commands.py b/tests/cli/test_ci_commands.py new file mode 100644 index 00000000..657a03b0 --- /dev/null +++ b/tests/cli/test_ci_commands.py @@ -0,0 +1,382 @@ +import json +import subprocess +from pathlib import Path +from unittest.mock import AsyncMock, patch + +from typer.testing import CliRunner + +from basic_memory.cli.main import app as cli_app + + +runner = CliRunner() + + +def _init_github_repo(path: Path) -> None: + subprocess.run(["git", "init"], cwd=path, check=True, capture_output=True) + subprocess.run( + ["git", "remote", "add", "origin", "https://github.com/basicmachines-co/demo.git"], + cwd=path, + check=True, + capture_output=True, + ) + + +def _write_pr_event(path: Path) -> Path: + payload = { + "action": "closed", + "repository": { + "full_name": "basicmachines-co/demo", + "html_url": "https://github.com/basicmachines-co/demo", + }, + "pull_request": { + "number": 7, + "title": "Add project update capture", + "body": "Closes #4", + "html_url": "https://github.com/basicmachines-co/demo/pull/7", + "merged": True, + "merged_at": "2026-06-04T18:42:00Z", + "merge_commit_sha": "abc123", + "labels": [{"name": "ci"}], + }, + } + path.write_text(json.dumps(payload), encoding="utf-8") + return path + + +@patch("basic_memory.cli.commands.ci.seed_project_update_schemas", new_callable=AsyncMock) +def test_setup_writes_workflow_config_and_prompt( + mock_seed: AsyncMock, + tmp_path: Path, +) -> None: + _init_github_repo(tmp_path) + + result = runner.invoke( + cli_app, + [ + "ci", + "setup", + "--project", + "team-memory", + "--repo-root", + str(tmp_path), + "--yes", + ], + ) + + assert result.exit_code == 0, result.output + assert (tmp_path / ".github/workflows/basic-memory.yml").exists() + assert (tmp_path / ".github/basic-memory/config.yml").exists() + assert (tmp_path / ".github/basic-memory/memory-ci-capture.md").exists() + assert "OPENAI_API_KEY" in result.output + assert "BASIC_MEMORY_API_KEY" in result.output + mock_seed.assert_awaited_once_with( + project="team-memory", + project_id=None, + workspace=None, + ) + + +@patch("basic_memory.cli.commands.ci.seed_project_update_schemas", new_callable=AsyncMock) +def test_setup_does_not_partially_write_generated_files_when_target_exists( + mock_seed: AsyncMock, + tmp_path: Path, +) -> None: + _init_github_repo(tmp_path) + config_path = tmp_path / ".github/basic-memory/config.yml" + config_path.parent.mkdir(parents=True) + config_path.write_text("project: existing\n", encoding="utf-8") + + result = runner.invoke( + cli_app, + [ + "ci", + "setup", + "--project", + "team-memory", + "--repo-root", + str(tmp_path), + "--yes", + ], + ) + + assert result.exit_code == 1 + assert "pass --force to overwrite" in result.output + assert not (tmp_path / ".github/workflows/basic-memory.yml").exists() + assert not (tmp_path / ".github/basic-memory/memory-ci-capture.md").exists() + mock_seed.assert_not_awaited() + + +def test_setup_rejects_non_github_repo(tmp_path: Path) -> None: + subprocess.run(["git", "init"], cwd=tmp_path, check=True, capture_output=True) + subprocess.run( + ["git", "remote", "add", "origin", "https://example.com/basicmachines-co/demo.git"], + cwd=tmp_path, + check=True, + capture_output=True, + ) + + result = runner.invoke( + cli_app, + [ + "ci", + "setup", + "--project", + "team-memory", + "--repo-root", + str(tmp_path), + "--yes", + ], + ) + + assert result.exit_code == 1 + assert "GitHub remote" in result.output + + +def test_collect_command_writes_context_and_github_outputs(tmp_path: Path) -> None: + event_path = _write_pr_event(tmp_path / "event.json") + config_path = tmp_path / "config.yml" + config_path.write_text("project: team-memory\nworkspace: product\n", encoding="utf-8") + output_path = tmp_path / "context.json" + github_output = tmp_path / "github-output.txt" + + result = runner.invoke( + cli_app, + [ + "ci", + "collect", + "--event-name", + "pull_request", + "--event-path", + str(event_path), + "--config", + str(config_path), + "--output", + str(output_path), + ], + env={"GITHUB_OUTPUT": str(github_output)}, + ) + + assert result.exit_code == 0, result.output + context = json.loads(output_path.read_text(encoding="utf-8")) + assert context["eligible"] is True + assert context["source_event"] == "pull_request_merged" + assert "eligible=true" in github_output.read_text(encoding="utf-8") + + +def test_agent_schema_command_writes_schema(tmp_path: Path) -> None: + output_path = tmp_path / "agent-synthesis.schema.json" + + result = runner.invoke(cli_app, ["ci", "agent-schema", "--output", str(output_path)]) + + assert result.exit_code == 0, result.output + schema = json.loads(output_path.read_text(encoding="utf-8")) + assert schema["title"] == "AgentSynthesis" + + +@patch("basic_memory.cli.commands.ci.mcp_search_notes", new_callable=AsyncMock) +@patch("basic_memory.cli.commands.ci.mcp_write_note", new_callable=AsyncMock) +def test_publish_command_upserts_project_update_note( + mock_write: AsyncMock, + mock_search: AsyncMock, + tmp_path: Path, +) -> None: + mock_search.return_value = {"results": []} + mock_write.return_value = { + "title": "PR #7: Add project update capture", + "permalink": "project-updates/github/basicmachines-co/demo/pr-7-add-project-update-capture", + "action": "created", + } + event_path = _write_pr_event(tmp_path / "event.json") + context_path = tmp_path / "context.json" + config_path = tmp_path / "config.yml" + synthesis_path = tmp_path / "synthesis.json" + config_path.write_text("project: team-memory\nworkspace: product\n", encoding="utf-8") + + collect_result = runner.invoke( + cli_app, + [ + "ci", + "collect", + "--event-name", + "pull_request", + "--event-path", + str(event_path), + "--config", + str(config_path), + "--output", + str(context_path), + ], + ) + assert collect_result.exit_code == 0, collect_result.output + synthesis_path.write_text( + json.dumps( + { + "summary": "Auto BM records project updates.", + "why_it_matters": "Future agents can recover project context.", + "repo": "evil/repo", + } + ), + encoding="utf-8", + ) + + result = runner.invoke( + cli_app, + [ + "ci", + "publish", + "--config", + str(config_path), + "--context", + str(context_path), + "--synthesis", + str(synthesis_path), + ], + ) + + assert result.exit_code == 0, result.output + mock_search.assert_awaited_once() + mock_write.assert_awaited_once() + assert mock_search.call_args.kwargs["project"] == "product/team-memory" + kwargs = mock_write.call_args.kwargs + assert kwargs["project"] == "product/team-memory" + assert kwargs["note_type"] == "project_update" + assert kwargs["overwrite"] is True + assert kwargs["metadata"]["repo"] == "basicmachines-co/demo" + assert kwargs["metadata"]["source_event"] == "pull_request_merged" + assert ( + kwargs["metadata"]["idempotency_key"] + == "github:basicmachines-co/demo:pull_request_merged:7" + ) + + +@patch("basic_memory.cli.commands.ci.mcp_search_notes", new_callable=AsyncMock) +@patch("basic_memory.cli.commands.ci.mcp_write_note", new_callable=AsyncMock) +def test_publish_command_preserves_existing_note_path_for_idempotency_match( + mock_write: AsyncMock, + mock_search: AsyncMock, + tmp_path: Path, +) -> None: + mock_search.return_value = { + "results": [ + { + "title": "Existing PR update", + "file_path": "custom/project-updates/existing-pr-update.md", + } + ] + } + mock_write.return_value = {"title": "Existing PR update", "action": "updated"} + event_path = _write_pr_event(tmp_path / "event.json") + context_path = tmp_path / "context.json" + config_path = tmp_path / "config.yml" + synthesis_path = tmp_path / "synthesis.json" + config_path.write_text("project: team-memory\n", encoding="utf-8") + + collect_result = runner.invoke( + cli_app, + [ + "ci", + "collect", + "--event-name", + "pull_request", + "--event-path", + str(event_path), + "--config", + str(config_path), + "--output", + str(context_path), + ], + ) + assert collect_result.exit_code == 0, collect_result.output + synthesis_path.write_text( + json.dumps( + { + "summary": "Auto BM records project updates.", + "why_it_matters": "Future agents can recover project context.", + } + ), + encoding="utf-8", + ) + + result = runner.invoke( + cli_app, + [ + "ci", + "publish", + "--config", + str(config_path), + "--context", + str(context_path), + "--synthesis", + str(synthesis_path), + ], + ) + + assert result.exit_code == 0, result.output + kwargs = mock_write.call_args.kwargs + assert kwargs["title"] == "Existing PR update" + assert kwargs["directory"] == "custom/project-updates" + + +@patch("basic_memory.cli.commands.ci.mcp_search_notes", new_callable=AsyncMock) +@patch("basic_memory.cli.commands.ci.mcp_write_note", new_callable=AsyncMock) +def test_publish_command_uses_project_id_without_workspace_qualifying_project( + mock_write: AsyncMock, + mock_search: AsyncMock, + tmp_path: Path, +) -> None: + mock_search.return_value = {"results": []} + mock_write.return_value = {"title": "Project update", "action": "created"} + event_path = _write_pr_event(tmp_path / "event.json") + context_path = tmp_path / "context.json" + config_path = tmp_path / "config.yml" + synthesis_path = tmp_path / "synthesis.json" + config_path.write_text( + "project: team-memory\nproject_id: project-uuid\nworkspace: product\n", + encoding="utf-8", + ) + + collect_result = runner.invoke( + cli_app, + [ + "ci", + "collect", + "--event-name", + "pull_request", + "--event-path", + str(event_path), + "--config", + str(config_path), + "--output", + str(context_path), + ], + ) + assert collect_result.exit_code == 0, collect_result.output + synthesis_path.write_text( + json.dumps( + { + "summary": "Auto BM records project updates.", + "why_it_matters": "Future agents can recover project context.", + } + ), + encoding="utf-8", + ) + + result = runner.invoke( + cli_app, + [ + "ci", + "publish", + "--config", + str(config_path), + "--context", + str(context_path), + "--synthesis", + str(synthesis_path), + ], + ) + + assert result.exit_code == 0, result.output + assert mock_search.call_args.kwargs["project"] == "team-memory" + assert mock_search.call_args.kwargs["project_id"] == "project-uuid" + assert mock_write.call_args.kwargs["project"] == "team-memory" + assert mock_write.call_args.kwargs["project_id"] == "project-uuid"