feat(plugins): manual-pages flow — manpage seed schema, flow docs, verification fixes (#971)

Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Paul Hernandez
2026-06-11 16:27:30 -05:00
committed by GitHub
parent 33e741fd29
commit a148e72f56
16 changed files with 568 additions and 16 deletions
+1 -1
View File
@@ -590,7 +590,7 @@ retention).
| `BASIC_MEMORY_IMPORT_UPLOAD_MAX_BYTES` | `104857600` | Max uploaded import size |
```bash
BASIC_MEMORY_LOG_LEVEL=DEBUG basic-memory sync
BASIC_MEMORY_LOG_LEVEL=DEBUG basic-memory reindex
tail -f ~/.basic-memory/basic-memory.log
```
+3 -3
View File
@@ -111,7 +111,7 @@ You can run Basic Memory CLI commands inside the container using `docker exec`:
docker exec basic-memory-server basic-memory status
# Sync files
docker exec basic-memory-server basic-memory sync
docker exec basic-memory-server basic-memory reindex
# Show help
docker exec basic-memory-server basic-memory --help
@@ -137,7 +137,7 @@ When using Docker volumes, you'll need to configure projects to point to your mo
3. **Sync the new project:**
```bash
docker exec basic-memory-server basic-memory sync
docker exec basic-memory-server basic-memory reindex
```
### Example: Setting up an Obsidian Vault
@@ -157,7 +157,7 @@ docker exec basic-memory-server basic-memory project create obsidian /app/data
docker exec basic-memory-server basic-memory project set-default obsidian
# Sync to index all files
docker exec basic-memory-server basic-memory sync
docker exec basic-memory-server basic-memory reindex
```
### Environment Variables
+2 -2
View File
@@ -184,8 +184,8 @@ finance/ (lowercase f)
Use Basic Memory's built-in conflict detection:
```bash
# Sync will report conflicts
basic-memory sync
# Index local file changes (conflicts are handled during the scan)
basic-memory reindex
# Check sync status for warnings
basic-memory status
+183
View File
@@ -0,0 +1,183 @@
# Manual Pages
Basic Memory's manual is written in the style of Unix man pages — and
implemented as Basic Memory notes ([#952](https://github.com/basicmachines-co/basic-memory/issues/952)).
Every page is a markdown note conforming to the `Manpage` schema, `SEE ALSO`
entries are real knowledge-graph relations, and every example on every page
was executed against a live project before the page shipped. The manual
documents the tools; the tools verify the manual.
## Where it lives
The canonical manual is the **`manual` project in the Basic Memory team
workspace** (cloud, shared). Anyone can build their own: the schema ships as
an opt-in seed at `plugins/claude-code/schemas/manpage.md` — copy it into any
project's folder and start writing pages against it.
Layout:
```
manual/
├── schemas/Manpage.md # the manpage schema (type: schema)
├── man1/ # CLI commands bm(1), bm-status(1), ...
├── man3/ # MCP tools write-note(3), search-notes(3), ...
├── man5/ # file formats bm-note(5), bm-observation(5), ...
├── man7/ # concepts basic-memory(7), semantic-memory(7), ...
├── playground/ # scratch notes for destructive examples
└── diagrams/ # canvas visualizations of the manual graph
```
### Why "man1", "man3", "man5"?
The folder names are Unix's, unchanged since 1971. The manual is divided
into numbered **sections**, pages physically live in directories named
after them (`/usr/share/man/man1`, `man5`, ...), and the number tells you
what *kind* of thing is documented — not importance, not reading order:
- **1** — user commands (`ls`, `grep`)
- **2** — system calls
- **3** — library functions / APIs (`printf(3)`)
- **4** — devices
- **5** — file formats and config files (`crontab(5)`, `passwd(5)`)
- **6** — games (really)
- **7** — miscellanea: concepts, conventions, overviews (`regex(7)`, `signal(7)`)
- **8** — system administration
That's also why man page names carry the parenthesized number —
`crontab(1)` is the command, `crontab(5)` is the file format, same name in
two sections. `man 5 crontab` picks the section explicitly.
This manual copies that layout with the sections that have a Basic Memory
analog:
- **man1/** — `bm` CLI commands → `bm-status(1)`
- **man3/** — MCP tools, our equivalent of the "library API" section → `write-note(3)`
- **man5/** — file formats: note syntax, observations, relations, schemas → `bm-note(5)`
- **man7/** — concepts → `basic-memory(7)`, `semantic-memory(7)`
- **8** is reserved for admin/cloud operations but has no pages yet; 2, 4,
and 6 have no analog (no system calls, no devices, and no games — yet)
When a page says `see_also [[bm-note(5)]]`, the `(5)` reads "the
file-format page," exactly the way a Unix manual cross-references — except
here it's a traversable relation in the graph instead of a typographic
convention. The manual explains its own conventions in `man-pages(7)`
fittingly, the same page name Linux uses for this, and that almost nobody
ever reads.
## Page anatomy
Pages use the classic headers where applicable: `NAME`, `SYNOPSIS`,
`DESCRIPTION`, `PARAMETERS`, `MCP USAGE`, `CLI EQUIVALENT`, `EXAMPLES`,
`GOTCHAS`, `SEE ALSO`. Frontmatter (validated by the schema):
```yaml
type: manpage
section: 3 # 1 | 3 | 5 | 7 | 8
name: write-note # page name without section suffix
summary: create or overwrite a markdown note in the knowledge base
generated: hand # hand | registry | typer (regeneration ownership)
tool: write_note # section-3 pages: the MCP tool documented
command: basic-memory status # section-1 pages: the CLI command documented
verified: 0.21.6 mcp+cli # version + path(s) that proved the page
```
Field knowledge accumulates as observations — `[gotcha]`, `[bug]` (with issue
links), `[pattern]` — and `SEE ALSO` entries are `see_also` relations, so the
manual is a navigable graph, not a folder of files.
## How to use it
Man-style reads (any MCP client or the CLI):
```bash
# read a page
bm tool read-note "man3/write-note-3" --project manual
# apropos — find pages by section, tool, or text
bm tool search-notes --project manual # then filter, or via MCP:
# search_notes(project="manual", metadata_filters={"type": "manpage", "section": 3})
# search_notes(project="manual", metadata_filters={"type": "manpage", "tool": "write_note"})
# traverse SEE ALSO from any page
# build_context(url="man3/write-note-3", project="manual")
```
A future `bm man <topic>` command is thin sugar over exactly these calls.
And for the real thing — `man bm` in an actual terminal:
```bash
bm man install # copies bundled groff pages to ~/.local/share/man
man bm # the overview page, rendered by man(1)
man basic-memory # same page via its alias
```
`bm man install` warns with a one-line `MANPATH` fix if the install root
isn't searched by your `man`. Agents with shell access can use `man bm` as
an offline quick reference; the full per-tool detail stays in the manual
project's section-3 pages.
## The verification discipline
Two rules make the manual trustworthy:
1. **Examples must have run.** An `EXAMPLES` (or `MCP USAGE` / `CLI
EQUIVALENT`) block contains only commands that actually executed against
the manual project. Destructive operations (`delete_note`, `move_note`,
destructive `edit_note`) run only against `playground/` notes — never
against pages. The `verified:` field records the version and which path
proved the page: `mcp` (live service), `cli` (dev checkout), or both.
2. **The schema is the linter.** Validate the whole manual any time:
```bash
bm tool schema-validate manpage --project manual
# → {"total_notes": 38, "valid_count": 38, "warning_count": 0, ...}
```
`bm orphans --project manual` confirms every page is connected to the
graph, and `schema_diff`/`schema_infer` report drift between the schema
and how pages are actually written.
Because verification exercises real tool calls against the live service,
building the manual doubles as an end-to-end smoke test. The initial build
found six bugs in one pass (#954#959) — including the verification rule
catching a test that asserted a bug as expected output (#958).
## Adding or updating a page
1. Run the commands you intend to document; keep the actual output.
2. Write the page with `write_note`, passing frontmatter through the
`metadata` parameter (nested YAML in content frontmatter is unreliable on
some clients):
```
write_note(title="my-tool(3)", directory="man3", project="manual",
note_type="manpage",
metadata={"section": 3, "name": "my-tool",
"summary": "...", "generated": "hand",
"tool": "my_tool", "verified": "<version> mcp"})
```
3. Link related pages in `SEE ALSO` with `see_also [[other-page(3)]]`.
Forward references to pages that don't exist yet are fine — they resolve
automatically when the target is written.
4. Validate: `bm tool schema-validate manpage --project manual`.
For mechanical updates to generated sections, prefer `edit_note` with
`replace_section` / `insert_after_section` so curated content (EXAMPLES,
GOTCHAS, SEE ALSO, observations) survives — that ownership split is what the
`generated:` field declares.
## Roadmap
- **Registry generator** — section-3 SYNOPSIS/PARAMETERS generated from the
MCP tool registry (docstrings + pydantic schemas), section-1 from Typer
help; the hand-written corpus is the template spec. Regenerate-and-diff in
CI becomes the drift gate.
- **`bm man <topic>`** — CLI sugar over `read_note` + metadata search.
(`bm man install` + a hand-written `bm.1` already ship — the first slice
of [#610](https://github.com/basicmachines-co/basic-memory/issues/610);
the generator will produce per-command pages from the same extraction.)
- **Docs site** — the notes remain canonical for sections 5 and 7, code is
canonical for 1 and 3; both render to the hosted docs site.
+55
View File
@@ -0,0 +1,55 @@
---
title: Manpage
type: schema
entity: Manpage
version: 1
schema:
gotcha?(array): string, sharp edges and surprising behavior learned from live verification
example?(array): string, worked examples beyond the generated synopsis
pattern?(array): string, recommended idioms and usage patterns
bug?(array): string, known defects affecting this surface, with issue links
see_also?(array): Entity, related manual pages — the SEE ALSO graph
settings:
validation: warn
frontmatter:
section(enum, Unix manual section number): [1, 3, 5, 7, 8]
name: string, page name without section suffix (e.g. write-note)
summary: string, one-line NAME description
generated?(enum, who owns the mechanical sections): [registry, typer, hand]
tool?: string, MCP tool this page documents (section 3 pages)
command?: string, CLI command this page documents (section 1 pages)
verified?: string, version and path that verified this page (e.g. 0.21.6 mcp+cli)
since?: string, version this surface first appeared
---
# Manpage
A **ManpageNote** is one page of a Unix-style manual implemented as Basic
Memory notes (issue #952): commands in section 1, MCP tools in section 3,
file formats in section 5, concepts in section 7, admin in section 8. The
manual becomes a knowledge graph — `SEE ALSO` entries are typed relations,
and pages are found by structured recall:
`search_notes(metadata_filters={"type": "manpage", "section": 3})`.
This schema is an opt-in seed for documentation projects; the canonical
manual lives in the Basic Memory team workspace `manual` project.
## What makes a good ManpageNote
- **NAME / SYNOPSIS / DESCRIPTION** — classic man-page structure, with
PARAMETERS, MCP USAGE, CLI EQUIVALENT, EXAMPLES, GOTCHAS, SEE ALSO where
applicable.
- **Verified examples** — EXAMPLES contain only commands that actually ran;
the `verified` field records the version and path (mcp, cli, or both).
- **generated** — declares regeneration ownership: `registry` (from the MCP
tool registry) and `typer` (from CLI help) pages get mechanical sections
rewritten; curated sections (EXAMPLES, GOTCHAS, SEE ALSO, observations)
are never overwritten.
- **gotcha / bug observations** — field knowledge accumulates on pages
without being clobbered by regeneration; bugs link their tracking issues.
## Frontmatter
`type: manpage` plus `section` makes the manual queryable like `man -k`:
by section, by `tool`, by `command`, or by missing/stale `verified` stamps.
Validation is `warn`, never blocking.
@@ -4,6 +4,7 @@ from . import ci, status, db, doctor, import_memory_json, mcp, import_claude_con
from . import (
import_claude_projects,
import_chatgpt,
man,
tool,
project,
format,
@@ -29,4 +30,5 @@ __all__ = [
"schema",
"update",
"workspace",
"man",
]
+76
View File
@@ -0,0 +1,76 @@
"""Install the bundled man pages so `man bm` works."""
import shutil
import subprocess
from pathlib import Path
from typing import Annotated, Optional
import typer
from rich.console import Console
from basic_memory.cli.app import app
console = Console()
man_app = typer.Typer(help="Manage the bm man pages.")
app.add_typer(man_app, name="man")
# Bundled groff sources ship inside the package (src/basic_memory/man).
_MAN_SOURCE_DIR = Path(__file__).parent.parent.parent / "man"
def _default_man_root() -> Path:
# Why ~/.local/share/man: manpath(1) derives man directories from PATH
# entries on both man-db (Linux) and BSD man (macOS), so ~/.local/bin on
# PATH — the pipx/uv tool layout — makes this root searchable without any
# MANPATH configuration.
return Path.home() / ".local" / "share" / "man"
def _man_root_on_manpath(man_root: Path) -> Optional[bool]:
"""Best-effort check whether man(1) will search man_root; None if unknown."""
try:
result = subprocess.run(["manpath"], capture_output=True, text=True, timeout=5)
except (FileNotFoundError, subprocess.TimeoutExpired):
return None
if result.returncode != 0:
return None
paths = [entry.rstrip("/") for entry in result.stdout.strip().split(":") if entry]
return str(man_root).rstrip("/") in paths
@man_app.command()
def install(
directory: Annotated[
Optional[Path],
typer.Option(
"--dir",
help="Man root to install into (default: ~/.local/share/man)",
),
] = None,
) -> None:
"""Install the bm man pages, then try `man bm`."""
man_root = (directory or _default_man_root()).expanduser()
man1 = man_root / "man1"
man1.mkdir(parents=True, exist_ok=True)
pages = sorted(_MAN_SOURCE_DIR.glob("*.1"))
if not pages: # pragma: no cover - broken packaging, not a runtime state
console.print("[red]No bundled man pages found — broken installation[/red]")
raise typer.Exit(1)
for page in pages:
shutil.copyfile(page, man1 / page.name)
console.print(f"installed {man1 / page.name}")
# Trigger: the chosen root is provably absent from manpath output.
# Why: a silent install into an unsearched directory looks like success
# but `man bm` still fails; say so and hand over the one-line fix.
# Outcome: actionable hint; unknown (None) stays quiet to avoid false alarms.
if _man_root_on_manpath(man_root) is False:
console.print(
f"\n[yellow]{man_root} is not on your manpath.[/yellow] Add it with:\n"
f' export MANPATH="{man_root}:$MANPATH"'
)
console.print("\nTry: [bold]man bm[/bold]")
+1
View File
@@ -24,6 +24,7 @@ if not _version_only_invocation(sys.argv[1:]):
import_claude_conversations,
import_claude_projects,
import_memory_json,
man,
mcp,
orphans,
project,
+1
View File
@@ -0,0 +1 @@
.so man1/bm.1
+134
View File
@@ -0,0 +1,134 @@
.TH BM 1 "2026-06-11" "basic-memory" "Basic Memory Manual"
.SH NAME
bm \- local-first knowledge base for humans and AI agents
.SH SYNOPSIS
.B bm
.I COMMAND
.RI [ ARGS ]...
.br
.B basic-memory
.I COMMAND
.RI [ ARGS ]...
.SH DESCRIPTION
.B bm
manages Basic Memory projects: plain markdown files that form a knowledge
graph. Files are the source of truth; SQLite provides indexing and
full-text search; the same operations are exposed to AI agents over the
Model Context Protocol (MCP) and to humans and scripts through this CLI.
.PP
Notes use semantic markdown: observations
.RB ( "\- [category] text #tag" )
and relations
.RB ( "\- relation_type [[Target]]" )
become queryable graph structure. Projects route independently to the
local API or to Basic Memory Cloud.
.SH COMMANDS
Knowledge operations:
.TP
.B bm tool
CLI access to the MCP tools (write-note, read-note, search-notes,
build-context, ...). These emit JSON and are the scriptable surface.
.TP
.B bm status
Show sync status between files and the database.
.TP
.B bm reindex
Index local file changes and rebuild search/embeddings. This is the manual
sync trigger when no MCP server is running.
.TP
.B bm doctor
Run end-to-end file/database consistency checks.
.TP
.B bm orphans
List entities with no relations in the knowledge graph.
.TP
.B bm format
Run configured formatters over note files.
.PP
Projects and schemas:
.TP
.B bm project
Add, remove, list projects; set the default; flip a project between local
and cloud routing.
.TP
.B bm schema
List, validate, infer, and drift-check Picoschema note-type contracts.
.PP
Data and cloud:
.TP
.B bm import
Import from ChatGPT, Claude, or memory.json exports. Imports write files;
run
.B bm reindex
afterwards.
.TP
.B bm cloud
Authenticate, sync (push/pull/sync/bisync), snapshots, and team workspace
administration.
.PP
Infrastructure:
.TP
.B bm mcp
Run the MCP server (hosts the live file watcher).
.TP
.B bm man
Manage these man pages
.RB ( "bm man install" ).
.TP
.B bm reset
Drop and recreate the database (destructive).
.PP
Most commands accept
.B \-\-project
.I NAME
to target a project and
.BR \-\-local / \-\-cloud
to override routing.
.SH EXAMPLES
Write and find a note from the shell:
.PP
.nf
.RS
echo "# Standup notes" | bm tool write-note \\
\-\-title "Standup" \-\-folder notes
bm tool search-notes "standup"
.RE
.fi
.PP
Pick up files created outside the tools:
.PP
.nf
.RS
bm status # shows pending changes
bm reindex # indexes them
.RE
.fi
.SH FILES
.TP
.I ~/.basic-memory/config.json
Projects, default project, per-project routing modes, cloud settings.
.TP
.I ~/.basic-memory/memory.db
SQLite index (derived; safe to rebuild with bm reindex).
.SH ENVIRONMENT
.TP
.B BASIC_MEMORY_FORCE_LOCAL
Force local routing regardless of cloud mode.
.TP
.B BASIC_MEMORY_LOG_LEVEL
Logging verbosity (e.g. DEBUG).
.SH SEE ALSO
Full manual (machine-readable, agent-traversable): the
.I manual
Basic Memory project \(em see docs/manual-pages.md in the repository.
Documentation: https://docs.basicmemory.com
.PP
For AI agents: the complete tool reference lives in the manual project as
section\-3 pages (write-note(3), search-notes(3), ...), queryable via
.B search_notes
with
.BR "metadata_filters={\(dqtype\(dq: \(dqmanpage\(dq}" .
.SH BUGS
https://github.com/basicmachines-co/basic-memory/issues
.SH AUTHORS
Basic Machines (https://basicmachines.co)
@@ -0,0 +1,17 @@
"""Shared loader for the bundled cloud-discovery markdown resources."""
from pathlib import Path
def load_discovery_resource(filename: str) -> str:
"""Read a bundled discovery markdown file with promo placeholders rendered.
The markdown carries a {{OSS_DISCOUNT_CODE}} placeholder so the promo code
has one source of truth (cli.promo); substitute before it reaches users.
"""
# Import here to avoid pulling CLI promo machinery (analytics, rich, config)
# into the MCP server import graph at module load.
from basic_memory.cli.promo import OSS_DISCOUNT_CODE
content = (Path(__file__).parent / filename).read_text(encoding="utf-8")
return content.replace("{{OSS_DISCOUNT_CODE}}", OSS_DISCOUNT_CODE)
+2 -4
View File
@@ -1,7 +1,6 @@
"""Cloud information MCP tool."""
from pathlib import Path
from basic_memory.mcp.resources.discovery import load_discovery_resource
from basic_memory.mcp.server import mcp
@@ -13,5 +12,4 @@ from basic_memory.mcp.server import mcp
)
def cloud_info() -> str:
"""Return optional Basic Memory Cloud information and setup guidance."""
content_path = Path(__file__).parent.parent / "resources" / "cloud_info.md"
return content_path.read_text(encoding="utf-8")
return load_discovery_resource("cloud_info.md")
+2 -4
View File
@@ -1,7 +1,6 @@
"""Release notes MCP tool."""
from pathlib import Path
from basic_memory.mcp.resources.discovery import load_discovery_resource
from basic_memory.mcp.server import mcp
@@ -13,5 +12,4 @@ from basic_memory.mcp.server import mcp
)
def release_notes() -> str:
"""Return the latest product release notes for optional user review."""
content_path = Path(__file__).parent.parent / "resources" / "release_notes.md"
return content_path.read_text(encoding="utf-8")
return load_discovery_resource("release_notes.md")
+5
View File
@@ -828,6 +828,11 @@ async def search_notes(
Formatted markdown text (output_format="text"), dict (output_format="json"),
or helpful error guidance string if search fails
Pagination note: `total` is exact only for text/title/permalink searches.
Vector and hybrid searches skip the count query (it would cost a second
semantic retrieval pass) and report `total: 0` even when results are
returned — use `has_more` for pagination in those modes.
Examples:
# Basic text search
results = await search_notes("project planning")
+77
View File
@@ -0,0 +1,77 @@
"""Tests for `bm man install` (#952 / #610: make `man bm` work)."""
import subprocess
from typer.testing import CliRunner
from basic_memory.cli.app import app
# Importing the module registers the man command group on the top-level app.
import basic_memory.cli.commands.man as man_command # noqa: F401
runner = CliRunner()
def _flattened(output: str) -> str:
# rich wraps console output at the terminal width, which differs between
# local shells and CI — collapse all whitespace so phrase assertions can't
# be split by a line break.
return " ".join(output.split())
def test_man_install_writes_pages_to_target(tmp_path):
"""Install copies every bundled page into <root>/man1 as valid groff."""
result = runner.invoke(app, ["man", "install", "--dir", str(tmp_path)])
assert result.exit_code == 0, result.output
bm_page = tmp_path / "man1" / "bm.1"
alias_page = tmp_path / "man1" / "basic-memory.1"
assert bm_page.exists()
assert alias_page.exists()
# groff sanity: a real title header and the alias .so include
assert bm_page.read_text().startswith(".TH BM 1")
assert alias_page.read_text().strip() == ".so man1/bm.1"
assert "Try:" in _flattened(result.output)
def test_man_install_warns_when_root_not_on_manpath(tmp_path, monkeypatch):
"""A root provably absent from manpath output gets the MANPATH hint."""
def fake_run(*args, **kwargs):
return subprocess.CompletedProcess(
args=["manpath"], returncode=0, stdout="/usr/share/man:/opt/man", stderr=""
)
monkeypatch.setattr(man_command.subprocess, "run", fake_run)
result = runner.invoke(app, ["man", "install", "--dir", str(tmp_path)])
assert result.exit_code == 0, result.output
assert "not on your manpath" in _flattened(result.output)
assert "MANPATH" in _flattened(result.output)
def test_man_install_stays_quiet_when_manpath_unavailable(tmp_path, monkeypatch):
"""No manpath binary → no false-alarm warning, install still succeeds."""
def fake_run(*args, **kwargs):
raise FileNotFoundError("manpath")
monkeypatch.setattr(man_command.subprocess, "run", fake_run)
result = runner.invoke(app, ["man", "install", "--dir", str(tmp_path)])
assert result.exit_code == 0, result.output
assert "not on your manpath" not in _flattened(result.output)
def test_man_install_treats_manpath_failure_as_unknown(tmp_path, monkeypatch):
"""manpath exiting non-zero → unknown, no warning."""
def fake_run(*args, **kwargs):
return subprocess.CompletedProcess(args=["manpath"], returncode=1, stdout="", stderr="boom")
monkeypatch.setattr(man_command.subprocess, "run", fake_run)
result = runner.invoke(app, ["man", "install", "--dir", str(tmp_path)])
assert result.exit_code == 0, result.output
assert "not on your manpath" not in _flattened(result.output)
+7 -2
View File
@@ -1,5 +1,6 @@
"""Tests for cloud discovery MCP tools."""
from basic_memory.cli.promo import OSS_DISCOUNT_CODE
from basic_memory.mcp.tools import cloud_info, release_notes
@@ -7,8 +8,10 @@ def test_cloud_info_tool_returns_expected_copy():
result = cloud_info()
assert "# Basic Memory Cloud (optional)" in result
assert "{{OSS_DISCOUNT_CODE}}" in result
assert OSS_DISCOUNT_CODE in result
assert "bm cloud login" in result
# Regression (#958): the template placeholder must never reach users.
assert "{{" not in result
def test_release_notes_tool_returns_expected_copy():
@@ -16,5 +19,7 @@ def test_release_notes_tool_returns_expected_copy():
assert "# Release Notes" in result
assert "2026-02-06" in result
assert "{{OSS_DISCOUNT_CODE}}" in result
assert OSS_DISCOUNT_CODE in result
assert "bm cloud login" in result
# Regression (#958): the template placeholder must never reach users.
assert "{{" not in result